Office

365 for IT Professionals (2019 Edition)

Published by Tony Redmond

© Copyright 2015-2018 by Tony Redmond.

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means without the
written permission of the authors.

The example companies, organizations, products, domain names, email addresses, logos, people, places and event
depicted herein are fictitious. No association with any real company, organization, people, domain name, email
address, logo, person, place, or event is intended or should be inferred. The book expresses the views and opinions of
the authors. The information presented in the book is provided without any express, statutory, or implied warranties.
The authors cannot be held liable for any damages caused or alleged to be caused either directly or indirectly by this
book.

Although the authors are members of Microsoft’s Most Valuable Professional (MVP) program, the content of this book
solely represents their views and opinions about Office 365 and any other technologies mentioned in the text and is
not endorsed in any way by Microsoft Corporation.

Please be respectful of the rights of the authors and do not make copies of this eBook available to others.

Fifth (2019) edition. Previous editions:

Office 365 for Exchange Professionals (May 2015 and September 2015).
Office 365 for IT Pros (3 rd edition – June 2016).]
Office 365 for IT Pros (4 th edition – June 2017).

The authors issue regular updates for this eBook until they publish a new edition, usually after a year. People who buy
the book through gumroad.com can use their accounts to download updates as they become available. Updates are
also available for the Kindle edition, but this depends on the willingness of Amazon to make purchasers aware that an
update exists. We try to persuade Amazon of this need about once a month.

The Kindle version of the companion volume for this book is available separately. This book contains valuable
information about Office 365, but it’s information that we don’t update as often.

This is update 9 published on 14 December 2018. You can find information about the changes made in each update
in our change log or through our Facebook page .

The photo used on the front cover is of a Galapagos land iguana taken on South Plaza island by Tony Redmond in
February 2018.

Foreword

During my time at Microsoft I've worked on productivity clients, servers, and services with
some of the most dedicated professionals in the industry – not only my colleagues here at Microsoft but also the larger
community of our customers, partners, and MVPs. We have been unified in the pursuit of our mission to use IT to
enable people and organizations everywhere to achieve more. Over the last 10 years, we have been on the journey to
deliver these capabilities through Office 365.

The role of IT has transformed. With cloud services and, especially, with Office 365, your role in IT can deliver more
critical and time-saving capabilities to end users, as well as to your peers in security, compliance, and developer roles.
Office 365 sits at the forefront of this, with the latest teamwork, productivity, and security controls that leverage the
intelligence of the cloud and AI to help instrument innovation, like real-time threat protection or tooling that makes
communicating ideas fast and effective. IT plays a critical role in how the resulting controls and configurations like
these are rolled out and delivered to end users. These controls are often incredibly detailed and customized to meet
the specific requirements of each organization, so each decision and policy requires thorough consideration and
domain expertise.

Providing feedback on these controls and capabilities is one of the most important roles the Microsoft MVP
community plays. Our Microsoft MVPs are individuals who have mastered our technology. Beyond technical depth and
expertise, they are a trusted connection point and a voice that carries with it untold amounts of customer
conversations, whiteboarding sessions, and detailed explanations. Leaders in the community like Tony Redmond, Paul
Robichaux, Brian Reid, Ståle Hansen, Jussi Roine, Juan Carlos González Martín , Gustavo Valez, Vasil Michev, and
others provide an unbiased and often critical feedback loop to help ensure Microsoft delivers the best experience for
IT. This goes beyond simply instrumenting new capabilities. Instead, they help us be more customer-centric, thinking
through each capability to better understand how new controls are experienced by users and how they land in the
admin centers of Office 365, in Azure, or in our APIs and scripted PowerShell implementations.

The team who created this book is an important part of our engineering and quality process. The content within these
pages has been curated by technology experts who have been partners with Microsoft on our Office 365 engineering
efforts. This book embodies the outside voice and an independent view that comes from years of real world, IT-driven
experiences. It is also a great example of the spirit of Office 365 – a team from a diverse set of backgrounds,
experiences, and geographical locations creating a living document to share their knowledge.

Rajesh Jha

Corporate Vice President, Office 365, Microsoft Corporation

Introduction
Welcome to the Fifth edition of “Office 365 for IT Professionals,” a guide to Office 365, Microsoft’s cloud office
applications platform for those who are already experienced administrators of on-premises deployments.

This eBook contains information to help administrators understand and exploit Office 365. This is not an official
Microsoft publication and none of the opinions expressed here are endorsed by Microsoft in any way. Instead, it’s a
collection of thoughts, ideas, and perspectives from a team of highly experienced Office 365 MVPs (members of the
Microsoft Most Valuable Professional program).

The book was originally titled “Office 365 for Exchange Professionals” and we published two editions under that title
(which accounts for some of the examples that you’ll find in the text). Although Exchange was the first application
many organizations moved to the cloud, it is obvious that anyone who approaches Office 365 from the perspective of a
single application and remains focused on that application runs a large risk of losing much of the advantage from their
investment. With that thought in mind, there’s lots to talk about concerning SharePoint Online, OneDrive for Business,
Stream, Teams, Planner, Microsoft Planner, Delve, MyAnalytics, and Yammer, all of which are covered here.

Organization of this book

The intent of this book is to help those who run Office 365 tenants take maximum advantage of the platform. The
topics include:

Introducing Office 365.

Why making the decision to move into the cloud is not such a big deal after all!
How the basic workloads run and the differences between the on-premises products and their cloud
counterparts.
How Office 365 Identities work including directory synchronization and single sign-on.
How to manage Office 365.
How to manage Exchange Online mailboxes and other mail-enabled recipients.
How Office 365 Groups work.
How Microsoft Planner and Microsoft Teams build on top of Groups to enable task-based planning and team-
based collaboration.
How messages flow within Office 365 and how to protect users against spam and other mail-transmitted nasties.
How to manage the various clients that can connect to Office 365.
How to use the Office 365 Data Governance framework to keep important data or remove data that you no longer
need using Office 365 labels and retention policies.
How eDiscovery works across Office 365 with content searches and eDiscovery cases.
How the Office 365 Security and Compliance Center is the focus for cross-service compliance and security
initiatives – and a new data governance framework for Office 365.
How to protect sensitive data with rights management and encryption.
How Data Loss Prevention works in both the new (Office 365) and older (Exchange Transport Rules) varieties.
How Delve helps users to find information using content indexes and the Microsoft Graph.
How the changeover from the Skype for Business Online client to Teams will happen.

It's a lot of information to cover in a single book. We hope that you'll like what you find here and appreciate the effort
we go through to keep the information up to date.
Companion Book
As we built this book, we removed some chapters and other material that appeared in previous versions and moved it
into a companion volume. The intention was to make space for new topics and content that we think is important to
Office 365 tenants today.

We don’t intend to update the companion volume as often as we will the main book. We still think that the information
in the companion volume is interesting and valuable, but its time in the limelight might have passed or the
information is less relevant to Office 365 operations and administration today.

Products
This book is about Office 365. It therefore follows that when we reference “Exchange” or “SharePoint,” we mean the
cloud version rather than the on-premises code. We have tried to be as explicit as possible. For instance, we say
“Exchange Online” when we mean that a feature belongs to the cloud version. If we say “Exchange” or “SharePoint,”
it means that the discussion applies to both cloud and on-premises versions.

Highlighting the important stuff

From time to time we will want to draw your attention to something that we think is important. Everyone loves
PowerShell, so we have included over nine hundred PowerShell commands and other snippets that are shown inside
boxes to highlight that this is real-life code that you can reuse in your tenant. An example is:

[PS] C:\> Get-Mailbox | Get-MailboxStatistics

We use three colors to highlight information. The first type is a note.

Note : This is some additional information about a topic that we're discussing. We've included it because we think it
adds some value.

We also have some warnings for things that you need to understand.

Warning: Warnings or other cautionary notes will appear like this. Try not to ignore these, the lessons were often
learned the hard way and we’d hate to see you suffer the same pain.

And there are lots of real-world observations that we think will interest you.

Office 365 Groups : Every group is represented as a group object in Azure Active Directory…

Book Updates
Reflecting the way that people consume information today, we designed this book to be read in multiple ways. The
content is available in PDF (PC), and EPUB (for iPad and other eReaders), and Amazon Kindle. We have learned a lot
about the formatting requirements for these platforms since we began to put text together in early 2015. Apart from
its multi-platform nature, we also wanted to avoid the static nature that content often takes in traditional technical
books. Given that Office 365 changes all the time, it didn’t seem to make much sense to say that any version of text
was definitive and that led us to the decision to create a book with ever-changing content.

Due to the fast-changing nature of Office 365, some of the user interface elements illustrated in this book might have
changed when you read this book. The same is true for details of how a feature works. Our tenants are all registered
for the "Targeted Release" option to allow us to see new features before Microsoft releases them to the general Office
365 base. It is possible that we have covered something here that you do not see yet in your tenant. Coping with a
fast-changing (or even ever-changing) environment is just one of the challenges faced by Office 365 administrators.
New features appear, options move around, and options function in a slightly different manner. Things are just very
different to the somewhat staid situation that often pertains in a typical on-premises infrastructure.

Microsoft’s documentation for Office 365 improves over time but still often fails to keep up to date with the rapid
change within Office 365. We do our best to keep an eye on what is happening and what changes, but we can be
forgiven if we overlook some detail that Microsoft has recently revealed or updated. Think of this as an opportunity to
demonstrate how good a detective you are in seeking the right answer based on the evidence presented in this book,
in Microsoft’s web sites, and in the voluminous amount of text that you will find in blogs scattered around the web. Of
course, blog authors are not seers either and their text begins to degrade as soon as it appears, so you must gather
evidence and put it in context with what you see in Office 365 at the time when you’re trying to solve a problem or get
something to work as you believe it should. Welcome to the world of cloud software!
To keep matters as up-to-date as we can, we release updated versions of this book on a monthly basis. If we find
glaring errors, we will update the version that is currently online. It’s just like fixing bugs in a software program.

The Author Team

Tony Redmond

( Tony.Redmond@Office365ITPros.com )

Over thirty years ago, I began my writing career by creating the “ALL-IN-1 Handbook.” This book contained many tips
about working with Digital Equipment Corporation’s popular Office Automation product (including email and
calendar) and extended to about 120 pages, all printed off on a “letter quality” daisywheel LQP02 printer. Fourteen
books later, ten of which cover the evolution of Exchange Server from V4.0 (1996) to Exchange 2013 (2012) allowed
me to experience the transition of the creation of technical books from a pretty manual process involving correcting
paper galley drafts to the point we are at now. Today’s rapid development and release cadence makes it terrifically
difficult for authors and publishers alike to bring out satisfactory printed books. The text begins to age as soon as it is
finished and is out of date when the book appears. Although the general description and commentary around a
technology remains pertinent, the devil is in the detail and detail is everything for the reader seeking to understand
and master a product. In fact, trying to write a book that matches the change in cloud systems is much like writing
software: you’re continually testing, finding bugs, and fixing problems.

One reason why we choose eBook publishing techniques is to keep the content accurate and useful. Unlike traditional
publishing techniques, we issue new releases of this book as required (and have done so weekly in the past). Given the
nature of cloud software and our need to learn, understand, and master something before we can explain it well, there
is always the potential that we do not get something quite right. But we will do our level best to update the content as
soon as possible as new information and facts come to our attention.

The “Office 365 for IT Pros” project has been enjoyable and productive. I have learned many new things about the
many moving parts that connect the various bits of the service. Hopefully you will enjoy the text too. I am responsible
for a lot of the content and for bringing the book together into what I hope is a readable shape. I have also had the
privilege of reading and commenting on the work of my co-authors. I learned a lot from what they wrote about their
topics. I am not so sure that they have learned anything from my ramblings. Follow Tony @12Knocksinna or read his
Thoughts of an Idle Mind blog.

Juan Carlos González

Juan Carlos is a Telecommunications Engineer with more than 14 years of experience in diverse Microsoft products
and technologies such as SQL Server, Visual Studio, and the .NET Framework. He currently works as a Modern
Workplace Team Leader focused on the SharePoint and Office 365 platforms. Juan Carlos is an Office Servers &
Services MVP and co-founder and coordinator of the SharePoint Users Group of Spain (SUGES), the Cloud Computing
Users Group (CLOUDES), and the Comunidad de Office 365. He is also co-editor and co-director of the free Spanish
digital magazine about SharePoint: CompartiMOSS . To date, Juan Carlos has co-authored 11 books and several
articles about SharePoint and Office 365 platforms in Spanish and English.

Ståle Hansen
 Stale.Hansen@Office365ITPros.com

Ståle Hansen is the Founder and Principal Cloud Architect at CloudWay in Norway. He is an Office Servers and
Services MVP with over a decade of experience with Skype for Business and are now focused on hybrid Office 365.
His core competence is to explain complex scenarios and make them understandable by combining technical insight
and business value. He loves to share his understanding as a blogger since 2009 and as a highly rated trainer with the
long running master class covering Lync 2010 to Skype for Business administration. You can follow his YouTube
channel on OneNote LifeHacks. He is regularly invited to speak at international conferences such as TechEd,
Microsoft Ignite, NIC and IT/Dev Connections. Follow Ståle on Twitter , and read his msunified.net blog.

Brian Reid
brian@nbconsult.co

Brian Reid is a consultant, instructor, and author specializing in Office 365, Azure, EM+S and
Microsoft Exchange Server and the UK director of Microsoft Gold Partner, NB Consult. With over 25 years of
experience in Microsoft and related technologies, Brian has helped many customers to design and integrate Microsoft
Exchange Server, Active Directory and Office 365. Brian specializes in helping his clients migrate to, and get the best
from, Office 365, Azure and the Enterprise Mobility + Security suite.

Brian also has a number of on-premises clients, and he designs and builds reliable on-premises solutions for Exchange
Server, including the migration from earlier versions of Exchange (and other email system) to the latest version of
Exchange or to Office 365. He is a Microsoft Certified Master and Microsoft Certified Solutions Master (MCM) in
Exchange Server, one of a handful people worldwide who are both MCM certified and an MVP. Brian served as an
instructor for the MCM program, specializing in email transport. He lives in the UK and is married with three
children.

Paul Robichaux
paul@robichaux.net

Paul Robichaux, an Office Servers and Services MVP since 2002, is currently the chief technology
officer at Quadrotech Solutions, where he leads the product development team for Quadrotech's family of Office 365
migration, automation, reporting, and security products. Paul's unique background includes stints writing Space
Shuttle payload software in FORTRAN, developing cryptographic software for the US National Security Agency,
helping giant companies deploy Office 365 to their worldwide users, and writing about and presenting on Microsoft’s
software and server products. Paul’s an avid (but slow) triathlete, an instrument-rated private pilot, and an occasional
blogger (at http://www.paulrobichaux.com ) and Tweeter (@paulrobichaux ).

Jussi Roine
 Jussi Roine, Chief Research Officer at Sulava (Finland), is a Microsoft MVP, Regional Lead, and
Microsoft Certified Master and says that he is even a nice guy to work with too. With over 25 years’ experience on the
Microsoft platform, Jussi now focuses on Microsoft Azure and Office 365, as well as traditional on-premises
deployments and architectures. He has worked with SharePoint and Office 365 for the past 10+ years, but recently
shifted his focus slightly to hybrid architectures and cloud-based approaches. Jussi can communicate and create value
at both technologist and executive level and is one of the few people to have attained the Microsoft Certified Master
(SharePoint) accreditation. When not at a customer site or in a classroom, Jussi is rock climbing, running or hunting
great Italian wines. Contact Jussi on Twitter or at Jussi.Roine@Sulava.com .

Gustavo Vélez
gustavo@gavd.net

Gustavo Vélez is a senior solutions architect specialized in the integration of Microsoft software. Over
many years of experience developing and working with Windows and Office applications, Gustavo has given
seminars/training in SharePoint as well as private consultancy work. His articles can be found in many of the leading
trade magazines in English, Dutch, German and Spanish. He is webmaster of the principal Spanish-language site
dedicated to SharePoint . Gustavo is author of eight books about SharePoint and founder and editor of CompartiMOSS
, the magazine about Microsoft technologies for the Spanish-speaking community.

Vasil Michev (Technical Editor)

( Vasil.Michev@Office365ITPros.com )

Vasil Michev is an Office Servers and Services MVP, specializing in Office 365. He has closely
followed the evolution of Microsoft’s Productivity Cloud offerings since the very beginnings with BPOS. With a career
that has spanned the industry, from Frontline Engineer to Consultant, Vasil has a unique, and wide-reaching
experience, encompassing all stages of the Office 365 adoption lifecycle. In his spare time, he enjoys getting involved
in various Office 365 communities, helping like-minded people, and updating his blog at michev.info .

Previous Contributors
We would be remiss if we did not acknowledge the contributions of previous authors to the success of this book. Paul
Cunningham and Michael Van Horenbeeck were authors for the first four editions and headed coverage in areas like
migration, management, mobile device management, and hybrid organizations. Jeff Guillet was the technical editor for
the first two editions and helped us to pick up errors that snuck into text. Without their efforts, this book would not be
as comprehensive as it is today.

Microsoft MVP Program

Paul, Tony, Michael, Ståle, Juan Carlos, Brian, Jussi, Gustavo, and Vasil are proud members of Microsoft's Most
Valuable Professional (MVP) program. See this page for more information about the MVP Program.
Thanks
Many people at Microsoft helped us to chase down technical detail or debate the finer points of how things worked.
Over the book’s history, we have received help from Jon Orton, Steve Conn, Brian Shiers, Sanjay Ramaswamy, Tariq
Sharif, Julian Zbogar-Smith, Amit Gupta, Christophe Fiessinger, Quentin Christensen, Ian Finder, Karim Batthish,
Wesley Holley, Cem Aykan, Timothy Heeney, Terry Zink, Tim McMichael, Monica Iacob, Eric Zenz, Jennifer Gagnon,
Kairaz Contractor, Noelle Beaujon, Rob Whaley, Ryan Fuller, Warren Johnson, Peter McAbee, Wendy Guo, Jack Kabat,
Amir Haque, Nino Bilic, Julia Foran, Vivek Sharma, Den Delimarschi, Shilpi Sinha, Kumar Venkateswar, Shashi
Singaravel, Jim Edelen, Sunitha Samuel, Ray Yang, Heath Hudgins, Wesley Holley, Rachi Messing, Dheepak
Ramaswamy, Ian Liu, Dan Stevenson, Ansuman Acharya, Jeff Kizner, Asaf Kashi, Vijay Nelson, Adrienne Trudeau,
Nishan DeSilva, Nick Robinson, Bhavanesh Rengarajan, Alfons Staerk, Greg Taylor, Scott Landry, and Maithili
Dandige. Our thanks to you all.

Sponsor content

It’s hard to find the time to gather information, make sure that it’s current, and write it up. An enormous amount of
effort has gone into the creation of the original book and the many revisions and rewrites required for this edition. We
could not undertake the task without the help and support of our sponsor, Quadrotech. We are very grateful for the
support extended by Quadrotech.

Chapter 26 is written by Paul Robichaux, Chief Technology Officer at Quadrotech, who also contributes four other
chapters to the book. Paul leads the product development team for Quadrotech's family of Office 365 migration,
automation, reporting, and security products. Paul's unique background includes stints writing Space Shuttle payload
software in FORTRAN, developing cryptographic software for the US National Security Agency, helping giant
companies deploy Office 365 to their worldwide users, and writing about and presenting on Microsoft’s software and
server products. Paul’s an avid (but slow) triathlete, an instrument-rated private pilot, and an occasional blogger (at
http://www.paulrobichaux.com) and Tweeter (@paulrobichaux).

The author team has reviewed and edited the material to ensure that it fits with the rest of the book.

Quadrotech prides itself on delivering technology that brings organizations “to, and through the Cloud,” maximizing
the efficiency, value, and insight customers receive from their IT investment.

Beginning as an email migration vendor, our solutions have expanded and diversified in order to go beyond simply
moving data. Quadrotech’s powerful, intelligent software addresses the entire Office 365 lifecycle:

Our migration suite (Archive Shuttle, PST Flight Deck, and Public Folder Shuttle) enables companies to move
their entire email eco-system to Office 365
Cloud Commander helps companies to manage the unique circumstances that happen in company splits, joins,
and mergers when Office 365 tenants must be combined or divided.
Radar Reporting is the most powerful and customizable reporting product available for Office 365, covering all
key workloads to drive adoption, increase security, and reduce unnecessary costs.
Radar Security & Audit records and analyzes Office 365 audit data, and provides insight to administrators to
what’s happening inside their tenant together with advanced filters, and on-event alerting.
Autopilot enables you to manage both cloud and on-premises environments in one interface. The tool streamlines
Office 365 administration and allows you to delegate control with increased flexibility and security.

These solutions are built with enterprise environments in mind, no matter the complexity or scale. Full information
about Quadrotech products can be found on the Quadrotech web site .

Comments and feedback

Comments about the content as well as pointers to where little errors might have crept into the text are always
welcome. Please send your comments, suggestions, and observations to BookComments@Office365ITPros.com or post
to our Facebook page .
Chapter 1: An Overview of Office 365
Tony Redmond

Office 365 is a set of cloud services, sometimes referred to as workloads. Several of the services are versions of well-
known on-premises products such as Exchange and SharePoint that have been reengineered to run at massive scale
and support hundreds of millions of users spread across multiple tenants. Microsoft has built the newer parts of the
Office 365 suite, like Delve, Groups, Planner, or Teams from scratch to use the software and data available within “the
service”, which is how people often refer to Office 365 within Microsoft. Originally launched in 40 countries in June
2011, Office 365 is now available in over 181 countries around the world.

A cloud service is a resource delivered over the Internet, including software resources (Software as a service, or
SaaS) or infrastructure (Infrastructure as a Service, or IaaS). Exchange Online and SharePoint Online, the two major
workloads within Office 365 are cloud-based services provided to Office 365 tenants. However, as will soon become
obvious, it is impossible to consider Exchange Online on its own. Office 365 is a very different environment to what
exists when a company deploys Exchange on-premises. The interconnection and interoperability between different
workloads is what makes Office 365 special.

The basic idea behind cloud services is that customers can transfer the responsibility for running workloads on
servers installed within their own premises to a cloud provider, who then charges a fixed monthly or annual fee based
on some unit of work, such as a user account or mailbox. To make this possible, companies must migrate their
workload across the Internet to one or more datacenters run by the cloud provider where enough computing,
network, and operational capacity exists to handle the workloads generated by hundreds of thousands of companies.
The value proposition is that the massive economy of scale created by cloud providers allows them to deliver the same
or better functionality at lower cost than is possible for on-premises IT to deliver. As more users move to the cloud,
the economics improve even more, which then lets providers to enhance and grow their services.

The investment needed to buy land for datacenters, datacenter construction, acquisition and installation of computing
resources, power and cooling, and automation of as many operations as possible to provide secure access to
applications to customers worldwide at a price-competitive point is a venture that can only be taken on by companies
with very deep pockets. Microsoft is one of those companies. Google and Amazon are other examples. Other cloud
providers exist, and some deliver the same application services as Microsoft does with Office 365. However, they
usually run at a regional or national level and compete with the global players by being more responsive to local
needs, offering more customized services, better access to support, or by addressing concerns such as government
access to data stored in datacenters controlled by U.S. companies.

Email is a relatively easy workload to transfer to a cloud platform. We have known this for a long time because many
companies have outsourced Exchange to third parties in a form of "private cloud". The success of consumer cloud-
based email services like Gmail and Outlook.com also show how far email has come from being a service once
delivered to a few privileged users to be a utility consumed by over 3.7 billion people daily.

The success of email in making cloud platforms a part of everyday life possibly prompted Microsoft CEO Satya
Nadella to refer to Exchange as the “ gateway drug to the cloud ” in October 2014. In making this analogy, Nadella
reflected on the history of Exchange and its importance to Microsoft when Exchange 4.0 (1996) was the first server
application to prove that Windows NT was a suitable enterprise platform. But Office 365 is not Exchange and
Exchange is not Office 365. In fact, no single application “is” Office 365. The service is the sum of the parts and more
than the sum of the individual applications.

The set of Office 365 applications described in this book is current as the time of writing. We make no attempt to
claim that the list is exhaustive because Office 365 is in a constant state of evolution and development. Microsoft
introduces new features on an ongoing basis to keep the software “evergreen”. Office 365 flexes and evolves to keep
pace with the innovation coming from Microsoft’s cloud competitors and to satisfy customer demand. A cloud service
that stays static is less attractive to its users than one where new updates appear all the time.

This chapter gives an overview of Office 365. Understanding Office 365 is important when it comes to knowing its
strengths and weaknesses and how a customer can fully exploit the service to move some or all its current on-
premises workload to Office 365.

Office 365 Infrastructure

Attention to detail and absolute adherence to procedure are hallmarks of how Office 365 runs. Without attention to
detail enforced through automated processes and procedures, it would be impossible to manage Office 365 for
millions of companies. Three aspects deserve attention: the degree of automation that exists within the service; the
attention paid to network events; and the sophisticated methods of monitoring used to detect problems.

Datacenters and Regions

Office 365 tenants share a single large logical infrastructure composed of hundreds of thousands of servers spread
across multiple Microsoft datacenters. Figure 1-1 shows the Office 365 datacenter deployment in early 2018. By its
very nature, this overview is incomplete as it does not show some of the infrastructure that is under development. In
addition, it does not convey the deep investment made to create “edge” network termination points set up by
Microsoft to bring user traffic quickly into Office 365 from all around the world or the internal network that
transports Office 365 tenant data between the datacenters.

Microsoft organizes the Office 365 datacenters into fourteen regions. The datacenter region selected to host the data
for new tenants is based on the country (location) selected by the tenant. Since the launch of Office 365 in 2011,
Microsoft has gradually built out the Office 365 datacenter infrastructure with the intention of keeping data as local
as possible (“ in-geo data residency ”) to accommodate customer choice and satisfy local regulations. Where Office
365 once concentrated service delivery from larger datacenter regions such as Western Europe (with datacenters in
Ireland, Finland, Austria, and the Netherlands), localized service is now available in individual countries like France,
Germany, and the U.K. The same is true in Asia-Pacific, where Office 365 services come from datacenters in Japan,
South Korea, Singapore, Australia, and China.

Apart from the ability to serve large customer populations, natural and economic advantages such as ambient
temperature (to reduce the need for cooling) or availability of cheap hydro power influence datacenter placement.
Obviously, security is of prime concern and Microsoft pays great attention to the physical security of the buildings
(you will not find large signs proclaiming Office 365 or Microsoft anywhere) as well as cyber-security for the data
contained within the buildings.

Because the Office 365 infrastructure is constantly growing and expanding, the live location for tenant data also
changes. In addition, Microsoft moves data to rebalance load on the servers in multiple datacenters (within the same
region) and to make more effective use of available resources. Even though the underlying infrastructure is changing
all the time, users can continue to work and access their information from anywhere around the world.

Figure 1-1: Office 365 datacenter map (source: Microsoft)

The Microsoft Cloud spans well over 100 datacenters in 54 regions to deliver service in 140 countries, but every
datacenter does not host Office 365. After a new datacenter comes online, a sophisticated migration process kicks in
to move tenants from other datacenters to the new location. The same is true when Microsoft creates a new Office
365 region. For instance, after the United Kingdom datacenters came online, some tenants asked to move their work
to those datacenters to keep their data remained “in country;” the same happened in France or when Australian and
New Zealand tenants moved to the Australian datacenters. This work happens behind the scenes (just like regular
mailbox moves) so that the eventual switchover is fast and painless. Microsoft has a documented process to help
tenants with specific data residency requirements to ask Microsoft to move their core data to a new region after it
comes online.

In addition to the Office 365 applications, Microsoft has migrated over 400 million Outlook.com users of the consumer
email service from the legacy Hotmail.com infrastructure service to run within Office 365. Outlook.com mailboxes
now use Exchange Online (the sole difference is the feature set exposed to users). These mailboxes run on the same
server, storage, and network infrastructure as Exchange Online to take advantage of features like Native Data
Protection and Exchange Online Protection and the same client set is available for both services. Although the same
engineering teams are available for both services, the functionality in Outlook.com is much less comprehensive than
that available to even the entry-level Exchange Online plan. However, the two services share some features (like the
method to connect the Outlook mobile clients) and Microsoft introduces some functionality into one service before
they decide to do the same for the other. For example, “Sweep” rules first appeared in Outlook.com and are now
available in OWA, while the calendar and calendar sharing features now available in Outlook.com originated in
Exchange Online. Taken together, the infrastructure shared by Exchange Online and Outlook.com delivers email
service to over 600 million mailboxes.
Although Office 365 is invariability hosted alongside Azure, Microsoft does not host other commercial work in the
Office 365 datacenters. The Office 365 datacenters listed in Table 1-1 are those that offer core services (Exchange,
SharePoint) in the current set of datacenter regions. Other services in the Office 365 suite, like Skype for Business
and Project Online, might also run in these datacenters, but the picture is less clear for applications like Planner,
Teams, Sway, and Yammer. In some cases, other datacenters within the region deliver specific services, as in North
America where the Sway and Planner services run from datacenters in California and Virginia. In others, some
services or backup for services come from other regions. However, the situation changes over time and if you are
concerned about data sovereignty, you should check with Microsoft to understand exactly where your data are for all
applicable applications. Azure Active Directory is another service that can come from another region. For instance,
the U.K. region uses Azure Active Directory running in EMEA datacenters.

Office 365 Region Datacenter Locations Home region for tenants in

Europe (EMEA) Dublin (Ireland), Amsterdam (The Netherlands), Vienna Europe, Middle East, and Africa
(Austria) and Helsinki (Finland). (except the UK, France, and
Germany)

United Kingdom London, Cardiff, and Durham United Kingdom

Germany (in Frankfurt am Main and Magdeburg Germany

partnership with T
Systems
International)

France Paris and Marseille France

North America Quincy (Washington), Chicago (Illinois), Des Moines (Iowa), North America (except Canada)
Cheyenne (Wyoming), Blue Ridge (Virginia), San Antonio
(Texas), San Jose (California)

Latin America Campinas, Sao Paolo, Rio de Janeiro, Fortaleza (all in Latin America
Brazil), Santiago (Chile)

Asia Pacific Hong Kong, Singapore, South Korea, and Malaysia Asia Pacific except China, Japan,
South Korea, Australia, New
Zealand, Fiji, and India

Australia New South Wales and Victoria Australia, New Zealand, and Fiji

India Mumbai, Pune, and Chennai India

Japan Saitama, Tokyo, and Osaka Japan

South Korea Seoul and Busan South Korea

China (operated by Shanghai, Beijing, and Hong Kong China

21Vianet )

Canada Quebec City and Toronto Canada

U.S. Government Des Moines (Iowa) and Boydton (Virginia) U.S. Government and state
agencies

Table 1-1: Office 365 Datacenter regions

Microsoft plans to introduce Office 365 datacenters in South Africa (Johannesburg and Cape Town), Switzerland, the
United Arab Emirates (Dubai and Abu Dhabi), and Norway, with services beginning over the 2018-19 period. They
have also announced plans to replace the dedicated German sovereign datacenter region with a new datacenter
region based in Berlin and Frankfurt to offer Office 365 services in 2020.
Creation of a new datacenter region normally means that Microsoft can offer customers the basic Office 365
workloads (Exchange Online and SharePoint Online) from the datacenters. It can take some time before the full suite
of capabilities is available, including applications (like Teams or Microsoft Planner) and utilities (like the Office 365
Import Service). You can find more information about the current Office 365 datacenters online .

Workloads Running Within Datacenter Regions

Microsoft distributes work across the multiple datacenters within a region to protect data against failure. For
instance, the active-active design for Exchange Online Database Availability Groups (DAGs) means that mailbox
database copies exist in at least two datacenters within a region. In addition, as Microsoft adds datacenters to a
region, the opportunity exists to spread database copies to those datacenters. For example, new DAGs built for use by
Exchange Online in the Western Europe region might include databases spread across the Amsterdam, Dublin,
Helsinki, and Vienna datacenters. Spreading data across so many datacenters reduces the risk that any individual
outage will affect a sizeable number of users. It is something that the average on-premises administrator could never
contemplate because of the investment needed to build out the underlying datacenters and network.

You can get some insight into the distribution of services across Microsoft’s global datacenter network by running the
Get-MsolCompanyInformation cmdlet after connecting to Azure Active Directory with PowerShell (see Chapter 4 for
more information). The cmdlet returns information about the connected tenant and shows the location of the services
consumed by the tenant. Here is an example:

[PS] C:\> (Get-MsolCompanyInformation).AuthorizedServiceInstances

BDM/NA026

To-Do/NA001

AadGraphNotifications/EUGB02

OfficeForms/NA001

Adallom/NA001

Deskless/NA001

MultiFactorService/NA001

GroupBasedLicensePropagation/NA001

AADPremiumService/NA001

TeamspaceAPI/NA001

ProcessSimple/NA001

PowerAppsService/NA001

MicrosoftStream/NA001

SCO/PROD_MSUB02

AccessControlService/US

Sway/NA001

ProjectWorkManagement/PROD_EU_Org_Ring_15

PowerBI/EU001

AzureAnalysis/SDF

YammerEnterprise/NA001

WindowsAzure/EMEA001

DirectoryToCosmos/EU001

ExchangeOnlineProtection/emea01-03

MicrosoftOffice/NorthAmerica

DirectoryToCosmos/D2C001

RMSOnline/EU

Metro/NA001

AccessControlServiceS2S/EU3
AccessControlServiceS2S/EU2

AccessControlServiceS2S/EU

exchange/eurprd04-2

SharePoint/SPOS0002

MicrosoftCommunicationsOnline/Instance03

The output shows that most of the services run inside the Western Europe region (they have EU or EMEA in their
names). However, several like the Microsoft Stream video service and Sway run in North America (NA). The
datacenter region that hosts a specific service is liable to change over time as Microsoft deploys new capabilities
across its datacenters.

Another way to understand what datacenter region supports tenant data is to access the Organization Profile through
the Office 365 Admin Center. Go to the Data Location section and you will see the region for some, but not all, of the
workloads used by the tenant (Figure 1-2 ). Although Azure Active Directory holds most of the information for tenant
accounts and configurations in the same datacenter region as a tenant’s Office 365 data, an exception exists in that
Microsoft stores five user-related attributes including the User Principal Name and password hash for tenant accounts
in the U.S. This is to make sure that authentication can happen as quickly as possible, no matter where in the world a
user is located. For more information on this topic, see Microsoft’s support article on the situation for European
customers.

Figure 1-2: Data locations for an Office 365 tenant

Sovereign Clouds
A sovereign cloud is an Office 365 datacenter region that exists because Microsoft must meet specific requirements
imposed by the target customer base or geography. Three sovereign clouds currently run inside Office 365. The U.S.
government cloud (GCC) is a special datacenter region to meet the specific security needs of U.S. government and
state authorities; the other two are China and Germany. The other datacenter regions, or clouds, are general-purpose
in that these regions serve customers from any country and any industry in the coverage area.

Depending on country-specific requirements, Microsoft tailors the Office 365 software to meet local regulations and
customer needs. Some software running in the general-purpose regions is unavailable and some changed. For
example, customers must enable modern authentication (MFA) to connect to Office 365 Germany (also known as
“Black Forest”) and functionality such as Azure Information Protection, Teams, and Outlook for iOS is not supported.
In addition, license costs per user are higher for Office 365 Germany than they are for users served by the Office 365
Western Europe datacenter.

The original decision to deploy a dedicated German datacenter region managed by T-Systems International in the role
of data trustee solved a problem for Microsoft in 2015. German companies were rightly concerned about data
residency and the notion that their information might be under the control of a U.S. company. Using a data trustee to
manage services based in Germany addressed the issue, but the combination of higher costs and reduced functionality
contributed to Microsoft’s decision in August 2018 to stop offering service from the sovereign German region to new
tenants and replace it with a new datacenter region for German tenants. Black Forest customers will migrate to either
the new German datacenter region or to the EMEA region.

Multi-Geo Office 365

Office 365 customers can opt to distribute Exchange and OneDrive for Business workloads across different Office 365
regions (multi-geo). This means that the tenant has a home region (known as the “central geo”) where most of their
workload runs, and one or more satellite geos. In September 2018, Microsoft said that they will enable multi-geo for
SharePoint Online and Office 365 Groups in early 2019. Sites including team, communication, and hub sites, created
by distributed users are provisioned in their assigned geo-location (otherwise known as the PDL, or “preferred data
location”). Microsoft says that they are exploring how to add multi-geo capabilities to other Office 365 applications.
The normal use case for multi-geo is an international company that needs to satisfy data sovereignty requirements.
For example, a U.S. company might have an EMEA subsidiary. Due to the sensitive nature of the work done in EMEA,
the company does not want to store the data in the U.S., as would be normal. With multi-geo, they can choose to have
user data for the supported workloads stored in-region.

Behind the scenes, once a tenant is enabled for multi-geo, Microsoft will transfer the data belonging to the selected
users to the satellite geos. During this process, users continue to work as normal until the transfer is complete, at
which point they switch over. Cross-region synchronization within the tenant’s Azure Active Directory instance
ensures that all the users within the tenant see a single worldwide picture and can continue to share work with
colleagues without hindrance. The only thing that changes is the location of user data.

Multi-geo capabilities are not a solution to poor network performance. Some people assume that this is the case
because “the data is closer to the users.” This is a fallacy because, in most cases, poor network performance is due to
issues such as lack of bandwidth or other problems in the link connecting users to Microsoft or poor routing inside the
tenant’s internal network. Once inside Microsoft’s datacenter network, traffic flows from region to region very quickly
and users do not see a difference when they move to a local region.

Multi-geo is available for most general-purpose Office 365 datacenter regions except Latin America and France. It is
not available for the sovereign clouds. Tenants must have more than 5,000 subscribed Office 365 users before they
qualify for multi-geo. Additional monthly fees of $2 per user are payable for a multi-geo tenant. The fees cover the
connection of the base workloads to different datacenter regions. Because of the extra cost, it is likely that only
multinational companies with complex data sovereignty needs and relatively large numbers of users will be interested
in multi-geo capabilities. For more information, see the multi-geo home page .

Tenant Domains
Individual client companies that use Office 365 are tenants of the service. Each tenant occupies a subdomain within
the overall Office 365 infrastructure. You can think of a tenant domain as the container for the company within Office
365 and the domain is a sub-domain under the onmicrosoft.com root. For example, the Contoso company might run a
tenant domain called contoso.onmicrosoft.com. Each user then has a separate identifier associated with the tenant
domain used to sign in (or connect) to Office 365 and as their email address. Thus, I might use
TRedmond@contoso.onmicrosoft.com to log on to Office 365.

Companies wishing to use Office 365 usually have a domain registered in the Internet Domain Name Service (DNS)
and want to continue to use that domain with Office 365. You can assign user identifiers that belong to your “old” (or
“vanity”) domain and use those identifiers to log onto Office 365 or for mail routing. For instance, if my old domain
name was contoso.com, I confirm the domain with Office 365 and associate it with my tenant domain and then change
the DNS pointers so that Internet traffic intended for contoso.com goes to Office 365. Within Office 365, that traffic
goes to the correct tenant domain, including user sign-ins with addresses belonging to the new Office 365 tenant, and
all is well.

Office 365 does not really care what name you give to a tenant. You can always use a “vanity domain” for external-
facing email addresses and have Exchange route messages sent to those addresses to the right tenant. However, other
parts of Office 365 like SharePoint Online surface tenant names in the URLs and other components that they use, so
some care and attention is necessary to ensure that the tenant name you use is the right one for your company.

Directories and Identities

Office 365 supports both cloud-pure and hybrid environments. A key aspect of the support is the ability to reliably
authenticate user accounts with Office 365 and on-premises applications. To enable this to happen, a variety of
directories and other tools are used to provision, store, maintain, and synchronize identities to be used by Office 365.

Azure Active Directory is the cornerstone of Microsoft’s cloud identity story and acts as the source of authority for
cloud user accounts. In hybrid deployments, where the on-premises Active Directory is always the master directory,
Azure Active Directory is synchronized with Active Directory to form a seamless view of user accounts and
configuration data drawn from both environments. The tool used for this purpose is AAD Connect.

Many of the applications running inside Office 365 have the need to store information that is specific to
their operation. For instance, Exchange Online uses EXODS, its own directory store, to hold information
about public folders that are not mail-enabled. These objects exist only inside Exchange Online and are
irrelevant to the other workloads, so there is no reason to store information about them in Azure Active
Directory. The same logic applies to workload-specific objects and configuration data used by SharePoint
Online (SPODS), Skype for Business Online (LYODS), and Teams. Chapter 3 presents a comprehensive
discussion about the directories used by Office 365 and the different forms of identities (on-premises,
cloud, and hybrid).

Licenses
Every active user in a tenant needs an Office 365 subscriber license, which defines the functionality available to the
user. A tenant can buy a mixture of licenses and assign those licenses to users who need various levels of functionality.
For example, some users might only need kiosk licenses that allow browser access to applications while others need
licenses that include the Office desktop applications and sophisticated Skype calling plans. Both sets of licenses exist
quite happily within the tenant and administrators can move licenses between users based on the available license
pool.
Office 365 also includes a category called “inactive mailboxes” that do not need a license. The idea here is that you
can mark a user as inactive after they have left the company to be able to keep their data online and available to
others until you have had the opportunity to transfer the information to “live” (licensed) accounts. Service accounts
such as those used to manage tenants also do not need licenses.

Office 365 Trial Tenants

You can set up an Office 365 trial and test it for up to a month without payment. This is a practical way to try out the
functionality available under a plan and decide whether it is a good fit for your company. It is also an excellent method
to allow administrators to become accustomed to managing an Office 365 tenant as they can make as many changes
to settings as they like without running the risk of impacting users. You can transform a trial domain into a production
tenant if you want, but in most cases, it is best to start over and treat the trial as a sandbox that can be discarded
when no longer needed. For this reason, never use a corporate domain name for an Office 365 trial.

Automation
Today’s Office 365 uses a sophisticated workflow engine called “Central Admin” (CA) that is capable of handling more
than a hundred million workflow tasks per month. The idea is to automate the common tasks needed to keep services
running as much as possible to remove the possibility that human error will compromise systems. The need to avoid
human mistakes in the management of cloud systems is seen in the DNS error that interrupted Office 365 service in
September 2011 and the command typo that knocked out part of the Amazon Web Services infrastructure in March
2017. A smoothly functioning workflow engine also achieves a reliable and robust throughput of actions across the
system. Tasks are defined to CA in the form of scripted workflows in either C# or PowerShell. CA is then responsible
for the execution of scheduled tasks to perform actions such as server deployment, database rebalancing within a
DAG, and so on. More complex tasks such as the addition of new capacity to the service still needs some human
intelligence and planning, but the application of a structured model and great attention to detail has enabled
Microsoft to reduce the time necessary to complete even very complex tasks down from weeks to days.

Office 365 servers use a standardized design which Microsoft has shared through OpenCompute.org. This does not
mean that the same components are used every time, as this would be impossible in an industry where components
change often. However, it does mean that a server will have the same general characteristics (CPU, disk, memory) and
that software is installed in the same way on all servers of a specific type. Low-cost components such as JBOD arrays
allow Microsoft to increase the storage available to tenants while still being market competitive. It also means that
servers are built from modules to eliminate cabling. Everything is optimized for mass production and servers are
integrated into racks at the factory and shipped to the datacenter ready to be plugged in and brought online.

Exchange is a good example of an application where sustained engineering investment has delivered huge
improvements in performance and made cloud economics possible. Levering the huge reduction in Exchange’s disk
demands since Exchange 2003, JBOD SATA drives are used throughout Office 365 to deliver cost-effective storage.
Using these disks implies a risk of a higher failure rate than the more expensive "enterprise-class" drives often found
in corporate datacenters and indeed, across Office 365, hard disk failures are the most common event of the more
than ten thousand hardware events that are handled monthly. The low cost of storage and Exchange’s performance
profile makes it possible for Microsoft to offer Office 365 enterprise users a basic 100 GB mailbox quota and auto-
expanding archives, and to hold a mass of non-user data (such as information about Files usage) in mailboxes. Without
low-cost storage, the monthly subscription to an Office 365 enterprise plan would be much dearer.

Software helps to insulate users from the effect of hardware failures. For instance, Exchange’s Active Manager will
failover a database to a new server quickly if a disk problem is detected. It will also create a new copy of the failed
database using the autoreseed feature if replacement disks are available. Across the entire Office 365 fabric, a CA
workflow called “RepairBox” constantly checks for hardware failures and will open support tickets automatically if an
issue is detected like a failed disk. A technician can then replace the failed disk (no attempt is made to fix the disk).
The same workflow monitors servers for inconsistencies in their state to detect and fix problems with configurations.

Even with such a sophisticated and smooth-running automatic support infrastructure, Office 365 still runs into some
problems. For instance, “stragglers” are servers that run out-of-date software versions that might deliver an
inconsistent service to users. Office 365 is in a constant state of server refresh to introduce new software builds that
have new features. As such, with so many servers and so many updates, it is expected that some server updates do not
happen as well as they should, which is the usual reason why a straggler exists.

Networks
Given that Office 365 is a cloud service, it should come as no surprise that network is its most precious resource.
Without enough high-quality bandwidth, users will be unable to connect to Office 365, migrations cannot transfer data
from on-premises servers, and hybrid connectivity will not work. Microsoft does not control the backbone used by
tenants to connect their internal networks to Office 365 as the links making up the backbone are managed by a large
set of Internet Service Providers (ISPs) around the world. Although the Internet was originally designed to survive a
nuclear holocaust, local failures caused by cable problems, ISP datacenter issues, and hardware failure can all
prevent access to Office 365.

Microsoft cannot control the Internet, but it can control traffic flow within the network that connects all the Office
365 datacenters. That network is dedicated and tightly controlled and monitored. Dark fibre optical connections link
the datacenters together to ensure maximum data flow across the network. Automatic redundancy is deployed so that
a temporary outage is contained and automatically addressed. Everything that can be done to ensure that the service
is maintained is done, but even so, like all cloud services, the SLA delivered by Office 365 can only be guaranteed at
the boundary of the cloud provider’s datacenters as defined by the edge servers that handle inbound and outbound
traffic for Office 365.

Monitoring
With so many servers in use, it should come as no surprise that Office 365 generates a reasonable number of signals
relating to server and application operations. Microsoft built a “Data Insights Engine” using Azure and SQL Azure to
process more than 500 million events generated per hour. The events are aggregated and analyzed to understand how
the overall service is running and to detect problems with individual components.

The general approach is that if a problem is reported by many entities or different signals, then it must be true. By
depending on signals from multiple resources you can get close to 100% fidelity when it comes to the automatic
detection of problems, or “Red Alerts” as they are known within Office 365. In addition, by analyzing signals from
diverse sources, Office 365 can focus on where the root cause of the problem is likely to be with a high degree of
accuracy and this, in turn, allows automatic recovery actions to be launched with a high degree of confidence that
they will fix the problem. Taking a data-driven and analytic approach to the detection and resolution of problems is
key to being able to run at scale.

In addition to its signal processing engine, Office 365 also uses much simpler techniques to know when something
might be going wrong. For instance, if a spike in page views occurs for the Service Dashboard, it is likely that
customers are checking the dashboard to know whether a problem exists with one of the applications running in the
service. Such a spike can often be correlated with an output from the signal processing engine but sometimes it leads
to a discovery of a problem that is identified by human beings. The characteristics of that problem can then be
captured as a recognizable scenario for future automatic identification and resolution. It is also important to say that
signals used to identify issues are both active (those generated by specific events) and passive (those generated by
servers and users during normal work). A process of triangulation is used to spot abnormalities between the two sets
that can point to a developing problem. As in all knowledge, learning how to measure the pulse of Office 365 and
figure out what is normal and what’s not is an evolving art that requires great dedication and ongoing observation by
both automated systems and humans.

A Constant State of Change

The software running inside Office 365 is under constant development and Microsoft introduces new features on an
ongoing and consistent basis. The changes range from tweaks to the web-based interface to the introduction of a
completely new feature that changes the behavior of an application. This is a major difference between the traditional
on-premises model where new software releases often appear on an annual release cycle that customers can then
factor into carefully-planned “change windows”. Becoming accustomed to the pace of change within Office 365 can be
quite a challenge for those used to the older way of deploying software updates but it is the method employed by most
major cloud services.

In August 2015, Microsoft announced that over 450 updates had been made to Office 365 in the preceding year. With
so much change happening inside Office 365, it can be a problem for tenants to understand what might happen next.
The problem is compounded if you use other Microsoft 365 components such as Enterprise Mobility and Security.

The best way for administrators to know what Microsoft is working on is to keep an eye on the online Microsoft 365
Roadmap (Figure 1-3 ). This version of the roadmap replaced the Office 365 roadmap in September 2018 and now
includes features coming to Office 365, Enterprise Mobility and Security, and Windows 10. Although the roadmap
sometimes misses an update and you always must keep your eyes peeled on what appears in the service to detect
some new functionality, Microsoft refreshes the roadmap regularly and its contents are comprehensive enough to
allow tenants to plan for new functionality.

Microsoft organizes the Microsoft 365 roadmap into the following sections:

Launched : Features that Microsoft has deployed to all applicable customers.

Rolling Out : Features that Microsoft is deploying across Office 365. As you can imagine with such a massive
infrastructure, it can take some weeks to deploy new software to every server running in every datacenter
around the world. Microsoft usually posts to the Office blog to inform customers about new features when they
begin the roll-out process. The appearance of a blog post is no guarantee that a new feature will show up in a
specific tenant anytime soon as this depends on whether the new feature belongs to the set available in the plans
the tenant uses, the length of time that the feature spends in Targeted Release (previously First Release) status,
and the time taken for Microsoft to deploy the feature to all applicable tenants after it reached standard release
status.
In Development : Features that Microsoft has announced are under development. Microsoft does not commit to
deliver any of the features listed here as reasons might emerge to change or cancel a feature before the code
reaches tenants.

Each roadmap item has a feature identifier (in the form Feature ID 61652). You can match the feature identifier
against change notifications announced in the Message Center, part of the Office 365 Admin Center (see Chapter 4),
where a notification includes text such as “This message is associated with Office 365 Roadmap ID 61652 ”. In
addition, each item is dated to tell you when Microsoft added it to the roadmap, when the new functionality should be
available, and when Microsoft last modified the information for the item.

It is important to understand that the roadmap gives general guidance as to what might change in the future. The
Message Center is a more authoritative view of new developments that apply to your tenant. Roadmap items give a
glimpse into the future, but they might not be available for six months or more. Once a development shows up in the
Message Center, it’s more likely to appear in the following few weeks, meaning that it is time to prepare for change.

Figure 1-3: The Microsoft 365 roadmap

You can apply filters to the roadmap to show just the most recently released features or to focus in on specific
products. Using filters to navigate the roadmap is useful as the sheer number of documented changes can be
overwhelming at first glance. Filters also allow you to zero in on functionality that is most important to your company.
This includes application level features (for example, OneNote or Outlook), service-level features (for example,
Exchange Online or SharePoint Online), and even sector capabilities (for example, features due for delivery in the
Government GCC sovereign cloud).

The roadmap supports a download facility, meaning that you can apply filters to find the set of information you’re
interested in and then download details of those features to a CSV file, which you can process later with Excel or load
into Power BI for further analysis.

Even if it is less active than it once was, a “ Change Alerts ” group in Microsoft’s Technical Network provides another
method to discover when new features appear inside the service as group members post reports when they discover
new features appearing in their tenants. Other groups in this network cover topics such as Exchange Online, Office
365 Groups, Microsoft Teams, and Delve.

Command and Control

Office 365 is a massive machine that moves forward at its own pace. Thanks to the roadmap, we know a lot more
about what Microsoft is working on for the future than we did in the first few years, but even so, you cannot get away
from the fact that customers cede an enormous amount of control to Microsoft when they use Office 365. You accept
that Office 365 will provide you with a wide range of functionality, but you don’t get to vote what functionality is
exposed to users or when changes show up. It just happens. This is a very different experience to the careful control
that most IT departments exercise over the computer systems use in-house.

Most of the time this is not a problem and users like to find that new features continually appear. Google proved the
attractiveness of an evolving interface to end users when it kept Gmail in what seemed to be perpetual beta for many
years. Even now, new features appear all the time in Gmail and Google Apps, so Microsoft is simply keeping pace with
its competition when it seeks to refresh Office 365 on a frequent basis.

However, dealing with a rapid update cadence can be problematic in the following ways:

User support : Two broad categories of users exist – people who access Office 365 through desktop applications
and seldom make use of the web-based applications such as SharePoint and Planner and those who use a
browser. In some cases, you don’t have a choice because some applications only have a browser interface. Many
Exchange users prefer Outlook because it allows them to continue working when offline. To some degree,
Outlook insulates users from the ongoing changes that occur within the service. New features only appear when
a new version of Outlook is installed and even the “click-to-run” version of Outlook is slow to introduce the user
interface necessary to support new functionality. On the other hand, those who use OWA to interact with
Exchange Online might see new features show up on an ongoing basis. Every change in a user interface creates a
potential flow of calls to local IT support as users seek information about why the change occurred and what it
means to them. This is a very different mode of working to the normal carefully-controlled and planned change
management practiced by corporate IT departments. On the other hand, Outlook is based on an old architecture
now and more modern desktop clients, like Teams, have auto-update capabilities which mean that they pick up
new functionality as soon as Microsoft releases it.
Administration : Those who manage an Office 365 tenant work mostly through a browser interface. A lot of the
Admin framework has been consistent over time, but the framework must change to accommodate new
applications and features. For example, Microsoft upgraded the Office 365 Admin Center in 2016 and has since
 updated the new center to include management settings for applications such as Teams. Keeping up to date with
the release of these features and learning how to manage them with the available tools (also usually dependent
on whatever Microsoft provides) can be a challenge for tenant administrators.
IT management : The role of IT management in an on-premises environment is well understood. They accept
needs and requirements from the business and work out how best to meet these requests in line with available
budget, the current IT infrastructure, and the knowledge and experience available to the company. The business
will continue to generate needs and requirements and Office 365 is now one of the ways that the requests can be
satisfied. The big difference is that IT management must accept whatever functionality is available in Office 365.
The task of matching need and capability is easy if a good match is available within Office 365. Things become a
little trickier when functionality changes, as a decision to invest in an on-premises solution or to buy in software
from another provider might be undermined if Microsoft decides to offer equivalent functionality in Office 365.

The issues listed above are less important to small companies than they are to large companies who tend to have well-
entrenched methods to control the introduction of new technology. In the same way, new companies usually embrace
new technology faster than older companies do. A further consideration comes from the employee mix, where younger
employees are more accustomed to rapid change than their older peers are.

Apart from preparing IT Support staff with information about new changes so that they can respond to user queries,
it’s hard to manage how functionality flexes and changes within the service because of its rapid release cadence. One
method that IT departments have successfully to help manage the interaction between Office 365 and end users is to
appoint an Office 365 change coordinator. This role is to monitor changes, collect information about the changes, and
make sure that the information about the changes gets to the right people in a form that makes sense to them and can
be used (whenever possible) by the business to gain advantage. Microsoft’s Office 365 Roadmap is a prime source of
information for this person but they should not limit their scan for information about Office 365 to Microsoft. A wealth
of tips and advice exists across blogs, social networks, and conferences that needs to be mined to form a complete
picture of what is going on within Office 365 at any point in time. And that picture has then to be placed into context
with the business goals and needs of the company so that the information is best used.

Office 365 Plans

Microsoft sells several plans for Office 365, each specifying the exact functionality that is available to users and
administrators. Details of the current plans are available online and range from the cheapest and least functional that
are targeted at individual professionals and small companies right up to the enterprise versions that include the
fullest range of functionality. Even the cheapest Office 365 plan is highly functional and includes access to email,
SharePoint document libraries, and Skype communications.

Some have accused Microsoft applying the Office 365 brand too liberally with the result that they confuse customers
about just what Office 365 really means. In the context of this book, Office 365 does not mean Office 365 Home , a
subscription-based service to allow people to run the latest version of the Office desktop applications on their
Windows or Mac computers. Instead, we discuss the set of cloud-enabled server applications running in Microsoft
datacenters that users access via the internet. The applications include Exchange Online, SharePoint Online, Skype
for Business Online, and Yammer, as well offerings that are not available in on-premises versions such as Teams and
Planner.

Users see the applications included in the plan assigned to their account when they open the Office 365 portal .
Figure 1-4 shows an example of the Office 365 home page, tailored for individual users to show the applications which
they use most often along with a list of recently-accessed documents. Because the user has some administrative
access, the applications listed include the Office 365 Admin Center and the Security and Compliance Center. Office
365 customizes the portal to show applications the user recently accessed. The full set of applications is available
through the Explore all your apps link.
Figure 1-4: The Office 365 portal for a user

Accessibility : Microsoft’s Office division has a program in place to make its software more accessible to the
disabled. Office 365 is part of this program and ongoing efforts exist to improve usability for people who have vision,
hearing, learning, and literacy impairments. For more information on this topic, see the Office Mechanics video .

Service Families
Microsoft divides Office 365 into a set of service families, each of which splits into the plans sold to customers. Plans
dictate the functionality available to users. Table 1-2 lists the service families and plans available as at November
2017.

Office 365 Service Family Available Plans

Office 365 Business Office 365 Business Essentials

Available for up to a maximum of 300 users. Office 365 Business Premium

Enterprise Office 365 Enterprise E1

Available for an unlimited number of users . Office 365 Enterprise E3

Office 365 Enterprise E5

Office 365 Enterprise F1 (first-line worker).

Education Office 365 Education

Available for an unlimited number of users

Nonprofit Office 365 Nonprofit

Government Office 365 Government E1

Available for an unlimited number of users Office 365 Government E3

Office 365 Government E5

Office 365 Government F1

Office 365 by 21Vianet See this page for information about the plans available in China.

Office 365 Germany See this page for more information.

Table 1-2: Office 365 Service Families and plans

The exact plans available in specific markets vary from country to country, as does the pricing. Sometimes differences
in tax regimes drive the price charged by Microsoft in a certain country and sometimes the competitive landscape
within the country is the primary influence. Customers buy licenses for plans based on their appropriateness, status,
and availability. For instance, you can only buy the Office 365 Education plan if your organization is a recognized
university or another educational establishment. Likewise, the nonprofit plan is only available to organizations that
meet Microsoft’s criteria to have nonprofit status. Microsoft process education and nonprofit plans at a large discount
compared to the enterprise plans.

The Government plans are available in the United States and are broadly like the corresponding enterprise plans.
However, new applications invariably take longer to show up in the government plans because of the need to meet
federal requirements for cloud services. Teams is a good example. Despite being generally available to enterprise
customers since March 2017, Teams is only available to tenants in the U.S. Government Cloud (GCC) since July 17,
2018 . Even then, some other Office 365 components used by Teams (like Stream) are not yet qualified for GCC and
are therefore unavailable for use with Teams inside GCC.

Microsoft targets Office 365 Business at small enterprises and includes the basic applications (Exchange Online,
SharePoint Online, OneDrive for Business, Teams, Skype for Business, Groups, Sway, and Planner). The difference
between the business and enterprise plans is in the detail and depth of the functionality rather than the individual
server applications. For example, if you need access to the widest range of compliance features, you need to buy the
enterprise plans because those plans include that functionality. The assumption here is that individual professionals or
small companies probably do not need quite the full range of compliance features.

A challenge that exists when writing about Office 365 plans is that they flex and change over time to reflect
competitive pressures, new offerings, and new markets. The best idea for anyone considering a move to Office 365 is
to do some Internet research to discover what plans Microsoft offers in your geography and what monthly price
applies per user. You also need to figure out whether you qualify for an academic, education, or non-profit variant of a
plan as the pricing for these plans usually come with a large discount. And if you work for a government agency, you
might find that you qualify to buy the government version of Office 365.

Office 365 is not always Office 365: The power of marketing means that the Office 365 name is applied to several
Microsoft offerings. This book is about Office 365 for business and not Office 365 Home (a five-user license for the
Office desktop applications) or Office 365 Personal (a one-user license for the Office desktop applications). Both are
worthy offerings, but they have nothing to do with Office 365 cloud-based application like SharePoint Online and
Yammer.

Sometimes the way Microsoft marketing comingles data points about Office products generates some confusion. For
instance, at BUILD 2016, Microsoft said that users make 3 billion minutes of Skype calls a daily. No breakdown was
given between calls made using Skype for Business, Skype for Business Online, and Skype consumer. Likewise, when
Microsoft talks about 1.2 billion Office users, these are people who use the Office desktop application suite or its
derivations (like Office for iOS) and has nothing really to do with Office 365 except that some Office 365 licenses
include the ability to use the Office desktop suite. Lies, damned lies, and statistics…

The Functionality of Office 365 Enterprise Plans

The Office 365 enterprise plans are the major subject of this book. A range of these plans are available. Although E1 is
the cheapest plan, it still includes the fundamental collaboration capabilities offered by Exchange, SharePoint, and
Skype for Business. The E5 plan is the most expensive and includes features that every user might not need or want to
use, like MyAnalytics and Advanced eDiscovery. However, instead of forcing tenants to use the same plan for every
account, you can mix and match plans by buying pools of licenses for the different plans and then assign the most
suitable license to the right users.
Table 1-3 compares the basic E1 plan against the higher-end E3 and E5 plans to illustrate the difference in
functionality that grows as the price of plans increase. All plans offer a wide range of functionality, including
Exchange Online, SharePoint Online, OneDrive for Business, and Skype for Business. The obvious differences between
the E1 and E3 plans are the lack of access to Office desktop apps, the compliance features (mailbox holds, data loss
prevention, information protection and encryption, and so on). The E5 plan introduces functionality such as Advanced
Threat Protection, Cloud App Security, Advanced eDiscovery, and the Skype for Business Online features that allow
companies to consider replacing traditional PBXs. The Microsoft 365 E3 and E5 plans include Office 365 E3 and E5
respectively, while Microsoft 365 F1 includes Office 365 F1. The Microsoft 365 E3 and E5 plans also include Windows
10 Enterprise and Enterprise Mobility and Security.

Feature Enterprise Enterprise Enterprise

E1 E3 E5

$8/month $20/month $35/month

Office Online (web versions of Word, Excel, etc.) Yes Yes Yes

Office for smartphones and tablets (up to 5 installs per user on each of No Yes Yes
PCs/Macs, tablets, and phones)

Office Pro Plus (click to run Office desktop applications) No Yes Yes

“Basic” Exchange email functionality (mail and calendars) Yes Yes Yes

Advanced Exchange email functionality (archiving, legal hold, unlimited No Yes Yes
storage, Data Loss Prevention)

Skype for Business Online meetings and instant messaging Yes Yes Yes

SharePoint Online (team sites) Yes Yes Yes

OneDrive for Business File Storage and Sharing Yes Yes Yes

Sway (“Professional Story Telling”) Yes Yes Yes

Yammer Corporate Social Network Yes Yes Yes

Microsoft Stream Yes Yes Yes

Delve and the Office Graph Search and Discovery Yes Yes Yes

Basic management (administration GUI and PowerShell) Yes Yes Yes

Enterprise management No Yes Yes

Rolling updates Yes Yes Yes

Azure Information Protection (rights management) No Yes Yes

Security and Compliance Center Yes Yes Yes

MyAnalytics No No Yes

Skype for Business Meeting Broadcast Yes Yes Yes

Advanced eDiscovery No No Yes

PSTN conferencing No No Yes

Modern voice with cloud PBX No No Yes

Advanced Threat Protection (for email) No No Yes

Office 365 Cloud App Security No No Yes

Advanced Data Governance No No Yes

FastTrack onboarding service (for qualifying customers) Yes Yes Yes

Table 1-3: Comparing functionality across Office 365 Enterprise Plans

Cheaper web-only (Office 365 F1 or Exchange Online K1) plans are also available to accommodate deskless workers
or those who use a shared PC. These plans include email and web-based versions of the Office applications. Mailbox
sizes are smaller than other plans (2 GB) and cannot have archives but are large enough to meet the needs of many
users. Apart from a mailbox, the F1 plan includes Skype for Business, Teams, 2 GB of OneDrive for Business storage,
Yammer, Sway, and Microsoft Stream.

Office 365 is flexible when it comes to altering the number and type of licenses that a tenant owns. Each user has a
separate license, so you can mix and match license types within a tenant. The flexibility in licensing means that you
can increase or reduce capacity as the number of users grows or declines over time. It also allows you to create a
customized mix of licenses in a pool for allocation to users based on their needs. For example, a company of 10,000
users might have some people who only need intermittent access to email and never join online meetings. Kiosk
(front-line worker) plans might satisfy these users while the enterprise plans serve other users better. Even within the
enterprise plans you might have a division between users who only need basic (E1) functionality and those who need
the more extended variety (E3). In some cases, the need for compliance features will drive the choice between E1 and
E3 while factors such as the need to replace an old PBX or the desire to use advanced security and threat
management functionality will convince tenants to upgrade to E5.

Microsoft publishes Service Descriptions and comparisons to guide customers through the functionality available in
the currently available Office 365 plans. As in all negotiations, before you can decide what Office 365 plan is right for
your company, you need to understand what functionality you need now, what might be necessary in the future, and
the features that you do not need. Once you know the functionality you need, you can discuss licensing requirements
and pricing with Microsoft.

Remember that the details of plans change over time, even within specific features. In addition, Microsoft has retired
plans in the past and forced customers to make a choice about a replacement plan. For example, Microsoft deprecated
the E4 plan in 2016 and replaced it with the E5 plan. Even if your selected plan is still offered by Microsoft, it is
sensible to use the annual subscription renewal cycle as a reminder to check the functionality offered in the various
plans and confirm which plan is best for your organization. Microsoft introduces new features and applications into
Office 365 on an ongoing basis. The plan you decided was best two years ago might not be the best plan now.

Dedicated Office 365 : When Microsoft began to offer cloud services on a commercial basis, some larger companies
declined to consider the idea of sharing resources in a multi-tenant deployment. The idea was that if the customer
had dedicated hardware and network resources, they would have a better guarantee of performance and robustness.
In addition, because customers had dedicated resources, those companies could exert more control over the software
configuration and updates. Microsoft duly obliged and offered dedicated Office 365 plans to large companies. Over
time, Office 365 proved that it is possible to deliver the kind of SLA demanded by large companies, even in a multi-
tenant environment and it became harder and harder to justify the added cost and complexity needed to run the
dedicated environments. Microsoft has now deemphasized dedicated Office 365 and is moving customers to the
standard multi-tenant environment as quickly as it can. Due to contractual arrangements, this process will take years
to complete.

Adding Cost to Office 365

Microsoft is not a charitable organization and it is in their interest to sell as much of their cloud services to customers
as they can. The likely sources of added revenue for Microsoft are:

Convincing customers to use higher-priced plans. For example, selling the E5 plan for use by a certain segment
of the user base.
Selling plan add-ons. If customers do not want to buy a higher-priced plan, they might be able to buy specific
functionality through an Office 365 add-on.
 Expanding to include other cloud services. Office 365 includes basic access to Azure Active Directory and you
might like to buy premium licenses to increase the overall security of the tenant and add functionality through
features such as conditional access policies, group access reviews, group expiration policies, and password write-
back to an on-premises directory used in hybrid deployments. The same is true of mobile device management,
which you can perform at a basic level through the ActiveSync management tools built into Exchange Online but
is easier and more functional when you deploy the Enterprise Mobility and Security suite. An alternate to buying
separate plans is to upgrade to Microsoft 365 to take advantage of the bundled price for Office 365, Enterprise
Mobility and Security, and Windows 10.

Opting to buy options can increase a tenant’s monthly bill by a large amount. On the other hand, if the functionality is
needed and it enables you to decommission older systems (especially on-premises servers), then the cost might be
justified. Increased security is also an important factor to consider when users depend on internet connectivity to
cloud systems. What is clear is that it is important that customers figure out the most effective set of licenses for
Office 365 and other Microsoft cloud offerings by balancing cost versus functionality. When a tenant is operational,
you can monitor what functionality is used and what is not, where added software might be needed, and keep a wary
eye on the available pool of purchased licenses to ensure that you do not pay monthly fees for unused licenses.

Plan Add-ons
The Office 365 plans include lots of functionality but sometimes you need just a little bit more. For example, assume
that you license the E3 plan for everyone, but some people have expressed an interest in MyAnalytics, which is part of
the E5 plan. You could simply buy some E5 licenses and assign the licenses to interested parties, but it is sometimes
possible to buy the specific feature through a plan add-on. If this is the case, it is usually cheaper to buy exactly what
you need rather than to upgrade to the next plan. To discover what add-ons are available to you, click Purchase
Services in the Billing section of the Office 365 Admin portal, which brings you to the service catalog . You can then
select whatever add-on you need and decide how many licenses to buy. Popular add-ons available in the Office 365
catalog include those listed below. Add-ons are licensed and charged for on a per-user, per month basis. The monthly
charge for an add-on varies from country to country.

Advanced Compliance.
Advanced Threat Protection.
MyAnalytics.
Customer Lockbox.
Threat Intelligence.

Enterprise Mobility & Security and Azure Active Directory Premium

Office 365 tenants use Azure Active Directory to store information about user accounts and settings. This is a basic
version of Azure Active Directory designed to support Microsoft Online Services. Although the basic version is enough
for Office 365, some organizations, especially those with hybrid deployments, might be interested in paying extra to
access the features exposed through Azure Active Directory Premium. You can buy Azure Active Directory Premium
licenses (US $6/user per month) or as part of the Microsoft Enterprise Mobility and Security products (EMS). Table 1-
4 lists the functionality available through the EMS plans.

Enterprise Mobility + Identity and access Managed mobile Information protection Identity driven
Security plans management productivity security

EMS Plan E5 Azure Active Directory Microsoft Intune Azure Information Microsoft Cloud App
Premium P2 Protection Premium P2 Security
($15/month)

EMS Plan E3 Azure Active Directory Microsoft Intune Azure Information Microsoft Advanced
Premium P1 Protection P1 Threat Analytics
($8.75/month)

Table 1-4: Functionality available in the Enterprise Mobility and Security plans

Table 1-5 lists some of the features supported by Azure Active Directory Premium P1 that are usually of interest to
Office 365 tenants. The full feature list enabled by Premium licenses is available online . Azure Active Directory
Premium P1 includes other features, such as conditional access policies for Exchange Online and SharePoint Online ,
which tenants can deploy to force users to sign-in with multi-factor authentication based on network location. The P2
plan adds identity protection and privileged identity management.

The decision to invest in Enterprise Mobility and security or Azure Active Directory Premium licenses depends on the
value that a tenant can gain from the available functionality. Some tenants will never need any of these features, some
will need just one or two features, and some will use all the features. As a decision to use these products is a factor
that can influence the overall cost of Office 365 for an organization, it is a factor to include in budget discussions.

Feature Use
Enhanced multi-factor Protects other workloads in addition to Office 365.
authentication

Password write-back Enables the write-back of user passwords to an on-premises Active Directory.

Connect health Delivers information about directory synchronization performed with Azure Active Directory
Connect.

Dynamic groups Allows groups to have dynamic membership calculated based on queries executed against
Azure Active Directory (including dynamic Office 365 Groups).

Group expiration Expires Office 365 Groups after a predetermined period and allows group owners to renew
policy their groups for a further period.

Advanced reports Includes reports such as password reset activity and irregular sign-in or anomalous sign-in
activity

Track protected Allows users who circulate protected documents to track who has received the documents and
documents where those recipients are.

Assign licenses via You can achieve a basic level of automation by using Azure Active Directory Groups to assign
AAD Groups Office 365 licenses to members of those groups.

Conditional access Control who can access your tenant and the conditions under which they are allowed access.
policies

Self-service password Allow users to reset or unlock their passwords without administrator intervention.
reset (SSPR)

Table 1-5: Azure Active Directory Premium features of interest to Office 365 enterprise tenants

Licensing Requirements for Azure Active Directory

As described above, some Azure Active Directory features need premium licenses. Microsoft says that “a proper
license is required if a user benefits directly or indirectly from any feature covered by that license.” Sometimes
Microsoft does not enforce a license requirement to block someone from using a feature and sometimes a partial
block is in place. For example, if an administrator account has an Azure Active Directory P1 license, they can use the
Azure portal to create dynamic Office 365 Groups or define a group expiration policy. The Azure portal enforces the
license requirement by not revealing the necessary UI unless the user has the correct license. However, no license
requirement exists in PowerShell, so you can use it to create dynamic groups or create a policy. It is also true that
dynamic groups work even if the accounts that come within the scope of a query do not have a premium license.
Microsoft could enforce the requirement at any time in the future, so it is both unwise and contrary to the licensing
agreement to take advantage of any gaps you find.

Microsoft 365
Microsoft 365 is an integrated bundle of Windows 10, Office 365, and the Enterprise Mobility and Security suite
(EMS), packaged in different forms to meet the needs of different customer sectors. Building on the experience of
Office 365, Microsoft launched Microsoft 365 as a subscription service in July 2017 to make it more attractive for
customers to buy a complete set of products. Although Office 365 is part of Microsoft 365, it is not replaced by
Microsoft 365, and continues to be sold separately.

Variant Target market Components

Microsoft 365 Companies with more than 300 users. Windows 10 Enterprise
Enterprise
Office 365 (E3 or E5)

EMS

Microsoft 365 Business Small to medium companies (up to 300 Windows 10 Business
users).
Office 365 Business Premium
 EMS

Microsoft 365 Frontline Customer service and support workers Windows 10 Enterprise

Office 365 F1

EMS

Microsoft 365 Educational establishments Same offerings as for Enterprise, Business, and
Education Frontline.

Microsoft 365 Non- Non-profit organizations Same as Microsoft 365 Business.

Profit

Microsoft 365 U.S. government and state agencies Same as Microsoft 365 Enterprise (E3 and E5
Government bundles).

Table 1-6: Microsoft 365 Variants

No difference exists in the Office 365 functionality available in the Microsoft 365 plans over what you get in the
regular Office 365 plans. When Microsoft 365 was launched, it was simply a matter of bundling several software
packages together. This isn’t a bad thing, because if Office 365 proves anything, it shows how more functional
technology can be when multiple applications are available. Microsoft Teams, for instance, cannot work if Exchange
Online, SharePoint Online, Office 365 Groups, and OneDrive for Business are unavailable. As time passed, evidence
like the replacement of the Office 365 Administration Center with an upgraded console that includes different aspects
of other Microsoft 365 components reflects how closer integration across the suite is happening. The same is true in
the Security and Compliance Center. Looking back, this progress is like what happened as Office 365 progressed from
being a loose collection of barely cloudified on-premises applications to an integrated ecosystem. That progression
took the best part of six years; it’s hoped that the integration of Microsoft 365 will happen sooner.

Over time, it is likely that an increasing percentage of the Office 365 base, particularly in medium to large enterprises
(over one thousand seats) will decide that their best licensing arrangement is one built around Microsoft 365. Many of
these customers already license Office 365 and the Enterprise and Mobility Suite because they want to use managed
devices and advanced Azure Active Directory features alongside Office 365, so the progression to full-blown Microsoft
365 is not hard. A statistic cited by Microsoft EVP Scott Guthrie during a keynote at the Worldwide Partner
Conference in July 2016, some 40% of enterprise Office 365 tenants use EMS. By September 2018, the number for the
EMS installed base cited by Microsoft had grown to 82 million. If we assume that most of these users also use Office
365, the evidence is that growing percentage of enterprise customers use both EMS and Office 365, which also shows
the logic behind packaging them together in Microsoft 365.

Buying Office 365 Through Partners

Remember that Microsoft is not the only company that sells Office 365. You can also buy Office 365 through Microsoft
partners to gain the benefit of the insight and knowledge that they have about how to deploy and use Office 365. In
many countries, local resellers package Office 365 with other services to suit the local market. You still get Office 365
in the same form as offered direct by Microsoft along with whatever added benefits the reseller can deliver. The
reseller might charge an extra fee on top of the Office 365 plan charge levied by Microsoft. The question therefore is
whether it ever makes sense to buy from a reseller.

The answer depends on exactly what added services you get for the extra fee. For instance, some resellers will take
care of the entire migration process for you while others provide “white glove” service where they take manage any
support issues that occur. Dealing with cloud support can be a particularly frustrating area so interacting with a local
services company that understands how cloud support works and has their own direct knowledge of Office 365 to help
solve problems without the need for escalation can be very valuable.

The Commercial Success of Office 365

Only Microsoft knows just how successful Office 365 is and how many paid subscribers exist. This is commercially
sensitive data and it should come as no surprise that they reveal as little real data about the numbers as they possibly
can. What Microsoft does report is the revenue associated with commercial cloud products, which covers Azure,
Enterprise Mobility, Dynamics 365, and Office 365. In FY16, Microsoft moved Office 365 into the “Productivity and
Business Processes” segment for financial results. Growth continues in this segment, but the changed definition
makes it a little more difficult to compare the results against earlier years.

In their FY16 Q4 results, Microsoft reported that the annualized revenue run rate (ARR) for commercial cloud
products had reached $12.1 billion . Fifteen months later, when they announced their FY18 Q1 results (October 2017),
Microsoft said that the run rate was $20.4 billion, a remarkable jump of $8.3 billion over five quarters that
comfortably allowed CEO Satya Nadella to attain his goal of $20 billion set two years previously. To calculate the
annualized run rate, Microsoft takes the result achieved in the last month of a reporting period and multiples it by 12.
In other words, they sold $1.7 billion of commercial cloud products in September 2017, the last month of the first
quarter in their 2018 fiscal year. Because Microsoft books varying numbers of customer projects over a year, a
difference always exists between the actual revenue achieved in a year and the ARR. Nevertheless, the headline figure
used for ongoing comparison is the ARR.

Although Azure is growing faster than Office 365 (93% versus 42% for revenue as reported in Microsoft’s FY18 Q3
results), the higher prices paid for Office 365 E3 and E5 subscriptions (or RPU, revenue per user) and the number of
Office 365 users indicates that Office 365 is a large part of the commercial cloud products revenue mix. At the FY18
Q3 earnings call, a Goldman Sachs analyst estimated that Azure brings in roughly $8 billion of annualized revenue.
Dynamics 365 probably accounts for another $1 billion, which implies that Office 365 is responsible for up to $15
billion annualized revenue, underscoring its importance to Microsoft. This equates to an annual RPU of $111 per
active user, or $9.26 monthly. The growth rate for Office 365 has slowed a tad recently, but the growth still shows a
continuing movement of workload from on-premises systems to the cloud.

Much of the early growth in Office 365 tenants in the 2011-2015 period came from in the small to medium business
segments as they moved from outdated on-premises servers to the cloud. Evidence since suggests that larger
enterprises are now comfortable enough with the security, privacy, and operational aspects of cloud services to move
an increasing amount of workload from on-premises systems. Although few large companies like to trumpet such a
fundamental change, some stories do emerge, like ABB’s migration of 125,000 Lotus Notes users to Office 365 , a
project reported as completed in just six months. Microsoft’s Office blog has more information about recent customer
wins for Office 365.

Table 1-7 lists how the annualized revenue run rate for Microsoft’s commercial cloud products has grown over time.
Although convincing large customers to embrace the cloud generates headlines, another sign of the momentum
behind customer movement to the cloud is in Microsoft’s assertion (January 2016) that 50,000 SMB businesses
become Office 365 customers every month . As of October 2017, Office 365 is available in 246 markets and 44
languages .

Quarter results Microsoft reported annualized revenue run rate (ARR) for commercial cloud
products

FY15 Q3 (April 2015) $6.3 billion

FY15 Q4 (July 2015) $8.0 billion

FY16 Q4 (July 2016) $12.1 billion

FY17 Q1 (October Over $13 billion

2016)

FY17 Q2 (January Over $14 billion

2017)

FY17 Q3 (April 2017) $15.2 billion

FY17 Q4 (July 2017) $18.9 billion

FY18 Q1 (October $20.4 billion

2017)

FY18 Q2 (January $21.2 billion (based on $5.3B revenue reported for the quarter)
2018)

FY18 Q3 (April 2018) $24 billion (based on $6B revenue reported for the quarter)

FY18 Q4 (July 2018) $27.6 billion (based on $6.9B revenue reported for the quarter)

Table 1-7: Microsoft reported results for annualized revenue run rate for commercial cloud products

The actual number reported by Microsoft for Commercial Cloud Revenue for FY18 was $23.2 billion. The difference
between the $27.6 billion run rate and the actual number is accounted for by the fact that revenue for the last quarter
of a fiscal year is often higher due to deals closed to get them into the year.
In September 2018, Microsoft announced that they would report results for LinkedIn in the commercial cloud
segment on an ongoing basis. LinkedIn contributed $3.4 billion in FY18, so the new baseline for comparison purposes
is $26.6 billion.

Monthly Active Users

In terms of people who use the service, the increase in Office 365 users reflects revenue growth. The reported
number is for accounts that log onto Office 365 and perform some action, like creating and sending a message or
scheduling a meeting. Table 1-8 lists the growth in monthly active Office 365 users as reported by Microsoft since
November 2015. At the current rate, Office 365 gains over 2.5 million users per month and might reach 160 million
users by the end of 2018.

Date Microsoft official number for active Office 365 users Monthly growth over previous number

November 2015 60 million N/A

April 2016 70 million 2 million/month

October 2016 85 million 2.5 million/month

April 2017 Over 100 million 2.5 million/month

October 2017 120 million 3.33 million/month

April 2018 135 million 2.5 million/month

Table 1-8: Growth in Office 365 monthly active users

Exchange is the largest workload running inside Office 365, so the number of people using Exchange for email is
hugely influential on the overall success of Office 365. In September 2017, Office 365 Corporate VP Rajesh Jha said
that he expects more than 70% of the Exchange installed base to be in the cloud by Microsoft’s 2019 fiscal year . Jha
did not specify when during the year he expected Microsoft to reach that point, so we assume the end, or June 2019.
Given that industry sources reckon Exchange to have more than 300 million users, this single data point says that
Microsoft aims to have circa 210 million active Office 365 users by June 2019. However, if we look at the growth in
monthly active users detailed above, that number is more likely to be in the range 175 million – 190 million.

The number of active Office 365 users reported in Microsoft’s financial results understates the total usage of Office
365 because it does not include accounts that Microsoft has provisioned for tenants but are not used yet, free
accounts (such as Microsoft's own use of the service), and trial domains. The numbers are even higher if we speak
about mailboxes instead of accounts. Few tenants do not use Exchange Online, and the number of cloud mailboxes is
considerably higher than the number of cloud user accounts because of shared mailboxes, inactive mailboxes,
resource mailboxes, and group mailboxes.

Earlier, Microsoft revealed that Exchange Online alone handles about 60 billion requests per day from all connected
clients (many people use more than one client, and each client handles multiple transactions). At the BUILD 2016
conference, Microsoft also said that Office 365 users create more than a trillion meetings monthly and that Office 365
had processed over 4 trillion outbound messages to that point. Another interesting statistic is Microsoft’s assertion
from May 2017 that 85% of the Fortune 500 use SharePoint Online.

Competition for Office 365

Google G Suite (formerly Google Apps for Work) is the bitter and enduring competitor for Office 365. In many ways,
Google set many of the standards by which we measure cloud applications today, including making a 99.9% SLA the
expected norm for service availability. Google also proved that the browser could be used as a fully-functional client
for common office applications such as word processing and spreadsheets and proved that users could depend on the
cloud to get real work done. Google exerts a huge influence over the development of Office 365 as Microsoft always
has one eye on what Google is doing.

Today, it’s probably fair to say that Google’s strength lies in the small-to-medium space while Microsoft’s is in the
enterprise, evidence of which is seen when companies like Facebook select Office 365 over Google. The two
companies compete aggressively in all markets on an ongoing basis, a factor that at least partly accounts for the rapid
pace of new feature introduction for applications. Moving from the original cloud focus on email, Google and
Microsoft both offer a range of cloud applications, summarized in Table 1-9 .

Microsoft Office 365 Google G-Suite

Email Exchange Online Gmail

Calendar Exchange Online Calendar

Team collaboration Office 365 Groups Groups/Google+

Yammer

Microsoft Teams

Personal and team file OneDrive for Business Drive

sharing

Document authoring Word Google Docs

Spreadsheets Excel Sheets

Presentations PowerPoint Slides

Audio and video calls Skype for Business Hangouts

Company Intranet SharePoint Online Team Sites Sites

eDiscovery Security and Compliance Center Vault

APIs Microsoft Graph and other service Wide range of application and administrative
APIs APIs

Table 1-9: Comparing Microsoft Office 365 and Google G-Suite

A simple table will always deliver a simple comparison. Table 1-9 doesn’t include some of the features available in
Office 365, including:

Delve.
Sway.
Microsoft Stream.
Microsoft Planner.
MyAnalytics.
Office 365 Cloud App Security.
Advanced eDiscovery.
Office 365 Retention Policies.
Supervision Policies.

Many observers say that Google does not have the same kind of enterprise-centric ecosystem developed around Azure
Active Directory (or directory services in general), including features like Azure Information Protection. Other areas
of comparison include:

Cost : The standard G Suite cost of $5/user per month is cheaper than any Office 365 enterprise plan (but not
Office 365 business essentials) while even the more expensive G Suite with unlimited storage and Vault
(archiving and eDiscovery) only costs $10/user per month. However, it is difficult to compare the Google plans
against the Microsoft plan because of the influence that the Office desktop suite has on pricing. If you want
Office, you’ll have to pay for it, one way or another. Another thing to take into consideration is the price of add-
ons from Microsoft or ISVs, which often increase the real average cost paid per seat. The low-cost entry point is
definitely a plus point for Google, even if Microsoft has a $4/month F1 (first-line worker) plan that serves
customers who want a low-cost solution. You can end up spending a lot more money on Office 365 licenses,
especially if you go for the E3 or E5 plans and add on some options such as Azure Active Directory Premium.
However, the point is that the price you pay should reflect the functionality needed by the business. If G Suite
basic includes enough functionality to meet those requirements, you’ll get it for $5/user per month. But if you
work in a complex multinational and need advanced eDiscovery, compliance, or want to host very large dial-in
teleconferences and prefer the Office suite, you will end up paying a lot more for Office 365 licenses. Discounts
are always a factor when customers sign up for large enterprise deals, so the list price is only a guideline for
starting a conversation about contracts.
Automation : The automation features in G Suite are weaker than in Office 365, where PowerShell delivers
 Microsoft an advantage for automating common management operations through scripting, while Flow,
PowerApps, and the Microsoft Graph APIs provide a common programmatic interface to data accessible through
multiple Office 365 endpoints, including Outlook, Groups, and Teams. However, Google does have Apps Script ,
which can be used to develop add-ons for Sheets, Docs, and Forms.
Clients : Both suites deliver good browser and mobile support for their applications across a range of devices.
Both support offline working, with Microsoft offering Outlook for email, calendar, and groups and OneDrive for
Business for offline access to documents, presentations, spreadsheets and anything else stored in a folder.
Google offers both Gmail and Inbox clients and supports offline access for Gmail, Calendar, and Google Docs,
providing you use the Chrome browser, which is a reasonable assumption to make if you opt for the Google suite.
In some respects, the Google apps enjoy an advantage because all their apps are cloud-based and designed to
work together in a very natural way. Google refers to their apps as “intelligent” with “real-time collaboration
built in from the start”. Microsoft was handicapped in this area because of the legacy of hundreds of millions of
users of Exchange, SharePoint, and Office restricted their ability to evolve to create better and more complete
integrations as quickly as they perhaps should have. However, applications like Office 365 Groups and the
gradual influence of Skype for Business across Office 365 proves that Microsoft can deliver an integrated
approach to collaboration and there’s no doubt that Microsoft absorbed some lessons from Google when they
built the online versions of the Office apps and new applications like Sway.
Mobile Device Support : Both suites provide methods to manage mobile devices using cloud consoles, including
functions such as the application of device policies and the ability to remote wipe devices. Apart from the basic
mobile device management provided in the Exchange Online Administration Center, Microsoft customers can buy
licenses for the Enterprise Mobility and Security product to gain additional capabilities.
Migration : Both suites provide migration utilities to help companies move data into the cloud. Microsoft’s
Office 365 Import service boasts the ability to ingest data from multiple sources, including PSTs, on-premises
SharePoint and file servers, and multiple third-party data sources such as Bloomberg.
Hybrid : Microsoft’s ability to support hybrid environments is a huge positive for enterprises who want to keep
some workload on-premises, including those who will eventually move everything to the cloud but need to move
gradually. Unified directories and the ability to transfer mailboxes quickly and easily from one environment to
another is a plus point for administrators who don’t want to conduct a big-bang migration. SharePoint also
supports hybrid scenarios to accommodate the gradual transfer of work from on-premises to the cloud, including
the ability to have on-premises content show up in unified searches.
Security and Privacy : Both suites expend huge effort to protect customer data and security, including when
data is in transit across the Internet. Both support protection and encryption (IRM) and Data Loss Prevention
(DLP), but Microsoft has an edge in both areas because of its application of IRM and DLP to non-email
workloads. Protection templates applied through email transport rules and encryption provide added levels of
control over how email and documents can be protected for internal and external use.
Data Sovereignty : The location of data at rest is increasingly a critical issue for businesses as they move to the
cloud. Microsoft runs a large network of datacenters globally for both Azure and Office 365, including in
important markets such as the U.K., Germany, Japan, Hong Kong, India, Canada, and Australia, with announced
plans to operate datacenters in France, South Africa, and other countries. Microsoft has also set down an
important principle where a local data trustee oversees operations in Germany. By comparison, Google’s
datacenters are mostly concentrated in the U.S. and Western Europe.

The decision about which cloud suite to use is a difficult one to make. If you use Microsoft on-premises server
technology today, the natural route forward is to Office 365, especially if user work habits are centered around
Outlook, Word, Excel, and PowerPoint. All the tools exist to make the transition as easy as possible and Microsoft will
help customers to migrate with their FastTrack program. In addition, Microsoft publishes a set of guides to help
customers understand how to move from the G Suite applications to their Office 365 equivalent.

G Suite Migration : Microsoft documents the process to move mailboxes from G Suite to Exchange Online , which is
based on using IMAP to extract data from Google and the Mailbox Migration service to move it into Office 365. In
November 2018, Microsoft announced that they are working on a more comprehensive set of migration services
based on Google’s REST APIs to cover email, calendar, and contacts. The new tools are expected in the second
quarter of 2019.

Leveraging the Breadth of Office 365

When Microsoft CEO Satya Nadella told the audience at the October 2014 Gartner conference that “Office 365 is the
new Exchange and one will cannibalize the other. The key is to ensure that current Exchange customers can transition
on their own terms ,” he reflected the reality that many Office 365 customers had selected email as the first workload
to move into the cloud. The reason why he called Office 365 “the new Exchange” is that the movement of email into
the cloud mimicked the movement of email from expensive mainframe and minicomputers to PC-based Windows NT
servers some twenty years previously. Exchange was the first Windows server application considered “enterprise-
ready.” In other words, Exchange could scale to offer the kind of email service needed by the world’s largest
corporations, a promise that the server delivered on to lay the foundation for a suite of other Office server
applications to build upon.

Exchange Online is a big reason why many companies decide to move workload to Office 365, but what of the other
cloud applications? After all, you pay for these applications alongside Exchange, so it is wasteful if you do not seek to
take advantage of them. Any plan to move to Office 365 should consider what advantages a company can derive from
the whole of Office 365 rather than applying a narrow email-centric focus. Here are the services that round out Office
365:
 SharePoint Online : Originally released as a departmental sharing and retrieval application in SharePoint Portal
Server 2001 (a version that shared the ESE database engine with Exchange), SharePoint has evolved over
several later releases to become a full-blown document management platform built on SQL Server. The
SharePoint API and features such as InfoPath forms and workflow management has enabled customers to build a
wide variety of applications on the platform. This is both a strength and a problem when it comes to cloud
transition because it is much harder to move an application’s code to SharePoint Online than it is to move a user
mailbox. Still, as time goes by, the process to plan for and to execute the move of on-premises applications is
becoming easier as new tools are available and more knowledge is acquired. The elimination of expensive on-
premises SharePoint farms is attractive from the economic perspective but expect this project to involve a lot of
effort and cost unless the company’s use of SharePoint is relatively unsophisticated. The first step is to
understand how SharePoint is used internally and to create an inventory of applications. This flows into an
assessment of who uses the applications and how to transfer the workload, including whether you need to run a
hybrid environment.
OneDrive for Business : OneDrive for Business is a direct replacement for personal shares on Windows file
servers. In other words, it is a network location where users can put their documents and other files rather than
storing them on the hard drive of their PCs. The storage used by OneDrive for Business is delivered by
SharePoint but some of the functionality is different, possibly to make OneDrive for Business more like its
consumer counterpart.
Yammer : Microsoft bought Yammer in July 2012 to gain its enterprise social networking technology. Since then,
Microsoft has integrated Yammer into Office 365 in several ways such as using Azure Active Directory for
identity management, connecting Yammer with on-premises SharePoint, and using the Office 365 Groups service
for Yammer Groups. Some commentators have said that Microsoft also acquired a fast-paced web-centric
development culture from Yammer that helped them in the transition to the cadence expected from cloud
services. That cadence exists today in the ongoing rapid release of features in Office 365. However, perhaps the
most interesting influence that has come from Yammer is the Graph technology, a project that originated as the
Enterprise Graph inside Yammer and was in turn derived from the Open Graph work done at Facebook.
Applications reference signals held in the Office Graph to highlight popular files, videos, and groups within
applications and as the basis for the information surfaced in Delve and MyAnalytics. Yammer, like email servers,
is all about sharing information. If your company uses Exchange for this purpose you might not want to use
Yammer to recreate the wheel. On the other hand, Yammer does things differently and it might give an answer to
improve internal communications and idea sharing. For this reason, it is a good idea to include an analysis of
Yammer along with the other methods for collaboration available in Office 365 in your migration plans.
Microsoft Planner and Microsoft Teams are two applications available to enterprise Office 365 tenants. Both use
Office 365 Groups to give a team identity and to manage team membership. Planner focuses on task-based
planning for team activities while Teams offers a chat-based workspace for users to work together. According to
Microsoft, over 200,000 organizations use Teams, so it is experiencing a rapid uptake. Chapter 13 describes
Teams and Chapter 15 Planner.
Skype for Business Online : The long-term direction for Skype for Business Online is to transition the client
functionality to Teams and the server functionality to the Microsoft Phone System. The transition takes time and
needs careful planning. See Chapter 16 for more information. From October 2018, new Office 365 tenants with
fewer than 500 seats must use Teams and do not have the choice of using Skype for Business Online.
Delve : All about exposing information that exists within a company, the current iteration of Delve does not
expose information held in Exchange mailboxes or public folders because the Office Graph does not use the
content held in Exchange databases when it looks for the connections that exist between users in a company.
However, if your company makes heavy use of SharePoint Online document libraries to store information used by
teams, Delve can do an excellent job of making documents more visible to people who have access to the
documents but might not realize that the information exists. Microsoft has introduced components to surface
more on-premises data in Delve, notably for SharePoint, but Delve cannot yet access the contents of on-premises
Exchange mailboxes.
Office 365 Video /Stream : Announced in November 2014, the Video service is a portal built on top of
SharePoint Online intended to provide companies with a single place to store and manage video content for
internal consumption. Quite how many companies need a YouTube equivalent for internal videos is open to
question, but it is indicative of the way that Microsoft positions Office 365 as the one-stop shop for all types of
content used within companies. Office 365 Video went live in mid-2015. Stream will eventually replace Office 365
Video, with the transition expected towards the end of 2018. All Office 365 commercial tenants now have access
to the advanced features of Stream. See Chapter 7 in the companion volume for more information.

Perhaps the most interesting technical aspect of Office 365 is how the barriers that exist between different on-
premises applications disappear when Microsoft has total control of deployment and operations. Relatively few on-
premises environments successfully integrate SharePoint and Exchange, but the configuration and operational
difficulties go away with the cloud versions. Microsoft takes advantage of this to build new applications that exploit
Exchange Online for email and calendaring and SharePoint Online for storage, adding whatever extra software is
necessary to complete the picture. Two examples are Office 365 Groups and Microsoft Planner. Both are unique to
Office 365 and do not exist in even the most up-to-date versions of the on-premises software. As such, you must
migrate workload to the cloud before people can use applications built deliberately to exploit the unique
circumstances that exist within Office 365.

If the right licenses are in place, all this functionality is available to Office 365 tenants. Lots of companies plan to
move to Office 365 to replace an existing email server and can benefit even more by exploiting the other technology
mentioned above. The best thing is that Microsoft takes care of all the work to deploy and manage the technology,
meaning that the normal learning curve needed to master the details of capacity planning, server deployment, and
application set-up is not needed. All effort can focus on the question of how best to use the features available through
SharePoint Online, Skype for Business Online, Delve, and Video to solve business problems. Companies that have
been using on-premises SharePoint and Skype for Business will find that their experience and knowledge will
transition to the cloud, subject to the caveats outlined above.

Like an Exchange migration, the migration of on-premises SharePoint and Skype for Business to their cloud
equivalents are projects that benefit greatly from experience. If you don’t have the necessary expertise on staff, you
should find some experienced consultants to help you plan the transition.

Will we move?
There is no doubt that Office 365 is impressive in terms of its physical infrastructure, feature set, and market success.
A time will come when most common office application workloads run in the cloud, and a lot of this work will happen
inside Office 365. Companies that currently run on-premises deployments have a real choice to make: should they stay
with on-premises software, embrace the cloud by going “all in” with Office 365 by themselves or using the assistance
of Microsoft or another partner, or implement a hybrid environment where some processing remains on-premises and
some moves to the cloud. Each company will have their own set of factors to consider as they make the choice, so that
seems to be a reasonable topic for discussion in Chapter 2.
Chapter 2: Making the Decision to
Embrace the Cloud
Tony Redmond

No one likes making an ill-informed decision, especially when that decision is fundamental to a facility that a company
depends on. Email often falls into this category. Deciding to move from an on-premises implementation to the cloud,
including the possibility of creating a hybrid environment, is a fundamental step. Companies have their own reasons
to consider moving to the cloud. Sometimes it is because their IT infrastructure is old and unsupported, as in the case
of those running now unsupported servers. Sometimes it is because they face the need to perform a complex upgrade
to the latest version of on-premises server technology coupled with the deployment of new server and storage
hardware, and the choice to use Office 365 seems easier and more straightforward. And sometimes it is because
people consider applications like email and document management to be utility functionality that is better bought
from Microsoft instead of being run in-house.

In this chapter, we consider some of the common arguments advanced when people go through the intellectual
process of figuring out whether on-premises or cloud technology is best for them. Like any other case advanced about
technology, it is critical that you put these arguments in context for your company. Your knowledge about how the
company works and its business goals will help you put the points made here into context. Never take a consultant’s
opinion as definitive!

Removing Cloud FUD

The use of fear, uncertainty, and doubt is a much-beloved tactic to delay or deflect a decision away from certain
technology. Salespeople have used FUD since the mainframe era to discourage the adoption of PCs, mini-computers,
client-server implementation, the web, social networking, mobile devices, and now the cloud. Some of the fears
expressed in the past are still valid concerns and should be considered when any decision to move to a platform like
Office 365 is contemplated; other fears are over-hyped.

Common doubts include:

You can’t depend on the cloud (or the Internet).

Cloud services are immature.
Cloud platforms are insecure.
On-premises IT delivers superior service to the company and end users.
You lose control when you move to the cloud.

It’s likely that other concerns will arise. After all, we are talking about technology. In each case, it is important to
separate fact from fallacy. Let’s look at each of the issues listed above.

Dependability
“You can’t depend on the cloud” can have several meanings. It could be that a lack of trust exists that users can
connect to cloud services as easily as they can with on-premises services. It could also mean that you think your IT
department can deliver a more robust service than you can get from a cloud provider.

There is no doubt that the network is the essential link between cloud services and end users. Three parts come
together to deliver end-to-end network connectivity: the internal network used within your company, including the
connection to the internet; the internet; and the network controlled by the cloud provider. As explained later, the
performance and throughput of any company network that predominantly focuses on in-house systems usually needs
upgrading and some tweaking to accommodate Office 365. This is logical – traffic that once flowed to on-premises
Exchange and SharePoint servers now goes to their online equivalents. In addition, you can expect more external
network traffic from by tasks such as hybrid connectivity, directory synchronization, and remote administration, not to
mention the transfer of data from on-premises to cloud platforms (and possibly vice versa). Moving work off old file
servers to SharePoint Online and OneDrive for Business also creates traffic, as does the synchronization of documents
back to user workstations. New applications like Teams and Planner can create further demand.

Anyone who plans to move to Office 365 needs to pay attention to their internal network and to their Internet
connectivity to ensure that enough high-quality network resources are available to handle the expected load. For
instance, is enough capacity available to handle the outbound traffic to Office 365 or how many access points exist to
process internet traffic? It is often a project management and budget issue rather than a technical challenge.

You can upgrade and revamp an internal network, but you can do little about the internet. It is true that some
locations are still badly connected to the Internet (the notion of a stretched string comes to mind), where the
connections are barely able to cope with some web browsing and transmission of email back to a central in-house
server. If a company has people working in remote offices where only poor network connections are available, then
cloud services are probably a bad choice. This situation might well change as the focus on user access shifts away
from PC to mobile devices and mobile networks take the place of traditional WAN connections to offices.
However, it is fair to say that Internet connectivity is improving all the time, and this should become less of a factor
over time. I have connected to Office 365 using different PCs from multiple locations all around the world and never
had much of a problem, even if I had once to resort to running the basic version of OWA through the browser of a
Linux-powered smart TV in a hotel in Abu Dhabi. Office 365 applications still worked, albeit slowly. Connecting to
Office 365 when you are on the road is not an issue if you can get to a Wi-Fi access point and connect to a public (free
or paid) network.

Microsoft accelerates user connections to Office 365 through a network of connection points designed to transfer
inbound connections into their network as quickly as possible. This is not an unusual situation as services like
CompuServe used similar points of presence (local telephone numbers) several decades ago. Once inside Microsoft’s
network, traffic flows across connectors with enormous capacity to the Office 365 datacenters. Microsoft experiences
occasional glitches in this network but enough capacity and alternate network paths exist to handle most issues.

The second aspect of dependability can be captured as the quality of the service delivered by Office 365. Let’s talk
about the Service Level Agreements (SLA), the formal agreement that Microsoft makes with Office 365 customers to
describe what they will deliver and how they measure the reliability of their service.

Maturity of Cloud Platforms

The second issue often raised by cloud doubters is the assertion that the platform is immature. In other words, the
technology used in on-premises deployments is better understood because it has been used for so many years and all
its challenges have been met and overcome. It is true that on-premises technology is a well-understood art and that a
huge amount of documentation and advice exists to offer best practice for various applications, operating systems,
and other aspects. However, it is also true that on-premises deployments suffer from inconsistency. Some are excellent
and exhibit all the desired characteristics of attention to detail, superb execution, operational maturity, and
dependable delivery of service to end users. On the other hand, there are many on-premises deployments that do not
function as well as they might and will take some work to improve before they can consider moving to any other
platform, whether that is to use a new on-premises version of Exchange, a hybrid deployment, or moving everything to
the cloud.

Cloud services tend to run at massive scale, something that is impossible unless all the characteristics listed above
are present. Although automation helps to improve reliability, datacenter operations have not typically posed a
problem for cloud services. Software has been more challenging. Specifically, software that was not designed or
written to handle the demands of massive scale, multiple tenants, data isolation, security, and global distribution. As
the engineers discovered, it is one thing to create an enterprise application like Exchange 2003 that was quite
capable of handling the demands of what was then the world's largest Exchange deployment (in the order of 500,000
on-premises mailboxes); it is quite another matter to come up with software that delivers the same functionality to
tens of thousands of tenants spanning tens of millions of mailboxes distributed around the world. Hiccups occurred
along the way because the software was immature in a cloud sense, but the technical innovation and advances that we
have seen in products such as Exchange and SharePoint as well as the evolution of Windows (without PowerShell it's
hard to know how Microsoft's commercial cloud platforms could run) over the last ten years have created what we
know as Office 365 and Azure today.

Although some might have questioned the wisdom of moving workload to services like Office 365, you can’t argue the
results. Cloud platforms deliver highly functional software in the form of highly accessible applications across the
Internet and do so with a level of reliability and security that would be the envy of many CIOs responsible for internal
IT systems.

Security of Cloud Platforms

No organization will move to the cloud if they believe that their data will be less secure than it is on-premises. The
same is true for data privacy, as no one wants their information to be exposed to others, including government
agencies. Microsoft is fortunate that some of the world's leading security experts work there, including those whose
expertise lies in breaking into computer systems. That expertise is used to construct the barriers that protect
customer data within Office 365.

Microsoft’s official stance is that Office 365 workloads are designed and implemented per the Microsoft Security
Development Lifecycle . This is described as “a software development process that helps developers build more
secure software and address security compliance requirements while reducing development cost. ” On a practical
level, this means that data is secured by being encrypted at rest, that auditing information is available, and that the
Office 365 management activity API is supported to allow customers and third parties programmatic access to
information protection and compliance events. For instance, BitLocker is used to protect Exchange Online databases
(see below) while files stored in SharePoint Online and OneDrive for Business libraries are individually secured with
their own 256-bit Advanced Encryption Standard (AES) key.

Apart from some broad guidelines, Microsoft does not usually provide specific details about the methods used to
protect information in Office 365 because that could potentially undermine security by revealing too much to those
who might contemplate attacking the service. However, a reasonable amount of information, including several white
papers that address the security topic in depth, is available in the Office 365 Trust Center and details of the data
encryption technologies used by the different Office 365 workloads is available online . The completeness of the
security features developed for Exchange Online and Office 365 in general was enough to warrant inclusion of
Microsoft in the “leaders” section of Gartner’s 2015 Magic Quadrant for Secure Email Gateways, a status Microsoft
has held since.

The methods used to protect Office 365 data include:

 Physical security : Anyone who has ever visited a Microsoft datacenter can attest to the extremely high level of
security processes and procedures that apply. Access is strictly controlled to the servers and other hardware and
every action is verified and logged. Any faulty disk drives are removed and demagnetized before they are
destroyed to ensure that no possibility exists that customer data might be compromised.
Very few Microsoft employees have persistent administrative access to applications or the servers running in the
datacenters. Although this means that it can sometimes take Microsoft longer than you anticipate to resolve
support calls, restricting access ensures that any interaction with customer data is controlled (and audited). The
Lockbox process ensures that administrators hold zero standing permissions, which removes the problem of how
to control the growth of those holding elevated permissions over time. Administrators receive just-in-time
permission to perform tasks and have the permissions removed when the task is complete. A similar approach
allows customer control over requests for access to tenant data. Any time an Office 365 engineer needs access to
tenant data, a request appears in the Office 365 Admin Center for a tenant administrator to review and authorize
(or deny). The idea is to give an added level of customer control over their data, hence the name "customer
lockbox". The Customer Lockbox feature works for Exchange Online, SharePoint Online, and OneDrive for
Business data and is bundled in the E5 plan. You can also buy Lockbox as an add-on for any Office 365 enterprise
plan.
Microsoft audits all access to tenant data and takes great care to ensure that data cannot leak from one tenant to
another (see this document for more information about how tenant data is isolated within Office 365).
Transport Layer Security (TLS) encryption protects information sent between the Office 365 datacenters and
clients and messages sent between Exchange Online tenants. You can also create transport connectors that use
TLS to protect messages sent to specific domains, such as those belonging to trusted partners. Any client,
including browsers, that connect to Office 365 must use TLS 1.2 as Office 365 services will not support
connections using earlier versions of TLS from October 31, 2018 .
Data at rest with the Office 365 datacenters is protected by BitLocker . Protection covers both Exchange Online
mailbox databases and SharePoint Online and OneDrive for Business document libraries. These steps ensure that
rogue administrators cannot remove customer information from Office 365 to sell it on to other parties or
otherwise misuse the content.
Users can choose to protect or encrypt messages using Azure Information Protection, or S/MIME.
To be complete, you might include features such as Data Loss Prevention (Chapter 22), Advanced Threat
Protection (Chapter 17), and Office 365 labels (Chapter 19) in the overall assessment of security capabilities.
Office 365 depends on Azure Active Directory, which has its own set of protections. For more information, see
this whitepaper .
Privileged Access Management allows tenants to control administrative access to PowerShell cmdlets that create
or edit data. Administrators must request access to specific functions and give a reason for the access. After
review, the access might be approved, in which case the administrator can go ahead and execute the task. The
first version of Privileged Access Management covers Exchange Online, with plans to cover other workloads over
time. Privileged Access Management is an Office 365 E5 feature.

Table 2-1 summarizes the different encryption technologies deployed to protect data in different Office 365 workloads.

Encryption Workload Algorithm Key management FIPS 140-

technology protected 2 Level 2
validated

BitLocker Exchange AES 256-bit Windows Certificate Store. The credentials to Yes
Online access the store cannot be obtained without
mailbox elevated approval within Office 365 management.
databases All access is logged.

SharePoint AES 256-bit As for Exchange Online Yes

Online
databases

Skype for AES 256-bit As for Exchange Online Yes

Business
Online

Per-file SharePoint AES 256-bit Master keys used to protect the per-blob keys are Yes
encryption Online held in two locations: the secure store (native to
(including SharePoint), and the secret store. The keys are
OneDrive for updated every 60 days.
Business)

 Skype for AES 128-bit Each piece of content is encrypted using a different No
Business randomly generated key. The key is stored in a
Online corresponding metadata XML file that is encrypted
by a per-conference master key, which is randomly
generated for each conference.

TLS between Exchange Opportunistic TLS 1.2 The TLS certificate for outlook.office.com is a 4096- Yes
Office 365 Online using an AES 256-bit bit SHA256 RSA certificate.
and clients cipher.

SharePoint The TLS certificate for *.sharepoint.com is a 4096- Yes

Online bit SHA256 RSA certificate.

Skype for TLS for SIP The TLS certificate for *.lync.com is a 4096-bit Yes
Business communications and SHA256 RSA certificate.
Online PSOM data sharing
sessions

TLS between Exchange TLS 1.2 with AES 256 Microsoft uses an internally managed and deployed Yes
Microsoft Online, Secure Real-Time certification authority for server-to-server
datacenters SharePoint Transport Protocol communications between datacenters.
Online, and (SRTP)
Skype for
Business
Online

Azure Rights Exchange Supports Cryptographic Managed by Microsoft or by customers through a Yes
Management Online, Mode 2. RSA 2048 is bring-your-own-key (BYOK) process. Thales HSM
Service SharePoint used for signature and devices protect the keys. See Chapter 24 for
Online, and encryption, and SHA- information about some significant reduction in
OneDrive for 256 for signature hash. Exchange Online functionality when BYOK is used.
Business

Table 2-1: Encryption technologies used to protect Office 365 workloads

Although the information presented in Table 2-1 is correct at the time of writing, this is an area that is prone to
change as technologies evolve to deal with new threats. Some differences also exist in the implementation used to
support government tenants to satisfy specific requirements. The data used by applications built on top of individual
workloads are protected by the arrangements put in place for the different workloads. For example, the data used by
Office 365 Groups are protected by Exchange Online and SharePoint Online.

The automated workflow engine for Office 365 operations described in Chapter 1 ensures that servers run known
configurations. The automated and ongoing updating of servers with new versions of the operating system and
applications means that a high degree of confidence exists that known bugs are not met in production.

Industry Standards
None of the practices used by Microsoft are rocket science. Although implemented at massive scale and with huge
attention to detail, the practices followed to protect and secure data inside Office 365 are well-known and can be
implemented by any company (and indeed, Microsoft documents how they handle security incidents reported for
Office 365). The one exception to this statement is found in the background knowledge within Microsoft about the
applications and environments running within Office 365. This information gives Microsoft an advantage when it
comes to protecting information during operations and is clearly not available within on-premises customers.

Of course, reassurance gained by consulting a web site is not really what you need. Independent auditing is done to
ensure that Microsoft adheres to standards such as:

ISO 27001/27002 : Information Management Security System and Best Practice for Security Controls.
SSAE 16 SOC1 Type II : Reporting on Controls at a Services Organization.
NIST 800-53 : Security and Privacy Controls for (U.S.) Federal Information Systems and Organizations.,
Microsoft’s audit controls for Office 365 are based on the NIST 800-53A (Rev. 4) special publication. The results
of the external audit showing how Microsoft’s internal control system meets this standard are available in the
Audited Controls section of the Security and Compliance Center.
HIPAA : U.S. Health Insurance Portability and Accountability Act.
 The HITRUST Common Security Framework.

A complete set of documents describing how Office 365 meets regulatory requirements and security standards can be
found in the Office 365 Service Trust Portal , including the latest audit reports covering Office 365 and other
Microsoft cloud properties. Another set of documents helps customers perform a risk assessment and understand the
compliance of Microsoft cloud services with industry standards and regulations. Microsoft sets out its approach to
securing online services in documents such as the Operational Security for Online Services Overview to help
customers understand the policies that it uses to secure Office 365 and other Microsoft commercial cloud services like
Dynamics CRM and Azure. Microsoft also publishes information about how Office 365 meets requirements that exist
in specific geographies, such as the European Union model clauses , which are “standardized contractual clauses used
in agreements between service providers and their customers to ensure that any personal data leaving the EEA will
be transferred in compliance with EU data protection law and meet the requirements of the EU Data Protection
Directive 95/46/EC .”

The European Union’s General Data Protection Regulation (GDPR), effective from May 25, 2018, affects how
companies active in the EU gather and handle data. More information on GDPR and how it relates to Office 365 is
available from Microsoft in the service trust portal and in Chapter 19. The Microsoft Compliance Manager gives
customers a framework to help them organize the steps to achieve compliance with regulations like GDPR.

In addition, industry-specific reassurances are also sought, such as the white paper authored by a U.S. law firm to
outline their opinion why the compliance features built into Exchange Online meet the data retention requirements of
rule 17A-4 of the U.S. Securities and Exchange Commission (SEC).

Office 365 does not exist in a vacuum and it is important that the other components that services like Exchange
Online depend on also receive external oversight and certification. For example, Azure is certified against the ISO/IEC
27018 code of practice for storage of personally identifiable information in public clouds.

Security for more than Office 365 : Although Microsoft assigns talented personnel to the tasks of building
security solutions and in protecting customer data within Office 365, there is no doubt that sometimes some extra
help is necessary for organizations to achieve full protection. For example, Office 365 offers tenants the chance to
deploy data loss prevention technology for many workloads, but this technology does not protect all the data that
exists within companies. It might therefore be necessary to deploy some other solutions to ensure that sensitive data
is not transmitted outside the organization. When a company moves workloads to Office 365, it is wise to consider
whether some more protection is needed for the full spectrum of data that is in use.

Companies who need more security than the basic suite delivered in Office 365 have a wide range of options available
to them to harden different components. Microsoft Cloud App Security is available to protect cloud applications at an
extra monthly cost per user. Other companies who are active in the provision of additional security capabilities for
Office 365 include Mimecast , ForcePoint , Proofpoint , and Skyhigh Networks .

Quality of Cloud Services

Quality can be measured in diverse ways. We have already discussed Office 365 performance against the published
Service Level Agreement and shown that Microsoft has an excellent track record in this respect. Software flaws is
another measurement. In other words, does Office 365 deliver high-quality software that works all the time? No
software is flawless, and Office 365 is no different. Bugs exist and become known all the time. Often people don't
understand why this is the case and wonder why Microsoft can't stabilize Office 365 so that all bugs are eradicated,
and functionality works as documented all the time. Although it would be nice to run services in a bug-free
environment, it's not going to happen if four factors exist. These are:

Competitive pressures : Microsoft does not run Office 365 in a vacuum and must respond to technical and
feature advances made by its competitors to ensure that Office 365 stays an attractive offering in the market.
Once you touch code to improve functionality or "make things better", the possibility exists that bugs creep in.
Customers demand more functionality : Customers also ask for new features in many different areas of Office
365. Requests vary from a slight change to the way some feature works to an update designed to facilitate the
onboarding of large enterprise customers. Microsoft monitors support tickets closely to focus in on areas that
cause users to have problems (such as deleted item recovery) and makes changes to improve matters.
Different clients are in use : A staggering number of client platform combinations (browser, Windows, mobile)
and protocols interact with Office 365. Clients have been known to introduce problems (for example, several
Apple iOS upgrades have caused issues for Exchange), and the support and testing matrix is very complex.
Testing complexity creates more possibility for bugs to creep into software and stay undetected until a user
meets a problem.
Software interaction : Although email is often the first workload moved to Office 365, it is important to view
the service as not being a single workload. A better and more productive view takes in Exchange Online,
SharePoint Online, Skype for Business Online, Yammer, Azure Active Directory, Azure Information Protection, the
Microsoft Graph, Delve, and so on. Behind the scenes, a complex set of interconnections link Exchange Online
and SharePoint Online (for features like Office 365 Groups), Exchange Online and Teams (so that conversations
are captured for compliance), Exchange and Delve (so that attachments show up in Delve views), and so on.
SharePoint Online is the storage for functionality such as the Office 365 Video Portal while Azure storage
supports other applications like Planner and Teams. Each of the software applications has their own set of
developers, testers, and plans. Maintaining the connections between the different applications can take
substantial effort, which is one of the reasons why some of these features are not available on-premises.
We will never be rid of bugs when software evolves and flexes all the time. We accept this situation because we like
new features and functionality. Remember that one of the reasons why customers adopt cloud services is the promise
of "evergreen technology", to move away from the more restrictive upgrade cycles used in on-premises deployments.
With evergreen or ever-evolving technology comes the risk that things don't work quite as well as they should from
time to time. That's when you enter the world of cloud support.

Office 365 Support

Microsoft gives free support for Office 365 tenants. Support varies from forum-only to telephone support to online
requests for support that flow to dedicated support personnel. Support is often referred to as the “Achilles Heel” of
cloud services because it is so difficult to deliver support in a timely and effective manner to tens of millions of
consumers. By comparison, on-premises services are often supported by in-house help desks and support teams. A
company has control over the quality of those services based on the investment made to staff up the help desk and
train support personnel. The transition from a position where you have total control over all the moving parts needed
to resolve a support ticket to depending on the intervention of cloud support, when you really do not control a lot, is
difficult for many organizations. IT professionals are frustrated with having to deal with phone support, users do not
like how long it takes for even simple issues to be resolved, and managers worry about the control they have ceded to
the cloud provider.

All of this is true. When you move from a customized in-house solution to use a shared utility infrastructure you must
make some adjustment. Think of it like this – if you use a diesel-powered generator to power your house, you have
total control over the infrastructure and offer all the support. But when you sign up with a power company, you hand
over responsibility for most of the operations to that provider. All you must do is make sure that equipment is plugged
in and turned on. The same is true of using a utility IT service. You don’t have to maintain the servers or make sure
that their networking equipment is functioning, but you do have to ensure that your own equipment is functional.

The horizon for in-house support provided by an IT department changes when workload transitions to the cloud.
Instead of driving problems to resolution, you now manage the interaction with Microsoft to work the problem as
effectively as you can within the constraints placed on both you and the Microsoft support organization.

You control local equipment and settings and can make sure that you cover all the basics – that the user’s client is
configured correctly, that they can connect to the Internet, whether the problem is unique to a certain user or is
shared by a group. Microsoft support is delivered remotely and will usually be invisible to the end user unless they
learn about it when they are asked to check some local setting or variable on their PC. Microsoft owns the
responsibility for recording the support ticket and following through until the issue is finally resolved. But you should
remember that Microsoft support professionals cannot make changes within an Office 365 datacenter. All they can do
is record, probe, and diagnose problems. Sometimes that will be enough to get a resolution and sometimes they will
need to escalate from first-level through second-level right up to the elevated heights of the few who can work directly
with servers, if that’s what is required to fix a problem.

Working a problem through many interactions with different levels of support takes time. Multiple phone calls and
email interchanges are likely going to be necessary to keep an issue moving toward resolution. Don’t blame the
support personnel – they follow a strict playbook to gather information, suggest fixes, and escalate when required.
Think of how you’d function if you had to understand the myriad different circumstances that customers who report
problems are in. It sometimes takes a lot of time to simply understand what’s going on before you can move toward
resolution.

Microsoft uses a mixture of subcontractors and Microsoft badged employees to provide first level support for Office
365 ( access is described here ). First level support is designed to provide first response to incoming calls. The calls
are registered in Microsoft’s ticketing system and a structured approach is taken to recording details of the incident
that can possibly lead to a fast resolution if the problem and its solution is already known. Calls that cannot be
resolved by first level staff are escalated to second level support. These individuals have deeper product knowledge
and are better able to diagnose and resolve complex support situations. Because Microsoft operates a DevOps model
for Office 365, the most difficult calls eventually escalate to the product group responsible for the area where the fault
lies. At all times, customer confidentiality and data privacy are respected.

When you report a problem with Office 365 to Microsoft, it is critical that you keep careful notes of the date and time
for each call or other contact and record to whom you spoke, what was discussed, what actions should occur, their
expected outcome and when this should happen. Apart from simply being good sense to record information about
support incidents, this information will be invaluable if you need to request an escalation or wish to review why a
problem is not being resolved, perhaps by contacting the supervisor of the support professional with whom you are
working.

Communication with end users is critical. They probably won’t realize the details of the complexity of cloud services,
nor should they be expected to understand the links between the client software that someone wants to use to
connect to Office 365. Making sense of the complexity and managing support calls is the new role for the local IT
department.

Limited responsibilities : Microsoft makes it quite clear what they will and will not support when it comes to Office
365 calls. The normal guidance is:

"Escalation scenarios that the Office 365 support team will not accept include but are not limited to the following:

Your networking infrastructure.
Your hardware.
Microsoft on-premises software that is not part of the Office 365 service offering.
Non-Microsoft software.
Your operational procedures.
Your architecture.
Your IT service management process errors, system configuration errors, or human error."

The Question of Losing Control

Consumers of cloud services do not get to vote about when functionality is introduced or changed. It just happens,
and it is done by the cloud provider to meet their goals and priorities rather than yours. This is part of the contract
that you sign and the control that you cede when you decide to start using cloud services instead of your own on-
premises service. It is just like what happened when people stopped using their own electricity generators or water
pumps and connected instead into the public electric and water utilities. They lost the control over how their
generator or pump worked when they traded it for the promise that the service delivered by the utilities would be
better, more predictable, and cheaper. Although they could no longer decide on questions such as what fuel is used by
the generator or how long a pump ran daily, they still had access to power and water.

Email is very much a utility at this point. SharePoint is less of a utility, but only in parts. Document libraries are a
utility, bespoke applications built on top of SharePoint are not. Skype for Business Online is a utility and so is Yammer
or Delve. Moving to use cloud-based versions of these applications is very much like making the decision to use a
public utility. And yes, a decision to move to Office 365 means that some control is ceded, but most people can
cheerfully give up the responsibility for installing the latest versions of Windows and Exchange on a server, bringing it
up to the latest patch level, and making sure that any required additional products are installed.

What can be more problematic is when the characteristics of the service change in a way that you prefer it did not.
For instance, in February 2015, Microsoft announced that the contents of the Deleted Items folder would no longer be
cleared out periodically by the Managed Folder Assistant. This was a great decision for small tenants who often care
very little about compliance, but it affected the compliance strategy of many large tenants, especially those who run
hybrid environments because different methods of processing the same retention policy then existed on the two
platforms. Microsoft made the decision to stop removing items from the Deleted Items folder for their own good
reasons, but in effect, this is an example when the interests of small tenants trumped a business and regulatory need
of enterprise tenants. However, Microsoft made sure that the change was easy to reverse – if you realized that it had
been made.

Another example is illustrated by the situation that occurred for Delve in June 2015. Microsoft found an error in the
code that loaded user profiles and acted to fix the problem. The side effect was that the Delve user interface reverted
to an earlier version, a change that caused disruption for end users and administrators alike. The problem lasted for
five days before the user interface was restored. No warning was given to tenants that the problem had happened and
what its effect would be.

The positive way of looking at how things flex and change without warning or any ability to influence an outcome
when you use cloud services is to not focus on any loss of control over the servers and other infrastructure that you no
longer manage. Instead, focus on how best to use all the added hours that you gain and figure out how to use that
time in a more productive manner.

Understanding Service Level

Agreements for Cloud Services
Two formal documents govern the provision of Microsoft Online Services to customers. Both documents are updated
quarterly. The Volume Licensing Online Services Terms lay out the terms under which Microsoft delivers the service
within Office 365. The document is available in multiple languages. More information is found in the Service Level
Agreement for Microsoft Online Services , also available in multiple languages. This document explains how Microsoft
measures the SLA for its Online Services.

Each of the individual workloads running inside Office 365 has its own SLA definition. For Exchange Online, Microsoft
defines downtime to be “Any period of time when end users are unable to send or receive email with Outlook on the
web. ” The actual calculation is a little more complex:

“The “Monthly Uptime Percentage” for a Service is calculated by the following formula:

((User Minutes – Downtime)/User Minutes) * 100

where Downtime is measured in user-minutes; that is, for each month, Downtime is the sum of the length (in minutes)
of each Incident that occurs during that month multiplied by the number of users impacted by that Incident .”

Not all users are affected by each incident, so the number of user minutes generated by an incident varies. Many of
the incidents noted by monitoring products are transient and last just a few seconds and can be put down to a
temporary condition that occurred somewhere along the path between client device and server running in an Office
365 datacenter. Over the course of a working day, micro-outages can happen five or six times without causing
disruption to end users. Outlook’s cached Exchange mode, for instance, enables users to continue working when the
server is temporarily unavailable and most mobile devices will reattempt to connect after a short period. These micro-
outages can sometimes be detected as a failure of a page to load or a server to respond in a timely manner. They
usually resolve themselves without the need to intervene on the part of either Microsoft or the tenant administrator.

Given the massive number of users who access the Office 365 services, it is fair to say that an incident that lasts
several hours but only affects a small group of users (for example, in one Office 365 datacenter or region) will not
have much effect on the SLA while a global outage can dramatically influence the SLA because so the incident
consumes so many user minutes. Due to the distributed nature of the infrastructure, incidents usually stay
constrained within a single Office 365 datacenter or even a single Office 365 region. Even the 7-hour Exchange Online
outage from July 2014 that affected North American tenants went by without the rest of the world noticing. The last
truly global outages occurred in September 2011. High-profile but region-constrained outages like those that affected
U.S. tenants in June 2014 and July 2015 seldom move the SLA needle very much. Indeed, a daily glance at the Office
365 service dashboard will invariably show that at least one of the applications is currently experiencing some level of
difficulty that might or might not affect your users. This is part of the joy and the pain of sharing in a massive multi-
tenant infrastructure.

If the reported Monthly Uptime Percentage falls under 99.9%, Microsoft guarantees that they will refund customers
with a service credit for a set amount ranging from 25% to 100% (for less than 95% availability).

An SLA of 99.9% (“three nines” in high availability parlance) allows a service provider to have downtime of up to 43.8
minutes per month, or 8.76 hours annually. As explained above, lost minutes for Exchange Online accrue when users
can’t send or receive mail with OWA. For SharePoint Online, it is “any period of time when users are unable to read or
write any portion of a SharePoint site collection for which they have appropriate permissions, ” while for Skype for
Business Online it is “any period of time when end users are unable to see presence status, conduct instant messaging
conversations, or initiate online meetings .” These conditions are documented in the Service Level Agreement for
Microsoft Online Services.

Counting minutes for SLA calculation only happens when Microsoft support records an incident and accepts
responsibility for the problem. In most cases, this means that the problem has to happen within a Microsoft
datacenter to a component that is under Microsoft’s control. Anything that happens within the control of the
customer, such as a misconfiguration of client software or a local network malfunction, does not count against the
SLA. This is logical because Microsoft can only be blamed for the parts of the service that it controls.

Microsoft is not the only cloud provider to limit SLA measurement at the boundary of its datacenters. No one controls
the Internet, and no one controls how data flows from your client to a datacenter. Many complex steps occur between
a client connecting to a cloud service like Office 365, perhaps using multi-factor authentication from a mobile device,
to being able to access and interact with data. It is therefore impossible for a provider to offer an SLA guarantee as
measured at the client. Well, perhaps possible because anyone can offer such an SLA, but certainly foolish in terms of
their chances of ever meeting the SLA.

Microsoft excludes scheduled downtime from any SLA calculation. This is downtime that Microsoft needs to maintain
its service and is advised to customers at least five days before it happens. On the other hand, any sudden outage that
comes without warning always counts against the SLA, if Microsoft accepts that the incident is caused by their
software or infrastructure and did not happen because of an action taken by the tenant.

At any single time, there are multiple incidents unfolding within Office 365. Some of these are very local and affect a
single server. Some are more widespread and degrade the service delivered to large numbers of tenants or even stop
an application working for those tenants. Some incidents are transient and go away on their own accord and some
linger for days, albeit perhaps without affecting tenants. The point is that an infrastructure that is so large cannot run
in a perfect state all the time. Because Office 365 depends on hardware, software, and humans, you can be sure that
something is always happening for the wrong reason. Of course, users will not care unless an incident stops them
sending email, accessing documents, conducting a teleconference, or some other operation.

In most cases, any issue that affects the SLA is soon obvious because users are unable to connect or to do work.
Microsoft has automated systems in place to interpret problems and map them against tenants so that the incidents
shown up in the Office 365 Admin Center are those that might affect smooth operations for your tenant. Details of
incidents appear in the Service Heath Dashboard (SHD). We discuss how the understand the SHD, navigate within it,
and file service incidents for your tenant in Chapter 4.

A summary of incidents for the last 30 days is also available in the Office 365 admin portal together with post-incident
reports (PIRs) by going to Service Health and then selecting View history . However, it is easy to forget to check the
portal from time to time to see what the status across all parts of the service. Unless you run a third-party product
designed to automate the health state of applications such as Exchange Online and SharePoint Online, it is possible
that the first sign you receive of a developing problem is through social media. Some of the third-party monitoring
products gather statistics about the availability of Office 365 as viewed through the lens of an individual tenant. That
data can be useful when it comes to discussing SLA performance with Microsoft if the need arises to make a claim for
poor service.

What is a PIR ? A PIR is a Post-Incident Report with the formal analysis of an “unplanned customer-impacting
service incident ” or outage where there was “broad and noticeable impact across a large number of organizations .”
Few incidents that you see in the Service Health Dashboard affect enough tenants for Microsoft to write and publish
a PIR. When these incidents do happen, Microsoft makes PIRs available to customers through the service dashboard.
The Office 365 support team publishes a preliminary PIR within 48 hours of the incident closure followed up by a full
and detailed PIR within five business days. A PIR includes the following sections:

Incident ID : Every Office 365 incident has a unique identifier. For example, EX29054 is an Exchange Online
incident.
User Experience : A brief description of what impact was suffered by end users. It might be that only
administrators were affected if the component involved is something like PowerShell.
Customer Impact : What business impact if any was caused by the incident.
Incident Start Date and Time : In UTC format.
Incident End Date and Time : Logically, the difference between the start and end date is the incident period
used to calculate time lost against SLA.
Root Cause : Microsoft’s explanation why the incident occurred.
Actions Taken : A timeline of all the actions taken from the time when the incident occurred to its resolution.
Sometimes several hours go by before engineers have correlated enough data from multiple tenants to be able
to home in on the components that might lie at the root of the problem.
Next Steps : What Microsoft proposes to do to ensure that the same problem will not recur.

As in all administrative documents, sometimes you must read between the lines to understand just what happened
and why it happened. The PIR also exists in internal and “customer ready” forms, the former being much more
detailed than the latter.

Social media is important to Microsoft when it comes to monitoring the service too. The sheer number of users who
connect to Office 365 means that any error in the service can quickly escalate to affect millions of people. Microsoft
monitors information flowing through social media to detect problems that users report with Office 365. This is just
one of the ways that they try to figure out the quality of service delivered to end users. Microsoft does its best to keep
the service dashboard updated with information about problems as they arise but because most incidents are confined
to a single datacenter or region, it can often be a challenge to understand whether any specific issue affects a specific
tenant.

Seeing a tsunami of updates about a problem with Office 365 is not a reason to panic because the problem might not
affect you. Remember that Office 365 is deployed in datacenters around the world and that any problem is likely to be
localized within one region. In the case of the high-profile incident in July 2014, the root cause was a failure in the
Azure Active Directory infrastructure that supports Office 365, but only in the part of the infrastructure dedicated to
handling inbound connections from North American users. When the problem happened, the directory service could
not handle the volume of validation requests created to check the addresses on inbound email, reducing the ability of
Exchange Online to process traffic. It was possible to send and receive some email during the outage, but in most
cases, customers had to wait until Microsoft restored the directory service to full health before their email flowed
normally.

The average administrator has no real chance of understanding the data used in SLA calculations because much of it
is invisible to anyone outside Microsoft. Factors such as Internet outages, local network delays, or client
misconfiguration are also excluded from the SLA equation. This means that one tenant might believe that Office 365
delivers an excellent performance against SLA, while another tenant experienced some problems, perhaps some of
their own making, and has quite a different perception of the service quality received. Beauty is in the eye of the
beholder and it is obvious that the sheer size of Office 365 creates a blurring effect across its regions. Service is
excellent overall (as seen in the reported SLA figures) but is awful when experienced by individual tenants affected by
different bugs or operational issues.

To be fair to Microsoft, they have met their financial commitment to refund customers when the root cause for obvious
Office 365 outages were mistakes that were under their control. It is important to recognize that most outages are of
short duration and only affect a small percentage of the overall Office 365 user base.

Office 365 Performance Against SLA

Microsoft reports the SLA performance for Office 365 against its 99.9% target on a quarterly basis. Microsoft
aggregates the data for all Office 365 regions to create a worldwide result and does not give SLA information for
individual Office 365 regions. Table 2-2 details Office 365 performance against SLA since figures first became
available in 2013. The highlighted figure is the most recent published quarterly result from Microsoft.

Q1 2013 Q2 2013 Q3 2013 Q4 2013 Q1 2014 Q2 2014 Q3 2014 Q4 2014

99.94% 99.97% 99.96% 99.98% 99.99% 99.95% 99.98% 99.99%

Q1 2015 Q2 2015 Q3 2015 Q4 2015 Q1 2016 Q2 2016 Q3 2016 Q4 2016

99.99% 99.95% 99.98% 99.98% 99.98% 99.98% 99.99% 99.99%

Q1 2017 Q2 2017 Q3 2017 Q4 2017 Q1 2018 Q2 2018 Q3 2018

99.99% 99.97% 99.985% 99.988% 99.993% 99.98% 99.97%

Table 2-2: Office 365 SLA performance since 2013 (source: Microsoft)

The blips in the Q2 2014 and Q2 2015 results are due to some high-profile incidents that deprived some North
American customers of access to email during those quarters, allied to a mass of smaller incidents that occurred
distributed across the other Office 365 regions. Although many smaller incidents have been recorded since, no major
outage has affected SLA results for Office 365 in quite the same manner.

SLA results for Office 365 are available six weeks after the end of the quarter. Remember that the SLA data reported
by Microsoft is a unified view taken from across the complete service. Your tenant might have experienced an inferior
level of service in a quarter, but even if a problem affects hundreds of thousands of users for several hours, the sheer
scale of Office 365 means that this incident might not be enough to move the needle for the reported SLA.

Be aware that failures in some dependent services that affect Office 365 have no influence over the Office 365 SLA.
For example, the default configuration for an Office 365 tenant is to use a free version of Azure Active Directory.
Microsoft makes no SLA commitment for the free version of Azure Active Directory (it promises an SLA of 99.9% for
the paid-for versions), so the significant failures in the Azure Active Directory multi-factor authentication service in
November 2018 did not count against any SLA except for tenants who bought the paid-for versions.

Making the Decision to Embrace the

Cloud
So far, we have concentrated on removing some of the FUD that surrounds cloud services. Now let’s move on to
consider some scenarios when the decisions to embrace the cloud and move to Office 365 is easy as well as when
complications might make things a little more “interesting”.

The Easy Decision

As you might expect, the decision to move to the cloud is much easier for some companies than it is for others.
Broadly speaking and accepting that these are high-level generalizations, the characteristics of companies in this
category include:

Start-up companies : Why would any start-up want to create its own IT infrastructure when most, if not all, of the
services they might need to use are available in the cloud? Apart from anything else, buying on a fixed cost per month
is usually better for cash flow. The exception to the rule is companies who have been created to develop a product
associated with an on-premises server product like Exchange. In this case, it is a good idea to eat your own dogfood
and run some Exchange servers.

Small to medium companies : Anecdotal evidence shows that much of the initial migration to Office 365 came from
companies with less than 500 employees, many of whom had run older generations of Microsoft servers such as
Windows Small Business Edition. Replacing old servers (some that resided in the proverbial cupboard or under
someone’s desk) with the promise of evergreen functionality offered by the cloud is very attractive, especially when
the migration of mailbox data is relatively straightforward as the volume is small enough to accomplish a smooth
weekend switchover. These companies tend to be “all in” the cloud and don’t need to operate hybrid environments so
they aren’t interested in aspects such as federation and single sign-on. Microsoft provides a FastTrack deployment
service to help companies like this who purchase 50 or more Office 365 licenses . The fact that Microsoft considers
that many of the onboarding operations necessary to move these companies to Office 365 can be executed on an
almost factory-like production-line basis indicates the relative ease the switchover can be, especially for small to
medium companies whose needs do not extend past email. More complex email scenarios (for instance, those
involving specific regulatory oversight) and SharePoint migration usually need the services of a specialized migration
partner.

Fortune 500 companies : Including Fortune 500 companies after small to medium companies might seem strange,
but the fact is that most if not all the Fortune 500 companies have some element of work running inside Office 365.
Sometimes a tenant domain is taken to reserve it for future deployment. Sometimes it is used to test the waters for a
planned move to Office 365. And sometimes it is because the company is in the process of moving some, but probably
not all, of its email workload to Office 365. Hybrid deployments are the usual rule of thumb for very large companies
in this category because this approach is most flexible and allows transparent co-existence across the two platforms,
including a shared directory, integrated mail flow, and access to data.

Moving from a non-Exchange email system : Microsoft supports the migration of IMAP4-based email systems to
Exchange Online. Older POP3 accounts are more of a challenge because these must go through an interim step where
messages are downloaded to a PST with Outlook and then imported into Exchange Online, again using Outlook. It’s a
manual process that can be tiresome. Then again, you can simply leave the old mail in the PST and not bother with the
import. Migration from other email systems such as Lotus Notes or Novell GroupWise can be done using separate
tools bought from companies that specialize in the migration area. Table 2-3 lists some of the well-known companies
in this space. Take the time to download trial versions of their software to establish the tool that is the right choice for
you.

Company Migration Tool

Binary Tree E2E Complete

BitTitan MigrationWiz

Code Two Office 365 Migration

SkyKick Enterprise Migration Suite

Quadrotech Mailbox Shuttle

Table 2-3: Email Migration Software for Office 365

Educational institutions : Many universities and colleges used earlier versions of Microsoft cloud-based email (such
as Live @EDU) or have experience running a competitor’s cloud-based email. It is obvious that these institutions will
find it easy to move to Office 365.

Again, these are generalized comments and individual country-level markets differ in terms of the mix of accounts
that have moved to Office 365.

The Harder Decision

The categories listed above are examples of organizations that are relatively easy to move to Office 365. Now let’s
look at some of the circumstances that can complicate matters.

Multi-location or multi-national : It is obviously much easier to coordinate the movement to Office 365 if everyone
(users and IT staff) share a common location. Planning becomes more complex as the number of locations grow and
more complex still if multi-national locations are involved. The needs and legal requirements of each country,
including security and privacy, should be factored into a migration project.

Email integrated with business processes : Although the API history of Exchange is speckled with false starts and
dead ends, there’s no doubt that many companies have integrated email with business processes. For instance, when
HR registers a new employee, a mailbox is automatically created, and a welcome message is sent to the mailbox.
Exchange Online supports EWS and PowerShell and it might be possible to move the email-enabled processes to the
cloud. Microsoft’s longer-term direction for extensibility is based on the Microsoft Graph covering all the Office 365
workloads. Alternatively, you could keep an on-premises server to handle email for those applications.

More than 25,000 seats : The more mailboxes you must move, the longer a migration to Office 365 will take.
Migrations are never popular exercises as they involve lots of repetitive operations (moving mailboxes) that are prone
to problems (corrupt items and other issues) that take a long time to achieve, especially when moving mailbox data
across the Internet. Given the right experience, good project management, and network connectivity, it is possible to
move several thousand mailboxes per week – but it is not easy.

Capable of running an internal cloud : Small companies have small IT departments and the people working in IT
are jacks-of-all-trades who are expected to be able to work with many different technologies. Larger companies can
afford to have specialists and specialization tends to enable a more sophisticated IT environment. These companies
might have the necessary expertise and operational maturity to be able to run an internal cloud where on-premises
versions of products like Exchange can continue run at an economic price point comparable to the costs of using
Office 365.

Poor network links to the Internet : Sometimes we forget that not every company enjoys high-speed and reliable
Internet connections. It might be the case that offices are stuck at the end of an extended hub-and-spoke network that
simply needs to be upgraded or it might be that poor connectivity is a fact of life in some or all locations. Office 365
depends on the Internet to connect users to Microsoft datacenters. The exact bandwidth and latency needed depends
on the client mix (Outlook creates a heavier demand on the network than OWA does), number of users, and working
patterns (always size for peak demand plus contingency). Remember that you also must move mailbox data to
Exchange Online, a process that won’t be quick if the network lags.

Old desktops: Exchange Online only supports certain versions of Outlook and browsers. Organizations that use
software such as Outlook 2003 or older browsers should factor upgrades into the migration. Deploying a new version
of Office or a complete desktop refresh is expensive and takes time and attention to detail to ensure that user
productivity is not disrupted. The click-to-run version of Office uses streaming and virtualization technology to install
the Office desktop products. It is an excellent way of addressing the issue of outdated desktop software as it allows for
automatic updates to be downloaded and installed on user PCs. Organizations can configure Office 365 Pro Plus to
manage software updates in-house if required.

Recent Exchange upgrade : A company that has recently (in the last two years) upgraded their IT infrastructure for
Exchange 2013, probably including a hardware refresh and potentially an upgrade to Windows Server 2012 R2 and
other components, might not have the appetite to go through the additional step of moving some workload to Office
365. A similar case can be made for companies who have moved to Exchange 2010. Although the software is now
approaching the end of its lifecycle, Exchange 2010 is still a fine email server, and an understandable desire probably
exists to extract a reasonable return from the investment put into the upgrade.

Other complexities exist that are not in this list. For instance, any company who is already running on a hosted email
service such as a managed service based on on-premises Exchange or a completely different platform like Gmail or
SMTP mail will have their own difficulties to overcome.

Now that we understand the various categories of companies that might consider moving to Office 365, let’s consider
the money question.

The Economic Imperative

On the surface, saving money is an excellent justification to move to the cloud. Many commentators focus on the fixed
per-month subscription fee and compare this to the all-in cost (if it is known) to run an in-house system. Salespeople
who want to push the virtues of the cloud love to talk about the fixed-price per month, especially if they can get you to
focus on the lower cost plans. Once you start thinking that you can buy cloud services for $6/month per user, that
impression fixes itself in your mind and a certain view that “the cloud is cheap” forms. It can be very difficult to
change this impression, even when you realize that you really need one of the more expensive plans and will pay
$20+/month per user.

Invariably the comparison is positive for the cloud because organizations often do not fully understand the complete
cost picture are therefore do not factor all the cost elements into the equation. It’s true too that the notion of paying a
fixed monthly cost is attractive because it allows the company to manage cash flow more precisely than the
sometimes-erratic spending patterns of IT. It is also clear that cloud systems are more flexible than in-house systems
when the time comes to add or reduce capacity.

Figuring Out Cloud Costs

You will never quite know what the actual cost base for a cloud deployment is until after you complete the deployment
phase (migration of users and applications) and operations stabilize. Only then will you know exactly how much the
cloud is costing and where those costs lie.

But before you make the decision to embrace the cloud, it is wise to figure out exactly what the current on-premises
environment costs. You will need this data to compare against cloud expenditure to show any savings Let’s discuss the
typical cost buckets that you should consider.

Hardware
Your on-premises deployment probably uses a range of different servers including Active Directory servers, Exchange
servers, mail hygiene or bastion servers, SharePoint servers, Skype for Business servers, and administration systems
(workstations and servers). Some of these servers might run as virtual machines on a hypervisor (like VMware or
Hyper-V). You need to capture details of all the servers involved in the workload to move to the cloud and figure out
their annual running cost, including capital depreciation, replacement cost, power consumption, cooling, and other
datacenter expenses.

Software
You won’t need software licenses for Windows Server and the server applications after you transition their workload
into the cloud, but you will need to continue to pay for some servers, especially if you plan to run a hybrid
environment. In this case, you’ll need to have servers for tasks such as directory synchronization and perhaps single-
sign on based on Active Directory Federation Services. You might route your email from the cloud to on-premises
servers for delivery, so you’ll need servers for that purpose – and those servers must have licenses. Depending on the
monitoring framework that you use, you might have to upgrade it to deal with cloud services. It’s very important to
figure out exactly what portions of the overall workload will remain on-premises because this will enable you to
understand what server hardware and software licenses to but, power, license, and run in tandem with your cloud
subscriptions.

Every Office 365 user must have a license before they can access the service. To calculate the monthly subscription
charge for Office 365, you need to know how many users will need licenses and what plans they use. Remember that
shared mailboxes do not need licenses (unless they have archives, are on hold, or use more than a 50 GB quota).
Logically, the more feature-rich plans are more expensive, so you need to understand what people need to use and
then match those requirements with a suitable Office 365 plan. Remember to include all on-premises functionality in
your calculations and then find what Office 365 plans best match your needs. Standalone Exchange Online and
SharePoint Online plans exist for those who do not need the full range of functionality available in a plan like Office
365 E3. Kiosk (web-only) plans exist for “frontline” workers, people who do not have workstations and only need
intermittent access to applications like Outlook or OneDrive.

Microsoft inevitability guides customers towards higher-priced Office 365 plans. Apart from increasing Microsoft’s
cloud revenues, there is good reason for this as customers can access many features in these plans that are
unavailable to the lower plans. You must decide whether you need functionality like Cloud App Security, Advanced
Threat Management, or Advanced Data Governance to justify buying Office 365 E5, and how many people need such a
plan. Another thing to consider is whether it is cheaper to buy a lower-cost plan and then buy add-on components to
make specific functionality available to selected users.

Remember that Office 365 does not run in a vacuum and that it depends on other Microsoft technology. Every Office
365 tenant has a basic version of Azure Active Directory, but you will need premium licenses if you want to use certain
functionality like policies for group naming and expiration. Many companies buy the Enterprise Mobility and Security
suite because it has functionality that they need, like Intune to control mobile devices, as well as premium licenses for
Azure Active Directory and Azure Information Protection. The point here is that calculating the cost for Office 365 is
not just the sum of monthly plans multiplied by users: the actual cost of what you want (or need) might be higher. Be
sure that you understand all the licensing requirements as you build the case for Office 365.

Microsoft Office
You might be quite happy to run the Office 2003 or Office 2007 applications on your desktops. Unfortunately, Office
365 won’t take the same view of the world as it needs up-to-date software before it will allow users to connect to its
services. If you do need to upgrade, you should consider using Office 365 Pro Plus (click to run) to ensure that the
Office desktop applications receive updates automatically over the Internet. Different channels dictate how often
users receive Office updates, so some control is available. Office 365 no longer supports Office 2010 and Office 2016
is the default version for Office 365 connectivity.

Applications
Most companies have applications that they have built in-house. These applications range from Visual Basic programs
and Excel macros to full-blown database applications. Some of these applications might have dependencies on the
workload that you plan to move to the cloud and might therefore need code changes so that they can continue to
function afterwards. That is, if you can find the code and have someone available who understands how to make the
necessary updates to the code. One major factor to consider is that Office 365 does not allow programmatic access to
the data that it manages in the same way that is possible on-premises when you control all the moving parts, so
programmers must use APIs such as the REST-based Office 365 APIs to interact with online services. In turn, this
might involve some retraining or knowledge acquisition.

Browsers
You might decide that you are going to use browsers to access Office 365 and that web clients suffice. After all,
modern web clients are highly functional and support features like offline access. The plan is viable, if users have
modern browsers on their PCs, which means Microsoft Edge, Internet Explorer 11, or the latest version of Chrome,
Firefox, or Safari (for Macintosh computers). In addition, it is possible that a browser is unable to access specific
Office 365 features. For example, only Internet Explorer supports S/MIME encryption with OWA. By contrast, at one
time, only Edge and Chrome supported the upload of complete folders to a SharePoint Online or OneDrive for
Business library while Internet Explorer could only upload a file at a time. Upgrading a browser on user PCs is a task
that varies in difficulty from easy to extremely hard depending on the browser, the number of PCs and the PC
management framework that you use. In all cases, you must use a supported browser (see the Office 365 system
requirements ). And of course, if you update browsers, make sure that the upgrade does not affect other applications.
It has been known to happen!

Networks
You won’t do anything with Office 365 unless you can access its services, and that means connecting across the
Internet to Microsoft’s datacenters. You need to understand just what kind of access your company will need to the
public Internet to carry the anticipated traffic to Office 365 and all other Internet traffic that your business needs,
plus a reasonable uplift for anticipated growth over the next few years (as there is no point in upgrading to a
connection that just about copes with Office 365 and runs into problems soon thereafter).

People
Running an on-premises environment needs knowledge and experience of Windows, Exchange, SharePoint, and all the
other products that combine to form an enterprise environment. Over time, the people who run IT systems build a
certain insight into the company and how it conducts its business. For that reason, I hesitate to say that companies
can achieve workforce savings by transitioning workload to the cloud. Perhaps it is possible to save by letting some
operations staff go because administrators no longer perform tasks such as server maintenance, but usually you find
that other work appears to fill the vacuum left by work now done in the cloud. For example, in a hybrid environment,
someone must manage tasks such as directory synchronization, single sign-on, certificates, and mail flow.
Even when Microsoft assumes total responsibility for the management and operation of all the workloads running
inside Office 365, someone must administer these applications and manage Office 365 on behalf of the company. User
accounts do not create themselves and there are always bits and pieces to keep everything working smoothly. Because
the mundane operations involved in server management no longer exist, administrators need fewer hours to manage
online applications, but this is not a reason to cut their positions. During the migration period, administrators will
have plenty to occupy themselves as workloads move to the cloud. Afterwards, they will have new tasks to perform
(such as hybrid management) and can take on new responsibilities such as setting up enterprise voice operations
based on Microsoft Teams or creating a document management strategy for the company that exploits SharePoint
Online and replaces old Windows file servers with OneDrive for Business. They can investigate new capabilities that
the company has never been able to exploit because staff were not available, such as figuring out whether the Outlook
app model offers any potential for business-specific solutions. And they can dedicate time to setting up a monitoring
framework that deals with both on-premises and cloud components and is available to administrators, support staff,
and potentially even users.

Another important role is to stay informed about changes that occur inside Office 365 and associated technologies like
Azure Active Directory. Change creates opportunities but also causes challenges for users and operations, so someone
needs to check the ongoing changes and assess their impact against company operations. In short, moving to the
cloud removes some work while also creating new work. The notion that companies can achieve considerable savings
by making IT staff redundant is dubious and often does not stand up when challenged.

Training
It’s obvious that a considerable upheaval occurs within a company’s technical infrastructure when a move to Office
365 occurs. That upheaval affects users, administrators, support personnel, and managers alike. Users need training
to work with new software (like how to find documents with Delve or the proper etiquette for using Teams) and how to
recognize and cope with common issues, such as a glitch with Internet connectivity that means they cannot connect to
Office 365. If you enable something like multi-factor authentication to augment security or use new features exposed
in Office 365 like rights management and message encryption, users need education on these points too.

Administrators need training on the new moving parts that connect your environment with Office 365. If you change
your monitoring and reporting framework, they’ll need to know about that too. Support personnel like the help desk
need to understand how to handle problem tickets when they have no control over a large part of the picture. They
need to understand how to work with Microsoft Support, how to figure out whether problems are internal to your
infrastructure or caused by Office 365, and how to track progress of problems escalated to Microsoft. Managers,
especially IT Directors or CIOs, need to understand how the world of IT is changing and how a decision to embrace
something like Office 365 brings its own set of advantages and disadvantages so that they can better guide the
progress of the company’s IT environment. In a nutshell, lots of change is in the air and will affect people. If you don’t
train them to cope with the change and how to exploit the new functionality, then you’re creating some hurt for users
and administrators and won’t maximize the potential benefit of moving to the cloud.

Add-on Products
Although Microsoft sells Office 365 as a complete solution, a lot of interesting functionality is also available in add-on
products created by Microsoft and third-party ISVs. It is difficult to offer precise guidance on this point because the
needs of individual companies vary so much. Examples of areas where you might look for extra functionality include:

Security, such as an alternate email hygiene service. Microsoft does not recommend that email traffic to Office
365 passes through other hygiene services because they say that this affects the ability of Exchange Online
Protection to detect and isolate malware and other threats.
Monitoring of Office 365 workloads and associates services such as hybrid connections and directory
synchronization.
Reporting and auditing to keep and analyze data for longer than the 90 days made available by Office 365 (or 365
days for E5 users). The retention of audit data for longer periods is important for any company that comes under
the scope of the European Union’s General Data Protection Regulation (GDPR) as you might need these records
to prove or disprove access to personal data.
Backup of mailboxes, SharePoint Online sites, and other information. Typically, these backups copy data to a
cloud datacenter run by a third party. The case for implementing backups outside Office 365 varies for the
different workloads. Native data protection and the volume of mailbox data possibly makes off-site backups for
Exchange Online less attractive than those for documents and other data held in SharePoint Online and OneDrive
for Business. Be aware that major challenges exist in the backup of data belonging to cloud-only Office 365
applications based on Azure data services like Teams and Planner.
Compliance, for instance, to extend Data Loss Prevention to cover more than Office 365 data.

Creating a Network to Support Cloud

Services
As discussed earlier, Office 365 is all about “the cloud”. In many respects, this is shorthand for “the Internet.” No
Office 365 project can succeed without reliable high-speed and high-capacity connectivity to the Internet, so the first
and most fundamental question that any company who wants to move from an on-premises infrastructure to use cloud
services is can their network infrastructure cope. Given the state of modern communications and the profusion of
fiber-powered networks around the world, it is surprising just how many companies discover that they might have
difficulty securing enough high-quality bandwidth, but it does happen. In these instances, all you can do is wait for the
local network provider to upgrade their pipes.

Organizations that move to Office 365 often discover that their internal network infrastructure is not fit for purpose.
This is quite normal and logical. You cannot expect a network design created to serve the needs of an internal
infrastructure where servers and clients are co-located on the same WAN to suddenly transform itself to be able to
cope with completely different traffic patterns and user demand. Thus, a move to embrace Office 365 is an
opportunity to review your network from several perspectives, including:

Are internal network upgrades necessary to accommodate the volume of client traffic to Office 365 and other
Microsoft cloud services? For instance, internal routers, proxy servers, or NAT-routers might not be able to
handle the increased load caused by all the TCP/IP sessions consumed by users as they connect to Office 365
services. Project experience shows that a single user can consume up to 20 sessions to connect to Exchange
Online, SharePoint Online, OneDrive for Business, Skype for Business, Teams, and other services. To help,
Microsoft publishes information about network planning and performance tuning for Office 365. Another page
covers network capacity and devices , including elements such as WAN accelerators. Another interesting source
of information is the Microsoft Cloud Networking for Enterprise Architects poster . This poster shows some of the
changes in an on-premises network that tenants might need to make to support connectivity to Microsoft cloud
services.
What ports and IP ranges must be available through firewalls to allow users to connect to Office 365? To help in
this task, Microsoft regularly publishes updates for the URLs and IP address ranges used by Office 365. Over
time, Microsoft plans to replace the HTML, XML, and RSS data for network changes with a web service that
customers can consult to learn about changes in Office 365 networks, including updates to route traffic to Office
365 and avoid latency and other network challenges. From August 21, 2018, Microsoft changed the platform
used to publish IP address updates. However, old links continue to work.
Will application-level changes force generate increased demand for network bandwidth and capacity? For
instance, both the cloud and desktop Office applications have an AutoSave feature to capture changes made to
documents during editing sessions so that users do not have to save files. AutoSave works for documents stored
in SharePoint Online and OneDrive for Business sites by creating new versions of the documents to store changes
generated during an editing session. The AutoSave activity uses network bandwidth to send the updates from
user workstations to Office 365; the OneDrive sync client might then synchronize the updated documents back to
workstations by the OneDrive sync client. These changes can generate an added network load that does not exist
in on-premises environments.
Are any changes needed in network security? For example, Office 365 supports multi-factor authentication
(MFA). If you deploy MFA to protect Office 365 data, can you use MFA to protect other applications?
Can you improve data routing within the internal network? For example, to carry traffic from outlying business
locations to an Office 365 front-end (point of connection).

Client traffic changes because the target servers are no longer within the internal network. Instead, clients connect
via HTTPS to cloud-based servers across the Internet to applications running inside Office 365 or to SQL or other
application servers running inside Azure. The outbound links that connect your company to the Internet must be able
to support the new workload. The exact type of link needed varies depending on the number of clients that you expect
to be active at any time, the type of clients, and the cloud services they will access. Although Microsoft has created a
large network of local front-end points of connection (or “edge nodes”) to speed traffic to its datacenters (Figure 2-1 ),
unless the link from your company locations to Microsoft’s network can carry the amount of user traffic, users will not
be able to work as well as they should. Microsoft’s cloud network, including other services like Azure, spans enough
fiber to stretch to the moon and back three times. The combination of edge nodes and fiber networks means that it is
usually possible to make a fast connection to Office 365, no matter wherever you are in the world. That is, if your local
or last-mile connection can carry the traffic.

Figure 2-1: Microsoft global network with Office 365 front-end shown (blue dots)

If you run on-premises servers today, you probably have some data to show how much network bandwidth applications
and users consume. One rule of thumb is to double (some say triple) this figure and use that amount as the basis for
planning. Advice and guidance on this topic evolves over time as Office 365 changes and knowledge increases, so it is
best to ask Microsoft or an experienced consulting company for their recommendation as to the capacity you need –
and to then add 30% or so to cater for growth in user demand and to accommodate new applications and features
used within Office 365. Because Office 365 evolves over time through the introduction of new applications and
features, usage patterns and consumption will change too. It is therefore important to keep an eye on changing
demand over time.

Remember that moving to Office 365 will not make a bad internal network any better. In fact, the extra demand that
the added functionality available in Office 365 might generate when users realize that it exists is likely to put increase
pressure on the internal network. If that network is barely able to cope with the demands of on-premises client-server
connections, it will do no better and is likely to do worse when cloud services take over.

Before beginning to migrate to Office 365, it is sensible to upgrade your external links so that they can handle the
new load. You can start the migration using existing network capacity, but you might find that the migration activity
interferes with other work, such as access to external web sites. In addition, as you move users to the cloud, those
users will want to access their new cloud-based mailboxes and other applications like Teams. They will quickly
become unhappy if cloud access is slow and unreliable because they hit problems like port exhaustion due to lack of
publicly routable IP addresses (see this support article ). In addition, if your organization uses central connections to
the Internet, you will probably find that a separate link per location delivers better results. A slow or constricted
connection to the Internet is not a good foundation for success with Office 365. If your connection cannot cope with
the traffic to the cloud, your Office 365 project will die a quick death.

Although an expensive option, it is possible to implement a dedicated network connection to Office 365, such as
Microsoft’s Azure ExpressRoute for Office 365 service. ExpressRoute uses a dedicated MPLS connection between
your datacenter and Microsoft's network to ensure that traffic flows as if it were on your internal network. In general,
Microsoft believes that careful management of an internet connection is enough to deliver the necessary connectivity
to Office 365, especially now that their network of local connection points is so extensive. Before you can use
ExpressRoute, Microsoft must assess your network to decide whether ExpressRoute will achieve a measurable
improvement in connectivity. One scenario where ExpressRoute might help is where many people in a single location
use the Microsoft Phone System because ExpressRoute can prioritize voice traffic over other traffic.

When you plan for new network capacity, make sure that you accommodate three types of traffic:

User connections to the new cloud environment plus existing work. Remember that users often make use of
company networks to access external social sites such as Facebook and Twitter during work hours. You can try to
restrict this access but, in many cases, this is like pushing water uphill, so it is best to include a certain overhead
for personal network activity. Remember too that different clients have different network characteristics. For
example, Outlook clients consume more bandwidth than OWA, especially if you use add-ins that need network
access. As explained in Chapter 16, voice and video conversations are sensitive to network conditions and need
certain basic capabilities to be able to function. New applications such as Teams introduce new demands on the
network. Still another thing to consider is the increased demand for mobile access to data. Office 365 has mobile
clients for most applications and the increased use of mobile clients inside company offices might affect Wi-Fi
networks if those networks cannot deal with the number of connections and the bandwidth demand. Microsoft
has network calculators for Teams, Skype for Business, and Exchange that help assess likely demand (further
information is available here ).
Migration of user data to the cloud. Moving 500 users might happen in a cutover operation during one weekend.
Migrating 50,000 users will probably take a little longer. In both cases, you still must move mailbox data from
servers in your internal network across the Internet to an Office 365 datacenter. Depending on how much data is
involved and the available throughput from your network to Microsoft, that movement could take longer than you
think.
Administration operations. Intelligent applications like Outlook isolate users from the effect of network outages
and allow them to work offline. However, all Office 365 administrative operations occur online and a reliable (and
fast) network connection is necessary if you want to be sure that administration can be performed without
difficulty.

Information presented by the Office 365 product group at the Ignite 2017 conference gives advice about:

Strategy .
Planning .
Implementation .

These sessions are valuable in terms of framing the discussion about how your network infrastructure might have to
change to protect the Office 365 user experience. Another interesting document is a Microsoft IT report outlining
some of the network challenges they faced in moving Microsoft users to Office 365. While your company might not be
the size of Microsoft, there is usually something to learn from descriptions of how other companies dealt with
technical issues. Finally, the top ten troubleshooting tops for Office 365 network connectivity is a useful document to
consult if you run into problems.

During the migration period, the volume of network traffic will be higher than normal because of the need to move
information like mailbox data and documents to the cloud. After the migration period is over, network demand should
settle into a more predictable load. There will be peaks and valleys in demand, but you should be soon well acquainted
with the general shape of network demand for different locations and be able to make whatever changes are
necessary.
Executive Buy-in and Communications
Few major projects succeed without executive buy-in and support. A good communication plan to explain why the
project exists and the benefits expected to accrue to both the company and employees is also important to offset the
upheaval that will probably be encountered as mailboxes are migrated, networks change, and clients are updated.

Executives should focus on simple messages like:

We are buying the best-of-breed Office software from Microsoft.

Using Office 365 will allow the company to be more flexible and adaptive to changing business circumstances.
Employees who worked on our previous messaging systems will now work on more important projects such as
figuring out the corporate strategy for using SharePoint Online, making better use of Skype for Business Online
(perhaps as a replacement for a traditional PBX), or determining whether Office 365 Groups or Yammer is the
most suitable collaboration platform for the business.
Moving to the cloud is a safe and secure choice.

The communication plan will cover:

A summary of what's happening – the company is upgrading and improving its email and collaborative
capabilities by moving to Office 365.
When the changeover will occur and when employee mailboxes will be migrated.
The functionality changes and improvements that users will see and how these updates will make work easier.
Some simple scenarios might be included, such as how to set up a group mailbox to support project teams.
Tips and techniques for making a smooth changeover.
Anything an employee must do to enable connectivity with Office 365. For instance, those running older versions
of Outlook or old browsers might have to upgrade their software.

Every company is different, and the suggestions outlined above are merely the start of the discussion about how to
lead people through change.

Dipping Your Feet into The Water

Even after doing extensive research to verify whether Office 365 is the right platform for your company, there is
nothing quite like getting your hands dirty to make a technologist happier with a technology. Office 365 makes test
drives easy by allowing you to sign up for a 30-day free trial. Essentially, you can create a fully functional test tenant
and use it for 30 days to decide whether Office 365 works for your company. And if you’re still not sure about Office
365 after that period, you can simply discard the original trial, extend the trail, or start off again with a new tenant.

Using a trial tenant as a proof of concept is a great way to get a sense of how Office 365 works in practice. The
exercise will not expose some of the more complicated challenges, such as how to migrate public folder data or how
hybrid connections or single-sign on work over an extended period. However, the information gained from a trial is
certainly enough to gain a broad understanding of the different parts of the service and how the administrative
experience differs from your on-premises environment. You’ll be able to figure out how to move workload to the cloud
and what extra functionality exists – or where functionality gaps exist because something like a third-party add-on is
unavailable within Office 365.

Some companies start off with a trial tenant and then convert the trial into a paid-for service after the 30 days. This is
OK if you plan for the eventuality and go ahead on that basis. However, you should be aware that no functionality
exists inside Office 365 to transfer data from one tenant to another. Third-party tools are available, but these come at
added cost.

You also need to make sure that any settings used to direct email to the trial tenant such as validated domains are
removed afterwards. All-in-all, it is usually a better idea to regard a trial tenant as no more than a test and to accept
that if the company decides to embrace the cloud, you will start over from scratch and use a new tenant.

Office 365 Partners

Microsoft promises free onboarding aid to Office 365 for companies who buy more than 50 seats and will provide
some funding to accredited partners to help with migrations. The funding available through FastTrack can be very
helpful to offset the overall cost of a migration. However, FastTrack migrations are very structured and follows a strict
playbook. It might or might not be enough for the needs of a small company that wants to move from on-premises
Exchange to Office 365, but it is unlikely to be enough for complicated migrations. This is when you might need to
either run the migration using internal resources or seek the help of a Microsoft partner who specializes in Office 365.

Given enough time to acquire knowledge and run some trial migrations, it is possible for any experienced Exchange
administrator to prepare an Office 365 domain and move mailboxes to it. The steps needed to configure hybrid
connectivity are largely automated and well understood so that attention to detail and good preparation will get most
administrators through the process. Of course, there are always examples of situations that are unique in certain
aspects, such as migration projects that propose to collapse several Exchange on-premises organizations into one
Office 365 tenant, but in general the migration of a normal Exchange setup composed of a single organization based
on a single Active Directory forest is usually straightforward. Outside of moving mailboxes, the kind of detail that
causes problems include single sign-on (SSO), federation, mail routing, message hygiene, and hybrid co-existence. We
discuss issues linked with migration the companion volume.

Given that email servers must continue running while work proceeds to prepare for migration, you might need to
engage some outside help to make the project happen in a reasonable period. The options are to use:

Microsoft Consulting Services (MCS).

A Microsoft partner specializing in Office 365, including the consulting and support arms of major IT companies.

Although expensive, MCS has the advantage of being part of Microsoft and offers the reassurance of being a single
throat to choke if anything goes wrong. On the other hand, an independent partner often offers a more realistic
opinion about how to run successful projects and the likely pitfalls that exist along the way. In addition, independent
partners often have deep knowledge of third-party solutions that can help solve some of the more complex situations
that occur in migration projects.

Microsoft partners exist in every market. In selecting a partner, consider the following questions:

Is the partner a member of the Microsoft Partner Network? A partner who is not a member misses out on many
important resources and tools that Microsoft makes available through its network.
How many Microsoft accredited specialists does the partner have on staff, how recent are the accreditations, and
in what technical disciplines do their people have experience and knowledge? Non-Microsoft accreditations such
as CISSP (security) might also be useful, depending on the areas covered in the project. Partners who hold silver
or gold competency levels for Office 365 can receive FastTrack funds to help in migration projects.
Remember that accreditation and certification is one thing. Being able to do a job is quite another (the IT
industry is littered with certified individuals who are incapable of doing any useful work). Ask about the
experience the partner and its staff have with Office 365 and what projects with similar requirements have they
been successful with in the past.
Ask about the roles played in the projects by the people that the partner proposes to involve in your project.
What areas of specialization does the partner cover within Office 365? A partner who specializes in Skype for
Business, SharePoint, or Yammer is probably not a good choice for companies who want to migrate Exchange
2010 to Exchange Online, especially when you throw the intricacies of hybrid connectivity and directory
synchronization into the mix. On the other hand, if your focus is on moving workload from SharePoint on-
premises to SharePoint Online, experience with Exchange will not be much help. Introducing Delve often needs a
reasonable amount of SharePoint knowledge because of the way that Delve can “uncover” poor sharing and
access control practices.
Some parts of Office 365 are relatively new. These include Office 365 Groups, Planner, and Teams. If you are
interested in these areas of functionality, you need to find out whether the potential partner has relevant
experience of deploying, managing, and troubleshooting these components.
The deployment of the Microsoft Phone System often needs experience with voice systems, especially when the
need arises to replace traditional PBXes.
The deployment of high-end features such as Office 365 Cloud App Security, Advanced eDiscovery, and
MyAnalytics also usually needs specific experience in these areas.
The world of application development is different inside Office 365. New tools like Flow and PowerApps do not
exist in the on-premises world and new APIs like the Microsoft Graph are available to expose more data to
applications than ever before. If you need to write some code, look for people who have knowledge of the new
tools and APIs before you decide on your partner.
Do not forget that Office 365 is at the center of an ecosystem. Microsoft does not have solutions to every possible
feature request or customer need and it is good to find a partner who has knowledge of solutions to areas such as
service monitoring and reporting, security, external archiving, backup, and so on.
How skilled is the partner at understanding how to exploit the Office 365 plans, add-ons, Azure Active Directory,
Azure Information Protection, and Enterprise Mobility and Security to meet the needs of your company?

Specifically, when considering the migration of workload from Exchange on-premises servers to Exchange Online, ask:

The experience they have with on-premises Exchange, specifically the version you run. Although they share some
characteristics, Exchange 2010 is very different to Exchange 2016.
Their knowledge of various aspects of Exchange that you might want to use when you move to the cloud, such as
encrypted email, different email clients, public folders, and so on.
Their experience with other areas associated with Exchange such as Exchange Online Protection, Azure Active
Directory, rights management and encryption, etc.

Apart from these points, you might also ask what contribution the partner makes to the local technical community,
how useful their web site or blog is in terms of the information that you find there, and whether any of their personnel
are Microsoft Office Servers and Services 365 MVPs as they have background channels to the development group that
might be useful in resolving problems that arise during the project.

The Value of Hybrid Connectivity

Microsoft has done a lot of heavy engineering to create hybrid connectivity for Exchange. In some respects, they had
to do this work to preserve the Exchange installed base. After all, most large companies will not countenance the “all
in” approach to the cloud as it is the nature of enterprise IT to move into new areas quite slowly after due diligence is
done to identify issues that must be solved before full implementation can proceed. Setting up a hybrid connection
between on-premises Exchange and Office 365 allows these companies to dip their toe into the cloud (a mixed
metaphor) by creating the links to enable interoperability between the two platforms. Some workload can then be
transferred to the cloud, the results observed, and a decision made as to the best way to proceed. In some cases, the
decision will be to move more workload, and perhaps even to proceed along the line so that most mailboxes are
transferred. In others, the decision will be to retain a substantial portion of work on-premises and use Office 365 for
tactical purposes, such to support mailboxes for specific types of employees. The transfer of workload follows the pace
set by the company and could extend out over many years.

Without hybrid connectivity, Microsoft would offer the same one-way trip to the cloud as Google does when it
proposes to migrate workload to Google Apps for Work. If no possibility exists to run on-premises and cloud platforms
together as a single environment, customers who want to move to the cloud would have to plan on an accelerated
transfer, much like the switchover approach used to move smaller companies to Office 365. This is a perfectly
acceptable technique when dealing with a relatively small number of users, mailboxes, and data; it is less satisfactory
when the numbers mount up. Other complicating factors that make it harder to plan for a fast switchover include the
distribution of users across multiple company locations, especially when international employees are involved.

Of course, it’s possible to extend a switchover period without hybrid connectivity by relying on SMTP to keep
everyone connected. Although this approach works, it means that you must operate two totally isolated environments
until the final mailbox is moved into the cloud. That might be easy in some circumstances, but issues such as single
sign-on, mailbox delegation, calendar sharing, public folders access, and so on tend to complicate matters in most
companies who have operated Exchange for a reasonable period.

Hybrid connectivity smoothens the issues by enabling connectivity between the two platforms. It’s not a total panacea
as some manual interventions are necessary to close the gaps, like the need to manually export and import retention
policies and tags to ensure that the same compliance regime exists in both platforms. However, these interventions
need a lot less effort than would otherwise be necessary to move to the cloud without a hybrid connection.

As explained in Chapter 8, hybrid connections are also supported for SharePoint. These configurations are less
popular than hybrid Exchange, but it’s certainly an option to consider, especially if your on-premises SharePoint
configuration includes applications or other customizations that cannot run in the cloud.

Hybrid connectivity also gives customers a “plan B” if moving to the cloud does not deliver the advantages expected
when the decision was made to use Office 365. The same facilities that allow mailboxes to be moved from on-premises
to Exchange Online can be used to move mailboxes back to Exchange on-premises. Again, some gaps exist – like how
to move public folders back because the public folder migration tools run in one direction – but overall, hybrid
connections are two-way and can therefore be regarded as a way to reduce dependency on Office 365 should the need
arise.

The Need for Plan B

No CIO wants to implement a new platform without knowing what will happen if things go wrong. All projects carry a
certain element of risk and a project to move workload to Office 365 is no different. Good planning, solid project
management, and technical expertise all mitigate the risk, but some cloud projects will end up at a point where
management decides that on-premises is the better option. You must then figure out how to move content back from
the cloud platform and resume operations on-premises.

Thanks to the engineering investment made by Microsoft to enable hybrid connectivity, Exchange is the easiest
application to move back on-premises. Mailboxes can be transferred back to on-premises servers as easily as they can
move to the cloud. The only downside is the amount of storage that might have to be deployed to accept inbound
mailboxes because users might have become accustomed to the liberal mailbox policies used within Office 365.
Another complication is how to deal with content stored in inactive mailboxes; it is easy to overlook these mailboxes
because they do not show up in all administrative interfaces and reports.

Mailboxes are relatively easy but everything else is hard. If you have elected to use Azure Information Protection
(Chapter 24) to protect information inside Office 365, you might find that it is difficult to access protected content
moved back on-premises because it is no longer possible to decrypt that content. A hybrid configuration will help if
you keep a connection to the Azure Information Protection service.

You must make sure that you move any customization made to Exchange Online back on-premises so that things like
PowerShell scripts, transport rules, Data Loss Prevention rules, retention policies and tags, and so on are moved
across. Again, if you are in a hybrid environment these settings should already be duplicated to ensure that the cloud
and on-premises platforms remain in tandem, but there's work to be done to make sure that everything is
synchronized.

For instance, if you make a big commitment to Teams as a collaboration platform with or without voice integration,
what do you do if things do not work out in the cloud and you decide that the best course is to revert to on-premises?
You can move voice communications to Skype for Business Server and documents back to SharePoint Server, but the
conversations and chat part of Teams is cloud-only. Much of Office 365 is now unique to the cloud platform and no
tools exist to move data to a form that is consumable by an on-premises application.

You can synchronize Office 365 Groups to hybrid on-premises environments, where they show up as standard email
distribution groups. However, Office 365 Groups do not exist in on-premises software (as is the case for other some
user-centric features of Office 365 like MyAnalytics). It is hard to know how to retrieve the information in group
conversations, shared notebooks, and document libraries because no export utilities exist, so again this will probably
need a great deal of manual effort to recover data from the cloud. Planner adds its plan metadata to the mix of data
that you might need to recover to reassemble plans into a reusable state. The same is true for the data used by
Microsoft Teams.

Microsoft makes it clear that tenants continue to own their data stored in Office 365:

"You own your data and retain all rights, title, and interest in the data you store with Office 365."

Although it is straightforward to consider the movement of basic data from Office 365 to other platforms, no obvious
way exists to move the complete spectrum of data contained within a tenant. For instance, you can copy documents
from a SharePoint library to your PC, but how feasible is this approach to capture the complete corpus of information
existing in SharePoint and OneDrive for Business sites within a tenant? That information includes lists, document
libraries, OneDrive for Business sites, and eDiscovery cases.

The Office 365 Video portal is another challenge. This is an application that allows tenants to set up a kind of “Internal
YouTube” for the company through channels that categorize videos for viewing inside the company. The video portal is
based on an integration between SharePoint Online and Azure Media Services. The portal is good at ingesting content
but does not have the same kind of export functions, which poses an issue should you need to recover the video
content and metadata at some point. And if you use Sway, you need a way to recover its files too.

Third-party software vendors offer utilities to help move content to Office 365, but the same capabilities do not exist
to move information out of Office 365 back to on-premises servers. Given current trends to move away from on-
premises servers, this situation is natural as it would take a bold decision to create a migration tool to move content
from the cloud.

Well-planned migrations usually proceed to plan and are successful. Most Office 365 migrations are, not least because
of the expertise available to assist companies in the transition. But wisdom and prudence dictates that you should
always have a Plan B – at least in outline – just in case your project experiences problems.

Cancelling a Tenant Subscription

Office 365 is obviously a commercial business and Microsoft is quite happy to keep your data online while you
continue paying the monthly subscription fees. If a license is removed from a user account, its data stays accessible
for 30 days and is then removed. The exception is when a mailbox is put on hold before the account is deleted. In this
case, the mailbox is regarded as “inactive” and is retained while the hold remains in force (see Chapter 6 for more
information).

However, you might decide that Office 365 is not for you and go ahead and cancel a tenant’s subscription. Hopefully,
you will have carefully considered this decision in advance and put steps to recover all the information that exists in
the different Office 365 applications. After a tenant subscription is cancelled, a formal lifecycle process begins that
eventually results in the complete and permanent removal of all tenant data from Office 365. Table 2-4 lists the steps
in the tenant removal process. For more information, see Microsoft’s online documentation .

Period Subscription Effect on data

Status

1-30 days Expired Users will see warnings about losing access but can continue accessing their
accounts and work with data.

31-120 days Disabled Only global or billing administrator accounts can access the tenant for the
purposes of either recovering data or to reactivate the tenant

After 120 days Deprovisioned An automated process starts to systemically remove all tenant data, including
inactive mailboxes and any others that were on hold.

Within 3 days of Expedited Upon customer request, the process to remove tenant data can be expedited to
ending subscription deprovisioning ensure that all data is removed within three days of the request.

Table 2-4: When tenant data is removed from Office 365

Note that the processes that remove tenant data after 120 days are constrained by resource availability. The data
might therefore be removed after 120 days, 121 days, or soon thereafter. If you need the data to be removed sooner,
you must ask Microsoft to start the expedited deprovisioning process by filing a support request. To confirm that
expedited removal should go ahead, Microsoft support gives the tenant a lockout code that the tenant must input to
the Office 365 Admin Center. After the code is entered, the data will be removed from Office 365 within three days. It
is critical to understand that once the data is removed it can no longer be recovered or reactivated.

Cancelling an Office 365 installation is a big step. The responsibility for the decision and what happens afterwards
stays with the customer, who can decide to let the cancellation process play out or reactivate the subscription within
the 120-day grace period.

The case of the annoying prompts : One thing that you are bound to notice after you cancel a subscription or a
trial subscription comes to an end is the number of annoying and persistent prompts Office 365 throws up to inform
administrators (or anyone who’s listening) that a subscription has ended or expired and some data might be lost.
And, of course, that all will be well if you would only buy the subscription now. The bad news is that there is no way
to suppress these notifications. You must wait until the countdown period (normally 30 days) expires and the
subscription disappears – or file a support ticket with Microsoft to ask them to eradicate the prompts.

The good news is that these notifications do serve a valid purpose in that they stop people forgetting that data might
be lost. It would be terrible if you overlooked a notification and ended up by losing some important data. Overall, it’s
best that the notifications are there, even if they are dreadfully annoying when you know that the subscription is
unwanted for good reason.

Identities
Before plunging into the intricacies of moving anything to Office 365, we should pause to consider some important
questions such as how people are identified within Office 365, how do they authenticate to the cloud to access
services, and the knotty topics of directory synchronization and federation. Fortunately, Chapter 3 has answers to
many of these issues, so that is our next destination.
Chapter 3: Identities and
Authentication
Paul Robichaux

Your online identity, and by extension the authentication process by which the identity is established, is the
cornerstone of security in Office 365. Once a user or object is successfully authenticated by Azure AD, access to any
authorized resource is automatically granted. Throughout the book, we’ll distinguish between two separate processes:

Authentication is when a system verifies the identity of a user. For example, the familiar Windows logon process
you go through is how Windows authenticates your domain or local user account.
Authorization is when a system decides whether a specific user should have access to an object or service.

In its simplest form, the Office 365 authentication process needs an identity, represented by an Office 365 User
Principal Name (which then becomes a security principal), and something by which their identity can be confirmed.
Often the authentication method is a password, but it can also be a certificate, a specific device, a biometric gesture,
or a combination of several methods (also referred to as multi-factor authentication). Once the service has confirmed
a user’s identity, it returns a token that an application or service can accept as proof of identity.

Identity and authentication are closely intertwined in Office 365. Depending on the identity model you choose, some
authentication options might not be available to you. For instance, if you decide to use online-only identities, you
cannot easily use on-premises Active Directory Federation Services to authenticate those identities to Office 365.

Although the default identity and authentication options suit most customers, not all organizations have equal security
requirements. Nonetheless, protecting your (digital) identity is an important task; both inside or outside of Office 365
or one of its workloads. After all, once someone can gain access to your account, they literally hold "the key to your
kingdom", granting them access to all the data accessible within the boundaries of your own identity. To meet the
needs of stricter security policies dealing with authentication, Office 365 supports a plethora of options to customize
and further secure the authentication process, as discussed in this chapter.

Because identities and authentication touch on so many aspects of Office 365, it is important to carefully consider the
implications of the various identity models and authentication options. The importance of factors such as complexity
and cost are different for every organization and can greatly influence the decision as to which solution is best for
you.

In this chapter, we explore the various identity models and authentication options that are available to you, spanning
both online-only and hybrid deployments. Before exploring the available options, we should first look at the Azure
Active Directory infrastructure which supports both identities and authentication for Microsoft online services.

Azure Active Directory

On-premises Exchange administrators are intimately aware of how Exchange relies on Active Directory to store much
of its configuration data plus information about mail-enabled objects. The original version of Active Directory,
introduced in Windows 2000, was like the Directory Store used between Exchange 4.0 and 5.5. Exchange 2000 was
the first enterprise application to go “all in” for Active Directory.

Exchange Online depends on Azure Active Directory and uses it in much the same way as on-premises Exchange uses
Active Directory, with some significant differences centered around how the Office 365 workloads leverage Azure
Active Directory and its own workload-specific directory, which we will get to shortly. However, the nature of the cloud
means that Azure Active Directory is a very different beast to its on-premises counterpart. The Azure Active Directory
deployment supporting Office 365 is, like most cloud implementations, designed to be a highly available multi-tier,
multi-tenant service that is capable of handling load at immense scale.

Every Office 365 tenant has a corresponding tenant within Azure Active Directory running within the service, and
each instance is isolated from others so that no tenant has access to data belonging to another tenant. In late 2017,
Microsoft claimed that Azure Active Directory contained nearly thirteen million separate directories belonging to
different companies and processes on average over ten billion client authentications weekly for more than 950 million
users (which grew to 1.1 billion by September 2018) and 5 million organizations worldwide . Azure Active Directory is
deployed in multiple Microsoft datacenters around the world in an infrastructure consisting of countless gateways,
front-end servers, application servers, and synchronization servers, all of which are found in each datacenter. In
addition to Office 365, Azure Active Directory is used by Microsoft Dynamics CRM, Intune, and other cloud services.
Custom- and third-party applications can also leverage Azure AD, either natively or published through something like
an Azure AD App Proxy.

Like other cloud services, Microsoft updates Azure Active Directory regularly to introduce new features and improve
quality. See the Azure Active Directory release notes for details. Many of these updates are focused on improving
Azure AD security. Microsoft faces an incredibly large volume of attacks daily against the service; for example ,
Microsoft reported seeing 23 million risky logon events in March 2018 and 4.6 billion attempts to replay passwords
stolen in various data breaches in May 2018 alone. The good news is that the improvements they make work to our
benefit as customers.

Azure AD SKUs and Office 365

When you first create a brand-new Office 365 tenant, the setup process also creates a new Azure AD tenant for you. I
refer to this as an “invisible” Azure AD tenant because you cannot see or directly manage it through the portal. It’s
not licensed separately, and you don’t pay anything extra for it. As you add users and groups, those objects go into
this Azure AD tenant, but you see and manage them through the Office 365 Admin center.

At any time, you can upgrade this invisible tenant to a full Azure Active Directory Basic tenant merely by going to the
Azure Active Directory link in the Admin centers section of the portal navigation bar. You’ll see the full Azure AD
dashboard. What you see is the functionality associated with what Microsoft calls the Azure Active Directory Basic
SKU. This includes most of the features we think of when we think of Azure AD, including the ability to set up
directory synchronization or federation. Many of the features you see in the Azure AD portal, such as reports on user
sign in activity or the ability to set up conditional access rules, require you to buy Azure AD Premium P1 or P2
licenses, but these upsell points are clearly marked so you’ll know what features require you to buy the additional
licenses.

Remember that guest accounts need user rights to use premium Azure Active Directory functionality. User rights for
guest accounts are acquired by assigning premium licenses to tenant users on a 5:1 (guest:tenant user) ratio. See this
guidance for more information.

Workload-Specific Directories
Because of the way that some Office 365 workloads work, Azure Active Directory cannot offer the necessary
functionality to support all the features required by those services. For example, the mailbox-specific attributes used
by Exchange Online are not in Azure Active Directory. The same is true for many of the user profile properties used by
SharePoint Online. To support workload needs, and to isolate the configuration and other data owned by a single
tenant inside the multi-directory Azure Active Directory, each service adds another layer on top of Azure Active
Directory. This layer is a workload-specific directory store and is named after the workload:

Exchange Online Directory Services (EXODS).

SharePoint Online Directory Services (SPODS).
Lync Online Directory Services (LYODS).
Yammer Directory.
Teams Directory.

This workload-specific directory is essentially a cache of all the workload-specific information held in Azure Active
Directory designed to deliver a certain level of redundancy against network or other disruptions. To further explain
the concept of workload-specific directory stores, we can use Exchange Online as an example.

Exchange Online Directory Services

Exchange Online uses EXODS to hold its configuration data, including information about mail-enabled recipients.
Exchange Online is deployed across multiple forests (30 or so according to the most recent Microsoft reports) with
tenants divided between the forests. Each forest is unique to an Office 365 region and has its own instance of EXODS
together with a synchronization endpoint that is used to replicate information with the directories used by Microsoft
Online Services (for user accounts and licensing), other applications (like SharePoint for mailbox information used in
eDiscovery), and Azure Active Directory. The 2017 introduction of support for spreading a single tenant’s data across
multiple Office 365 regions means that Microsoft has made additional changes to the way these forests are structured
and synchronized so that all locations where a tenant’s data may exist have complete and consistent data.

Synchronization occurs across the directories on an ongoing and continuous basis to ensure that the latest
information is always available to all parts of Office 365. Inevitability, some glitches can happen from time to time and
you might have to wait for a new Microsoft Online Services account to be visible to Exchange or vice versa.

Exchange Online can create new Microsoft Online Services accounts during the mailbox creation process. In this case,
synchronization pushes the information about the new account to the Microsoft Online Services directory and to
Azure Active Directory, which serves as the overall master directory for Office 365. Exchange Online is unique in this
capability, which exists because many customers have built extensive mailbox provisioning workflows with PowerShell
or Exchange Web Services that would break if forced to connect to Azure Active Directory or Microsoft Online
Services instead. When on-premises identities synchronize with Office 365 via the directory synchronization process,
for example in hybrid deployments, Azure Active Directory is the target and the resulting objects will synchronize to
EXODS afterwards.

The dependency of Exchange Online on Azure Active Directory and the geo-distributed nature of the cloud was proven
when a seven-hour outage for North American customers occurred on June 24, 2014 due to a failure in the segment of
Azure Active Directory that served the region. This meant that the service was unable to handle the inbound load from
clients (authentications) and email traffic (directory lookups). The net result was that huge queues of inbound and
outbound messages accumulated, and users could not send or receive email. Users outside North America were
blissfully unaware of the outage because Azure Active Directory remained fully functional in those regions. Since then
Microsoft has deployed local caches to handle lookups if a directory lookup fails so that Office 365 can accommodate
short-term outages without causing catastrophic queues to build.

A similar but different problem occurred on December 3, 2015 when a link created in error between production and
test environments caused user authentication traffic to be routed into the test infrastructure where it met a bug and
caused a five-hour outage for European Office 365 tenants. Outlook and Exchange ActiveSync clients continued
working during the incident because they could use cached credentials to connect to Exchange Online, but web-based
clients (including the service health dashboard) failed to authenticate and therefore were unable to connect.

To be fair to Microsoft, the kind of outages experienced by Azure Active Directory in June 2014 and December 2015
are like those found in on-premises deployments where customers scaled the number of clients connecting to
Exchange without due regard for the number of domain controllers and global catalog servers required to handle the
number of connections generated by clients. Since then, Microsoft has clearly learned their lesson as no major outage
of the same kind and scale have been reported since.

An Azure Active Directory Outage Affecting Office 365

On the other hand, this is not to say that there aren’t new, cloud-specific outages that can happen. In early September
2018, Microsoft suffered a worldwide outage that affected a wide range of Azure and Office 365 customers in several
different parts of the world. Microsoft’s root cause analysis (RCA) report for the outage states that the outage began
after a lightning strike near the San Antonio data center caused a voltage excursion that knocked cooling equipment
offline. This triggered a chain of events that resulted in a widespread outage affecting Azure AD, Visual Studio Team
Services , Office 365, and a host of other ancillary services. The cascading failures were caused by a combination of
planned safety measures and unexpected dependencies between services. Here’s an excerpt from the RCA:

As infrastructure shutdown from 9:29 UTC, Azure AD authentication requests routed as

expected to sites outside of South Central US. Requests were successfully being
managed until 11:00 UTC when the first signs of degradation occurred for customers
located in North America, predominantly due to three factors. First, the AAD autoscale
operation that is designed to scale ahead of demand did not complete. The autoscale
failure was due to a dependency on the Azure ASM API which, as discussed earlier, was
unable to process service management operations. Second, as AAD sites neared safe
utilization thresholds, Quality of Service throttling mechanisms engaged, resulting in
authentication failures and timeouts. Third, the engineering team discovered a bug that
triggered an unexpected behavior in Office clients which resulted in aggressive retry
logic, exacerbating the increased load on the service.

Note that the problems occurred due to a dependency between AAD autoscale and another service (ASM) that doesn’t
support automatic failover. The failure of AAD autoscale meant that the QoS throttling mechanisms were invoked, and
worked as designed to restrict load, with the side effect of preventing clients from connecting; on top of it all, there
was a bug that had lain undiscovered in the service for some time that was only triggered under these conditions.

This type of complex, multiple-failure event is not common, thankfully. However, it’s a good illustration that failures
can still occur in the cloud, even given Microsoft’s huge investment in service quality and management.

Identity Models for Office 365

There are two options for how you establish Office 365 user accounts (also referred to as “identities”):

Standalone identity : In this model, user accounts exist only within the Office 365 environment and are not
linked or related to any other Active Directory environment. This means that a user may have an account in an
on-premises Active Directory, as well as an account in the Azure Active Directory that supports the organization’s
Office 365 tenant. The accounts may happen to have the same username and password, as well as other
attributes, but are separate and independent objects, and need to be managed independently as well. This
duplicates administrative effort and introduces risks if attributes conflict or are not maintained correctly.
(Microsoft used to call this configuration “cloud-only identity” but now they call it “standalone” and we’ve
adopted their terminology).
Hybrid identity: In this model, user accounts are created and managed in the on-premises Active Directory
environment and synchronized to Azure Active Directory (and thus to Office 365). The synchronization is
performed by a directory synchronization tool; the only supported tool is Azure AD Connect. In the hybrid
identity model, the on-premises Active Directory is the “source of authority”, with objects and attributes
replicating from on-premises to the cloud. Because the on-premises Active Directory is now the source of
authority, an administrator can only change limited aspects of the synchronized identity in Office 365. More
information on these limitations is provided in Chapter 5 of the companion volume.

A synchronized identity exists in the cloud, but users still need a way to authenticate using that identity plus a
password. There are three ways to do this:

Password hash synchronization : In this model, a cryptographic hash of the user’s password, but not the
password itself, is synchronized to the cloud. When a user requests access to a service component, the password
they provide is hashed; if that hash matches the stored hash, the passwords are considered to match. This
process relies on directory synchronization but doesn’t require any other servers or components.
Pass-through authentication (PTA): in this model, user accounts are synchronized from on-premises domain
controllers to Azure Active Directory. Authentication requests from the service are sent to a queue, where they
are retrieved by a small agent that runs as part of Azure AD Connect. The agent validates passwords and returns
an access token if the validation succeeds.
Federated identity: In this model, user accounts are also populated in Azure Active Directory through directory
 synchronization, but authentication requests from the service are passed to an on-premises federation server.
This can be Microsoft Active Directory Federation Services (AD FS), or a third-party federation service such as
Ping Identity or Okta. The federation server is responsible for authenticating the user by passing an
authentication request to an on-premises AD domain controller, then returning an authentication token for the
user to access cloud services. This process provides the end user with a single sign-on (SSO) experience that can
be used to access various Office 365 services.

Each approach has its own pros and cons. For example:

Standalone identities are convenient because they do not need an on-premises Active Directory. Most small
organizations using Office 365 don’t have, or want, on-premises AD. However, if an on-premises Active Directory
is available, cloud identities can be more time consuming to manage because two separate accounts need to be
managed for each user. There is no arbitrary threshold for user objects to manage before synchronizing identities
makes more sense. It typically comes down to how much time and effort an administrator must put into
managing dual identities versus maintaining a synchronization infrastructure.
Directory synchronization simplifies administration because all changes are typically made to the on-premises
Active Directory and then synchronized to the cloud by a synchronization infrastructure that must be managed
and maintained. However, if password hash sync (PHS) is used, Office 365 can authenticate users when the on-
premises Active Directory is unavailable (for example, if a power failure occurs at the on-premises datacenter).
Even when using directory synchronization, you can still create and use cloud-only identities (for instance, for the
group objects created by Office 365 Groups, or for external users who only need access to Office 365 resources).
Federated identities give organizations much greater control over how to enforce security policies such as logon
hours, third-party multi-factor authentication, and network locations from which users can access Office 365
resources, as well as any other cloud service that uses AD FS. However, the identity federation infrastructure
must scale to meet performance and workload requirements. It must also be highly available and resilient to
failure because Office 365 may not be able to authenticate user logons if the federation service is not available. It
is possible to use password synchronization as a fallback option when your federation infrastructure has failed.
However, this is not an automatic process and must be considered during the planning and deployment phase.

You can deploy mixtures of these models. Although it is more common to settle on a single identity model,
organizations can choose to combine different identity solutions to fit their specific needs. Ultimately, the decision
about which identity model to use is driven by the business and technical requirements of the organization, as well as
the selected migration method. Several migration methods are available to move accounts to Office 365. Each comes
with a different approach to identity creation and management. Migration methods are covered in Chapter 2 of the
companion volume.

A late 2017 report show s that standalone and federated identities are, by far, the most popular methods in use today:
21% of all authentications against the service are standalone identities, but a whopping 46% of total authentications
flow through AD FS. PHS claims 25%, and the balance is made up by other services and tools. Only 3% of all
authentications are performed by non-Microsoft federation or identity services, such as Ping Federate, Okta, Centrify,
etc. That only a small number of organizations opt to deploy a third-party solution is not surprising considering that
using a third-party solution adds cost and often introduces additional management and complexity. The May 2018
announcement by Microsoft of full support for Ping Federate in the Azure AD Connect tool may change this somewhat.
Low adoption rates doesn’t mean that third-party solutions are not valuable. They often deliver additional features on
top of what AD FS provides, including the ability to federate with non-Microsoft applications, which is why many large
companies such as Visa, Pacific Gas & Electric, and HCA use them. As such, they can help solve very specific
problems or allow you to integrate with third-party solutions beyond Microsoft’s capabilities.

Standalone identities
The on-premises versions of Exchange, SharePoint and Skype for Business all use Active Directory as the repository
for user objects and to process authentication requests. In the case of Exchange, every mail-enabled object is
represented by an Active Directory object. The properties of the objects are managed by different tools, including the
Active Directory Users and Computers console, the Exchange Admin Center (EAC), and PowerShell. Much the same
division of responsibilities exists in the cloud versions of these services, with the exception that Azure AD is used
instead of on-premises domain controller. For example, Exchange Online uses the same properties for mailboxes,
public folders, groups, and other mail-enabled objects as the on-premises version, but these properties are divided
across, and managed through, both Azure Active Directory and in the Exchange Online Directory Store. Although a
synchronization process keeps the two online directories in step, Azure Active Directory is always the master. The
Users and Groups sections of the Office 365 Admin Center replaces the Active Directory Users and Computers
console and a modified version of the EAC provides options to manage Exchange Online.

Every Office 365 user account has a unique user identifier, known as the User Principal Name (UPN), which is used to
log on to Office 365. For example: BWatts@Office365ITPros.com

By default, Office 365 sets the primary SMTP address for an object to be its UPN and things usually stay that way.
However, this does not have to be the case as the two properties serve different purposes: the UPN identifies the
object to Office 365 and the primary SMTP address routes messages to its mailbox, if it is mailbox-enabled. More
information about how to use the EAC and PowerShell to manage Office 365 objects is in Chapter 5.

It is reasonably common for administrators to need to change the User Principal Name, SMTP address, and display
name for user accounts. For instance, this would be the case when someone marries or divorces and wants their
identity to reflect their new status. In the following example, we use PowerShell to switch names for a cloud-based
user from Jane Smith to Jane Jones and update the UPN and primary email address to reflect the same. Note that we
indicate the new primary SMTP address by prefixing the address with “SMTP” (in capitals) while prefixing the other
SMTP addresses with “smtp”. It is good practice to retain old addresses assigned to accounts in the past so that
Exchange can still route replies to those old addresses to the correct mailbox. To set the new values, use the Set-
AzureADUser cmdlet. The updated information will synchronize from Azure Active Directory to the workload-specific
directories such as EXODS. Finally, we use the Set-Mailbox cmdlet to add the new primary SMTP address to match
the updated User Principal Name.

[PS] C:\> Get-AzureADUser -ObjectId "Jane.Smith@Office365ITPros.com" | Set-AzureADUser –Surname "Jones" –DisplayName "Jane Jones" -
UserPrincipalName 'Jane.Jones@office365ITPros.com'

[PS] C:\> Set-Mailbox –Identity ‘Jane Jones’

–EmailAddresses SMTP:Jane.Jones@Office365ITPros.com, smtp:Jane.Smith@Office365ITPros.com

PowerShell is the preferred method to execute commands for multiple accounts or to action adjustments originating
in another system, like a HR system. If you only need to change the User Principal Name or the email addresses for a
small number of accounts, you can do this from the Office 365 Admin Center by:

Selecting the account to update.

Editing the user/name/email aliases.
Adding the new email address (or alias ) to the account
Clicking Set as primary to make the new alias the primary email address for the account. This also makes the
address the User Principal Name for the account.

When you change the User Principal Name for an account, the account owner must provide the new User Principal
Name the next time they sign into any Office 365 service. If they use Skype for Business, previously scheduled online
meetings must be updated, as the join-URL for those meetings will have changed. The Skype for Business Online
meeting migration service will automatically do that for you, if your mailbox is also hosted in Office 365. If your
mailbox is still on-premises, you must manually run Microsoft's Meeting Update tool from the client.

Changing a UPN for a cloud-only identity is straightforward. However, when an identity belongs to a federated
domain, things can become a little more complicated. More information on how to change the UPN for a federated
identity is available in the Federated Identities section below.

Service accounts
Not all the accounts you use with Office 365 belong to users. Some applications, such as those that monitor workload
health or report on various aspects of Office 365, might require the creation of special service accounts. that are
solely used for administrative purposes. These accounts usually do not need a mailbox or a license, but probably need
the assignment of some form of administrative permissions to be able to view and access data on behalf of the tenant.
A good example of the directions given by vendors to set up a suitable service account can be found here .

Most service accounts are created so that their passwords never expire. This is done on the basis that password
expiry usually happens at the most inappropriate time, so it is better to assign a complex password to the service
account and leave it alone thereafter. You might also find it useful to set non-expiring passwords on other
administrative accounts. This is easily done with PowerShell. The first example below sets the “password never
expires” flag on an administrator account. The second scans the set of user accounts and reports those that have this
flag set. Note that these examples only apply to cloud accounts. If you synchronize an account from the on-premises
Active Directory and password synchronization is enabled, the password of the corresponding cloud account is
automatically set to not expire. In such case, it suffices to also configure the on-premises account’s password to not
expire. Note that when the password policy is successfully updated, no message is returned. However, if an error
occurs, a message is displayed.

[PS] C:\> Get-AzureADUser -ObjectID Administrator@Office365ITPros.com –PasswordPolicies "DisablePasswordExpiration"

[PS] C:\> Get-AzureADUser | Where {$_.PasswordPolicies –eq "DisablePasswordExpiration"} |

Format-Table DisplayName, UserPrincipalName

Hardening User Passwords: Office 365 allows accounts to have passwords of up to 16 characters. Although most
Office 365 account passwords are likely shorter than this, some find that 16 characters is not secure enough.
especially when a tenant might allow passwords to not expire. Increasing the password length by just a single
character dramatically increases the time to brute-force guess the password. However, as computers become
increasingly more powerful, the processing needed to brute-force guess a password also reduces and continually
increasing the password length is not a long-term solution. A paper written by Microsoft Research makes the
following recommendations:

1. Maintain an 8-character minimum length requirement (and longer is not necessarily better).
2. Eliminate character-composition requirements.
 3. Eliminate mandatory periodic password resets for user accounts.
4. Ban common passwords, to keep the most vulnerable passwords out of your system.
5. Educate your users not to re-use their password for non-work-related purposes.
6. Enforce registration for multi-factor authentication.
7. Enable risk based multi-factor authentication challenges.

In addition to hardening user passwords, you should also monitor authentication attempts to track and report on
suspicious activities. Both options are discussed later in this chapter. For even better security, you should enable
multi-factor authentication (MFA) on your user accounts.

Synchronized identities
Many small organizations rely solely on cloud-based identities. However, organizations that have existing on-premises
Active Directory are much more likely to use directory synchronization to synchronize their on-premises identities to
Azure Active Directory. Although directory synchronization is most often associated with a hybrid Exchange
deployment, many organizations use it even when Exchange on-premises is not in the picture as it is the only way to
avoid duplicate management of objects. There is no magical number that states what the threshold is for
implementing directory synchronization, nor is there any lower limit. Obviously, the larger the on-premises
organization, the more sense it makes to deploy directory synchronization.

Depending on what type of object is synchronized, Azure Active Directory will have a matching object that, in turn, is
synchronized with an instance of a workload-specific directory service. For example, an on-premises mailbox is
represented as a mail-enabled user account in Exchange Online and vice versa. As mentioned previously, only a
handful of properties of a synchronized object can be managed directly in the cloud. Attempting to edit other
synchronized properties through the Office 365 Admin center or online PowerShell will generate an error. For
example, this error is flagged when an administrator tries to hide a user from the Address Book by running the Set-
Mailbox cmdlet in Exchange Online:

[PS] C:\> Set-MailUser MSpencer -HiddenFromAddressListsEnabled $True

The operation on mailbox "Mark Spencer" failed because it's out of the current user's write scope. The action 'Set-MailUser',
'HiddenFromAddressListsEnabled', can't be performed on the object 'Mark Spencer' because the object is being synchronized from your
on-premises organization. This action should be performed on the object in your on-premises organization.

This happens because directory synchronization is a mostly one-way operation that flows from the on-premises
organization to Azure Active Directory. The on-premises version of the object is the authoritative source. Objects that
are created directly in Office 365 are considered cloud-only objects and are, by default, not synchronized back to the
on-premises Active Directory. The only exception to that rule is when you have a writeback feature enabled. Writeback
is a capability in Azure AD Connect which permits the synchronization of some recipient types (such as Groups) and
attributes (such as mobile device partnerships) back to the on-premises directory. Note that the decision to implement
writeback can affect how hybrid recipients are managed. The various writeback capabilities and their limitations are
discussed later in this chapter.

The danger of bidirectional updates : If Microsoft allowed you to make changes to a synchronized object in Office
365, the online object might end up having different attribute values in its on-premises counterpart because changes
are, by default, not written back into the on-premises Active Directory. This would cause some features to break. Let's
consider the following scenario: Julia Spencer's mailbox is moved from the on-premises Exchange organization to
Office 365. As part of the process, the on-premises object is converted into a mail-enabled user object and the
targetAddress attribute of that object is built using the mailbox its routing domain to create a value such as
jspencer@office365itpros.mail.onmicrosoft.com .

The email address set on the account allows messages sent from the on-premises organization to be routed to the
mailbox in Office 365. When trying to deliver a message to the recipient, the on-premises Exchange servers will see
that the targetAddress attribute is not empty and use the email address in the targetAddress attribute to forward
messages to. This means that messages addressed to Julia.Spencer@office365itpros.com are ultimately forwarded to
jspencer@office365itpros.mail.onmicrosoft.com . Now, imagine that an administrator updates the online object
because the user changes her last name to “Miller.” As part of the update, the administrator decides to also update
the email address and various aliases including the one for the routing domain. Provided that the administrator does
not make the same changes on-premises, this means that the online object and the on-premises object are now in an
inconsistent state. If someone in the on-premises organization sends a message to the user, the on-premises
Exchange servers will still attempt to forward the message to jspencer@office365itpros.mail.onmicrosoft.com . When
the message arrives in Exchange Online, Julia's mailbox now has a different routing address stamped to it (for
example, jmiller@office365itpros.mail.onmicrosoft.com ). As a result, Exchange Online cannot deliver the message
and a Non-Delivery Report (NDR) is generated.

Despite the general rule that objects synchronized from on-premises cannot be managed using online tools, some
mailbox features can be managed in both environments. More specifically, this applies to features for which the
synchronization process will write back attributes to the on-premises environment, or for which no on-premises
management capabilities exist. For example, a litigation hold can be enabled on an Exchange Online mailbox
associated with a hybrid identity because the update made to the mailbox attributes can be synchronized back to
Exchange on-premises.

To complicate things a little more, some features of a hybrid recipient can be managed within Office 365. For
instance, in a hybrid deployment which does not use Active Directory Federation Services or password hash
synchronization, the password for the user account is managed by Azure Active Directory. You will find that in this
case certain actions, like resetting the password or configuring it to not expire, can be performed using online
management tools despite directory synchronization being enabled.

Identifying which objects are synchronized from Active Directory is straightforward. If you manage user objects from
within the Office 365 portal, the default view includes a status field ("Sync Type") which states Synced with Active
Directory . Similarly, you can use the PowerShell Module for Azure Active Directory to run the following query:

[PS] C:\> Get-AzureADUser -Filter "DirSyncEnabled eq true"

ObjectId DisplayName UserPrincipalName

-------- ----------- -----------------

4f9f6f4e-f12c-4528-949a-86bfb588048c Marc Spencer mspencer@o365itpros.com

7561973e-3ec1-4c42-bdbc-aa336dededc4 Billy Weaver bweaver@p365itpros.com

As this interesting blog post points out, there’s a quirk in the behavior of the Get-AzureADUser cmdlet when
retrieving details of synchronized or non-synchronized users: it returns true or null to indicate whether an account is
synchronized. In the above example, the filter property uses server-side filtering to find matching objects before
passing retrieved objects to the pipeline. This can significantly increase performance when working with larger sets of
objects. However, server-side filtering does not support the use of all operators. For example, you cannot use
“DirSyncEnabled ne true ” to query for non-synchronized accounts. As explained in the blog, all (user) objects go to
the pipeline. While this approach can consume more time and memory – especially when working with large object
sets– it allows for more flexibility, such as using the “NotEquals (-ne) ” parameter which allows you to filter for non-
synchronized users, etc.

Operating a hybrid Exchange deployment does not mean you are limited to hybrid recipients. An organization can
mix-and-match various recipient types. For instance, alongside the synchronized objects, you can have several cloud-
only objects such as tenant administrator accounts. Unlike the synchronized objects, these cloud-only objects can only
be managed using the online tools.

Real-world use case: The approach of also using cloud-only recipients in a hybrid deployment is one that is
commonly found in organizations that need to collaborate with external partners but have no need to provide those
partners with access to on-premises resources. In such case, adding a user account on-premises and synchronizing it
to Office 365 will cause more overhead than provisioning a cloud-only account in Office 365.

Changing the User Principal Name for a synchronized user

Historically, the directory synchronization process did not update the User Principal Name (UPN) for a synchronized
user when an Office 365 license is assigned to that user. Although you could still change the UPN as explained in the
Cloud-only Identities section, this approach needs a manual update for the UPN, which is not very practical if you
need to update many UPNs at once.

In a change Microsoft made to its synchronization service, tenants created after June 15, 2015 automatically update
the UPN of a synchronized account in Office 365 through the directory synchronization process, even if it's licensed.
Tenants created before that date must enable the feature by issuing the following command in Azure AD PowerShell:

[PS] C:\> Set-MsolDirSyncFeature -Feature SynchronizeUpnForManagedUsers -Enable $true

The change in how the synchronization service handles UPN updates for managed (synchronized) accounts does not
apply to federated identities. More information on how to change the UPN for federated identities is available in the
Federated Identities section.

Hybrid Writeback Attributes

Although the synchronization of objects is usually a one-way operation to Office 365, a handful of object attributes are
written back into the on-premises organization if you enable hybrid mode while setting up the directory
synchronization tool. The writeback attributes are necessary to support some features in a hybrid deployment, such as
the ability for an on-premises mailbox to access a cloud-based archive. Currently, there is no way to select which
attributes are synchronized back into the on-premises organization from Office 365. Table 3-1 lists the various
attributes that are currently written back into the on-premises Active Directory from Office 365.


Attribute name Function

msExchArchiveStatus Enables the hosted online archive feature for an on-premises mailbox.

msExchUCVoiceMailSettings Used in on-premises Lync or Skype for Business deployments to show if a user has
voicemail in Exchange Online.

msExchUserHoldPolicies Enables Exchange on-premises to figure out which (online) mailboxes have Litigation
Hold enabled. This is important to ensure that the Litigation hold is persisted when a
mailbox is moved back from Office 365.

proxyAddresses Used to ensure that mail flow continues to work after a mailbox is moved back from
(LegacyExchangeDN as Office 365 to the on-premises Exchange organization.
X500)

msExchSafeSendersHash The information that Outlook collects as part of its safe list aggregation is written into
these attributes and then synced back. This is particularly important in hybrid
msExchBlockedSendersHash deployments that have the centralized mail flow option enabled and use an on-premises
filtering solution which can take advantages of these attributes; for instance, an
msExchSafeRecipientsHash Exchange Edge Transport Server.

msDS- This attribute is introduced in Windows Server 2016 (or Exchange Server 2016). It is
ExternalDirectoryObjectID derived from the ObjectID attribute in Azure Active Directory. The value of this
attribute equals the ObjectID of the connected user account in Azure AD, prepended
with “User_”. For instance: User_fca22808-8f7c-4443-af29-ddc3d10bb8f5. Although the
documentation states the attribute is new in Exchange Server 2016, it also becomes
available if you introduce a Windows Server 2016 Domain Controller into your on-
premises Active Directory environment. As such, you might find that you run Exchange
2010 and need to configure the appropriate permissions to allow synchronization to
write this attribute too!

publicDelegates This attribute stores information about who has (delegate) access to this mailbox.
Exchange Online uses the cloudPublicDelegates attribute to control which users have
Send-On-Behalf-Of permissions, while on-premises Exchange uses the publicDelegates
attribute. Versions 1.1.553.0 and later of Azure AD Connect enable writeback on this
attribute, which means that you can now synchronize sending permissions in hybrid
organizations.

Table 3-1: Writeback attributes

Note: Microsoft is working to enable the ability to write back more than the limited set of attributes that
synchronization currently supports. When Microsoft delivers that capability, it might enable scenarios where hybrid
recipients can be managed from both environments. Today, various writeback capabilities are available in preview.
Each of these features has its own set of objects and object attributes that are written back into the on-premises
directory. We will discuss the various writeback capabilities later in this chapter.

The idea that the directory synchronization process can potentially modify object attributes in the on-premises
environment can be scary at first. However, it is important to understand that updates happen securely. First, SSL
encryption protects communications between the on-premises directory synchronization server and Azure Active
Directory. Secondly, the account created in the installation of Azure AD Connect only holds the minimum set of
permissions required. The account can write back to objects by assigning explicit object attribute permissions to a
Security Group called MSOL_AD_Sync_RichCoexistence , of which it is a member. We will explore the permissions and
how to create a service account for a custom Azure AD Connect setup later.

Extended Authentication Capabilities

In the early days, the directory synchronization engine only allowed to synchronize identities. At that time,
authentication was either performed in the cloud (for standalone identities) or through an on-premises authentication
solution (Federated Identities). While this is still true, Microsoft have added the ability to configure password
synchronization and, more recently, added support for pass-through authentication and single sign-on without having
to federate the domain.

Federated identities
Federated Identities are very like synchronized identities, with the exception that authentication is no longer handled
solely by Azure Active Directory, but by an on-premises Active Directory Federation Services farm or a third-party
federation solution. A by-product of configuring identity federation is that it can provide a single sign-on experience
(SSO). SSO is when users authenticate to both the on-premises organization and Office 365 using the same username
and password without having to type their credentials again whenever they access an Office 365 resource. Don’t
confuse this experience with what you see when password hash synchronization is enabled—in that case, users use
the same password to log in to on-premises and cloud services (leading to the nickname “same sign-on” for this
configuration) but they have to log on separately to each type of service.

When an object synchronizes with Azure Active Directory, it automatically becomes a synchronized identity. Despite
offering a similar sign-on experience, password synchronization is not an identity federation solution as Azure Active
Directory still confirms authentication requests.

When you federate a domain, the responsibility for validating authentication requests is shifted towards the federation
solution. The application that’s asking the federation system to perform an authentication is known as the relying
party (because it’s relying on the federation service). Third-party federation applications generally makes requests to,
and gets responses from, the federation server using a protocol known as the Security Assertions Markup Language
(SAML), while AD FS uses a competing standard named WS-Fed/WS-Trust. Both of these standards specify the format
and content of data and metadata that the service and federation broker can use to negotiate and perform the
authentication. Both use the concept of claims, which are statements made by one party about another. For example,
the application might send a claim to the federation server that says “the user requesting authentication is coming
from IP address 172.16.0.204” and the federation server might reply with a set of claims that includes “the email
address associated with the account you’re trying to authenticate is paul@office365itpros.com . ” You can configure
claims rules that allow or restrict authentication based on the contents of these claims.

Federation is enabled per domain and all users in Azure Active Directory for whom the domain portion of the UPN
(UPN suffix) matches a federated domain are automatically considered to be federated identities. Even with federated
identities, the first authentication request is still received by Azure Active Directory. The user's UPN suffix is
examined and, if the domain name is federated, Azure Active Directory will redirect or proxy the authentication
request to the customer's federation solution. This process is called home realm discovery and needs to be performed
for each initial authentication (such as the first time that someone logs on). You can see this process at work when you
have federation set up and you visit an Office 365 logon page; at first you see the standard Microsoft logon dialog,
which contains code that looks at the right-hand side of the entered UPN and determines if it’s a federated domain,
redirecting to the organization’s federation solution if so.

The details of the customer's federation solutions and all the relevant details such as the federation endpoint and
certificate information are stored in Azure AD during the federation setup process. The initial request (home realm
discovery), and the subsequent redirects all happen within a matter of seconds, often transparent to the user.

Most organizations choose to federate all their registered domains, but that is not a requirement. As such, you can
have multiple authentication methods for different users in different domains of your organization. Information on
how to configure identity federation with Active Directory Federation Services is described later in this chapter.

When authenticating to Office 365, you will notice that most of the time the user is asked to enter their email address.
However, that is not entirely true. Although the web page or application might ask for the "Email Address" on screen,
the user should enter their logon name (User Principal Name) to complete the logon. Because of this, the general
recommendation is to align the user's UPN and primary email address to remove any potential for confusion for the
end users.

Changing the UPN of a Federated User

It is best to ensure that the UPN and primary email address match from the start. In some cases, this might mean that
you must update the on-premises UPN value for all your users prior to synchronizing them to Office 365.

If you previously already synchronized the objects, and now need to update the UPN, there are some things you must
keep in mind. Although it is possible to change the UPN prefix (everything before the @ sign) using the process
described earlier, you cannot change the UPN suffix (domain name portion) without an intermediate step. Additionally,
changing the UPN of a federated identity, without changing the UPN on-premises can also negatively affect the end
user experience.

Consider the following scenario: You have two federated domains, office365itpros.com and
office365exchangepros.com. A user with UPN BWatts@office365exchangepros.com synchronizes with Office 365. You
want to change the UPN of the user into BWatts@office365itpros.com . First, you change the UPN on-premises, but
then discover that the directory synchronization process did not synchronize the change. Instead, the synchronization
engine reported a synchronization failure for that account. You then decide to use the Set-MsolUserPrincipalName to
change the UPN into its new value, only to see a meaningless error "Unable to complete this action ."
This is the expected behavior: you cannot move a user from one federated domain to another without first changing
the user into a non-federated identity. As such, the correct procedure is to modify the UPN of the user into something
like BWatts@tenantname.onmicrosoft.com (or another non-federated domain registered in the tenant), and
immediately afterwards update the user's UPN to BWatts@office36itpros.com .

Keep in mind that you should also update the UPN on-premises to align the UPN in Office 365 with the on-premises
UPN. Once you have made the update in Azure AD and the on-premises Active Directory, the synchronization error for
that account will automatically disappear. Note that, while making these changes, you will affect the user's ability to
sign in, albeit briefly.

Alternate Login ID . Just like many other cloud services, Office 365 requires the logon ID to be ‘internet routable’
because ownership of non-routable domains like internal domains ending in “.local” or “.ad” cannot be verified.
Users for which the on-premises UPN suffix does not match any registered domain are given a UPN suffix based on
the default routing domain. For instance: mspencer@office365itpros.onmicrosoft.com . Note that the tenant's default
domain cannot be federated.

By default, when you configure AD FS, the UPN is used as the primary logon ID for Office 365. Generally, it is
recommended to ensure the UPN matches the user’s email address so that they only need to remember their email
address to sign in to Office 365. Unfortunately, sometimes you might not be able to change the UPN to match the
email address. This is often the case when some legacy application expects a specific UPN value and cannot be
modified easily. If you are unable to change the UPN for your users, for example because a legacy application needs a
specific value, you can use an AD FS feature called “Alternate Login ID". This feature allows you to specify which
attribute – other than the UPN – should be used to sign on to Office 365. For instance, you can configure the actual
Email Address attribute to be the new identifier. Although Microsoft recently decided to support Alternate Login ID
for Hybrid deployments, there are some severe drawbacks from an end-user perspective (like extra authentication
prompts) Unless you absolutely cannot reconfigure the UPN, I would not recommend choosing Alternate Login ID.

In this article, Microsoft describes the end-user experience for various applications and protocols if Alternate Logon
ID is configured. Connections from within the corporate network are most likely to work just fine. External access,
regardless of the client, can be problematic and result in extra authentication prompts. No matter what approach you
use, the end user experience suffers as soon as Alternate Logon ID is enabled. If you decide to enable Alternate Login
ID, I absolutely recommend enabling Modern Authentication (described later in the chapter) as it helps to alleviate
some of the problems related to the added authentication prompts.

To Federate or Not?
Various elements influence the decision of whether identity federation is the right solution for you. Apart from the
added infrastructure that is needed, complexity is often the reason why organizations avoid federation, especially
when they must manage the federation solution themselves. It is unsurprising that password hash synchronization is
the fastest growing authentication method in Azure Active Directory, as it's much easier to deploy and maintain, and it
offers a similar login experience to end users. In addition, users will still be able to authenticate, even when the on-
premises infrastructure is down; the only thing that would (temporarily) stop working in such a scenario, is the actual
synchronization of passwords.

On the other hand, identity federation (for example, through Active Directory Federation Services) can solve very
specific problems, like whether to block authentication from outside the corporate network or to limit access to Office
365 services to members of a specific group. In addition, it is also the only way to integrate with third-party multi-
factor authentication solutions.

The introduction of pass-through authentication (PTA) in Azure AD Connect complicates your choice a bit further. The
decision point on whether to use PTA or federation boils down to whether or not you need true federation. For
example, you can use AD FS to provide single sign-on with Amazon Web Services, service desk management solutions,
employee benefits systems, and other cloud applications from third parties, whereas PTA doesn’t support these
applications. For most organizations, PTA is simpler to implement, and still allows you to take advantage of features
such as conditional access through Azure or Enterprise Mobility + Security.

Of course, the biggest perceived benefit is that identity federation allows end users with corporate domain-joined
devices to login to Office 365 services without having to continually re-enter their credentials. As outlined in Table 3-2
that statement is only true in very specific scenarios and it also depends on the version of the client and whether
Modern Authentication is enabled. More information on Modern Authentication, and the client experience in
combination with a federated identity is described in the Modern Authentication section.

Rich Applications (Skype for Web-based Apps Exchange Clients (Outlook,

Business, Office) – Sign In (OWA, EAS) – Basic Authentication
Assistant SharePoint)


Cloud Identity Specify Cloud Specify Cloud Specify Cloud

Username and Password Username and Username and Password

Password

Federated Identity – Potential Single Sign-On (no Potential Single Specify Active Directory
Domain Joined Client prompt) Sign-On (no
prompt) Username and Password

Federated Identity – Specify Active Directory username Specify Active Specify Active Directory
Non-Domain Joined and Password Directory
Client username and Password
username and
Password

Table 3-2: End-User Sign-In Experience

Note: Most organizations allow users to save their credentials to avoid having to re-enter them every time an Office
365 workload needs authentication. The single sign-on experience also depends on how users access their domain-
joined computer. For SSO to work, they must use their UPN to log on to their computer. Remote users, workgroup
users, Mac users, and users on mobile devices will still have to sign-in to AD FS every time they access an Office 365
workload for the first time.

Third-party Federation Solutions

Active Directory Federation Services is not the only federation solution to enable Single Sign-On. Several third-party
solutions, such as those from Okta, Ping Identity, and OneLogin take a similar approach to federation, albeit without
needing the deployment of extra on-premises servers. Many of these solutions have gone through a validation process
called “ Works with Office 365 ”. This means that these solutions have been tested by Microsoft and that you can
substitute AD FS with one of these solutions. In fact, in April 2018 Microsoft formally added support for Ping Federate
to Azure AD Connect, putting Ping’s solution on equal footing with AD FS.

Whether to use a third-party solution for Office 365 or not depends on several factors such as integration with other
cloud systems. In addition to being simpler to implement than AD FS, the benefit of these third-party federation
solutions is that you don’t have to deploy servers on-premises and they often support many other cloud systems,
allowing you to deploy a single solution to support all your cloud applications. After all, it is likely that an organization
is using other clouds solutions besides Office 365, and that these cloud platforms do not necessarily support AD FS.
Instead of keeping both AD FS and a third-party solution, it is probably easier to use a solution that works with all
platforms.

Note: Azure Active Directory also offers advanced SSO features, like those supported by third-party vendors. In
addition to these SSO features, Azure Active Directory contains additional features such as enhanced multi-factor
authentication and self-service password management options for Office 365. Many of these features require users to
have a license for Azure Active Directory Premium or the Enterprise Mobility & Security Suite (at an additional cost),
but it is worth evaluating along with other third-party solutions.

Authentication
Establishing an Identity in Office 365, whether as a cloud-only or synchronized identity, is only half of the work. Once
you have an identity in Office 365, you must first authenticate before you can access resources online. Leveraging the
power of Azure AD, Office 365 offers a variety of authentication options, ranging from the default username/password-
combination to more advanced authentication solutions as part of the Enterprise Mobility & Security suite offerings.

Before diving into the advanced scenarios, let's first look at how authentication is performed in Office 365. The
simplest credential that can be used for authentication is the combination of a username and a password. This option
is also the default in Office 365 for all non-federated identities.

When a user attempts to access resources in Office 365, the user is prompted for credentials. Depending on what
client is used, the authentication prompt comes in various forms. One such example is the username/password form
on the Office 365 portal web page, but each client can implement its own interface. Microsoft has worked hard to
standardize the appearance of the Azure AD-based logon interface across clients and platforms, although starting in
late 2017 they began tweaking it again and so you may notice some differences between clients and platforms. No
matter the interface, once the user provides a credential, various things happen behind the scene depending on what
client is used and what authentication method has been configured.

Basic Authentication
When looking at the authentication process in Azure AD, one can distinguish two main approaches to authentication:
basic authentication and modern authentication. The former is used when legacy clients, those that do not support
modern authentication capabilities, connect to Office 365. For example, this is the case when an ActiveSync client
connects to a mailbox in Exchange Online.

When a basic-authentication client connects, a basic authentication prompt is shown. Once the user enters their
credentials, they are forwarded to the service (Exchange Online) which, in turn, will connect to the underlying
authentication solution (Azure AD or AD FS) to validate the credentials. If the credentials are successfully verified, the
service will authorize the client to connect to the resource it requested access to.

This process is quite different from Modern Authentication where the client communicates directly with the
authentication endpoint.

Note: Microsoft has been pushing customers to stop using legacy authentication-- or, more precisely, to upgrade
their clients to versions that support Modern Authentication-- because legacy authentication methods, such as basic
authentication, impose major limitations. For example, you cannot perform multi-factor authentication using basic
authentication, nor can you exert the same levels of controls (through policies) as with the modern authentication
options.

Disable Basic Authentication for Exchange Online

In mid-October 2018, Microsoft announced a public preview of a long-awaited feature: you can now turn off Exchange
Online support for basic authentication. The blog article announcing this change makes for very interesting reading; it
points out that there is no telemetry that allows tenant admins to see which users are using basic authentication, so
there’s a risk that you might accidentally shut off access for some clients if you disable basic authentication
indiscriminately. The documentation on how to apply authentication policies to enforce this change is mandatory
reading if you’re even thinking about making this change.

Modern Authentication
Modern Authentication (also known as ADAL authentication) uses the Active Directory Authentication Library (ADAL)
to enable improved authentication for a variety of clients, including Office 2013 (with the latest updates), Office 2016,
Office for iOS and Android, the Outlook mobile application, and the Teams and Skype for Business clients. Modern
Authentication is based on the use of OAuth 2.0, an authentication protocol first used by Microsoft for server-to-server
authentication for Exchange and Lync integration but quickly adopted for client authentication as well. Modern
Authentication allows enabled clients to take advantage of the following:

SAML-based sign-in with third-party multi-factor authentication providers for Office 365.
Smart card-based authentication.
True multi-factor authentication.
SSO across apps through token sharing (for example, the Office 2016 desktop apps).
Figure 3-1: Modern authentication flow

Authentication Flows
As briefly discussed earlier, the biggest difference between Modern Authentication and legacy authentication methods
lies in how the authentication is performed and how communications flows between the client and Office 365. Figure
3-1 depicts the authentication flow of a client with modern authentication enabled:

1. The Outlook desktop client connects to Exchange Online and is redirected to Azure AD for authentication.
2. The client connects to the Azure AD authentication endpoint and is prompted for credentials. Because modern
authentication is enabled, the client displays the forms-based modern authentication credential dialog.
3. After the user submits their credentials, Azure AD first verifies the identity type of the user. It does so by looking
at the domain portion of the User Principal Name and verifying what type it is.

1. If the user is federated, the client is redirected to the on-premises identity solution using an HTTP 302 redirect.
Depending on what client the user is connecting with and how the endpoint is configured, the user might need to
enter additional credentials. By default, the client automatically passes the logged in credentials of the user to
the endpoint. If those credentials do not work, the user is prompted for credentials by the federation server.
2. If the user's password is managed in Azure AD (cloud-only identity, or using password hash synchronization), the
username/password combination is verified by Azure AD itself.
3. If pass-through authentication is enabled, the user's request is passed on to the on-premises Active Directory
through dedicated Azure App Proxies (Azure AD Connect).

4. When the credentials are successfully validated, one of the following happens:

1. the application receives a SAML token (from the AD FS server) and is then redirected to Azure AD where the
user receives a JSON web token (access and refresh token) instead.
2. if multi-factor authentication is enabled, the user is first requested to enter additional verification after which (if
successfully validated) the tokens are created and returned.

5. The client is now redirected to the service it initially connected with (for example, Exchange Online). Using the
access token previously received from Azure AD, the client can now access the online resource.

You’ll notice that the description above refers to two tokens: an access token and a refresh token. The access token is
short-lived, meaning it has a limited lifespan; 1 hour by default, and is used to authenticate to Office 365 (Azure AD).
The refresh token, which can remain valid indefinitely, is used to request new access tokens when they expire. With a
valid refresh token, the user does not have to re-authenticate. Instead, the refresh token is used to obtain a new
access token from Azure AD.

The flow described above assumes the client has no valid refresh token, which would be the case when the refresh
token expired or when the client has not yet connected to Office 365 before.

The validity of the refresh token is controlled by several things, such as how often it is used or whether the user
changes his password or not. When the refresh token expires, the user must sign in again. Depending on the client,
and how authentication is implemented, this might require a user to provide their credentials again. If true SSO is
used, re-authentication should happen automatically and is fully transparent for the end user. If no SSO is configured,
the user will be presented with an authentication form.


Note: Not every application on every (mobile) platform currently supports Modern Authentication. For instance,
Office (and thus Outlook) on Windows Phone or Windows 10 Mobile does not support Modern Authentication while
Outlook on iOS or Android does. Built-in ActiveSync clients, however, do not (yet?) support Modern Authentication.

Revoking access/tokens
As mentioned in the previous sections, authentication in Azure AD uses two types of tokens: access and refresh
tokens. The access token ultimately grants you access to the online service: with a valid access token, you can access
the requested service and perform an action like opening a mailbox. You can continue to work with the service until
the access token expires or you interrupt or close the active session (like closing the browser window). When an
access token expires, the client can acquire a new access token using a valid previously-received refresh token. If a
valid refresh token exists, the user does not have to re-authenticate.

Because of how these tokens are used, organizations sometimes face a new challenge: what if you need to revoke
someone's access immediately? The same problem also exists in an on-premises organization to a certain extent: when
you disable an account in Active Directory, subsequent authentications are no longer allowed, but the user is not
forced to close any open session/applications being used at that time.

In Office 365, the complexity lies elsewhere: access tokens cannot be revoked. This means that once someone gets a
valid access token, they can keep using that token for up to its maximum lifetime (by default 1 hour). You can,
however, revoke the refresh tokens which prevents the user from acquiring a new access token when it expires, and
thus force the user to re-authenticate (which will fail, if you disable the account). To revoke valid refresh tokens for a
user, run the following command from an Azure AD PowerShell instance:

[PS] C:\> Get-AzureADUser -SearchString Tony | Revoke-AzureADAllRefreshToken

Although not ideal, this will at least prevent the user from continuing to obtain valid access tokens. Remember that a
refresh token is, by default, valid indefinitely! Some organization find that the default access token lifetime of 1 hour
is too long. If required, the access token lifetime can be reduced to a minimum of 10 minutes. For more information on
how to configure custom token lifetimes, read this article . If you don’t read it, be aware that Microsoft is deprecating
the use of this method for terminating a user’s access, and that the existing mechanism doesn’t work with SharePoint
Online

Modern Authentication in a hybrid environment

As of Exchange Server 2013 CU19 and Exchange Server 2016 CU8, Modern Authentication is also supported for on-
premises Exchange servers, provided that a full hybrid configuration has been set up. This is good news, as it allows
you to align the authentication capabilities (and experience) cross-premises.

The way Modern Authentication works in an on-premises organization is very similar to how it does in Azure AD and
Office 365. However, instead of being a native capability, the on-premises Exchange Servers will pass authentication
requests to Azure AD and use the OAuth tokens Azure AD returns instead of using the native Exchange on-premises
mechanisms. In effect, when a user attempts to connect to Exchange, they will be redirected to Azure AD to
authenticate there. Depending on the configuration of Azure AD, the user will authenticate using one of the available
options like password hash synchronization, pass-through authentication, or even an on-premises redirection to an AD
FS server farm. Once a user is successfully authenticated, they will receive access and refresh tokens from Azure AD
which can then be used to authenticate to Exchange on-premises.

Note that enabling Modern Authentication for Exchange on-premises is an all-or-nothing scenario; you cannot enable
it on a per-user basis.

Multi-Factor Authentication
Because of how people use them, passwords are inherently insecure. Some years ago, it would take a computer a very
long time to crack or guess a password. As such, even the simplest passwords were secure enough to protect against
most forms of attacks. However, as hardware became more powerful, the time required for a computer to crack a
password decreased exponentially. Today, with the right hardware, a simple six-character password can be cracked in
a matter of minutes! To make it harder for passwords to be brute-force guessed, or at least to increase the time it
takes, we are taught to use longer and more complex passwords. Despite the clear benefit of longer passwords, it is
surprising to learn that the maximum password length in Azure Active Directory, and thus Office 365, is only 16
characters. While 16 characters is probably longer than most passwords, it doesn’t offer much additional resistance to
advanced attacks.

When you think about it, all authentication can be based on a combination of three things: something you know (such
as a pass phrase), something you have (like a physical token), or something you are (a biometric marker of some
kind). Traditional password-based authentication combines your user name with something you know-- but anyone
else who knows the same thing can pretend to be you.

To explain why you should use multi-factor authentication to secure access to user accounts, let's first take a
somewhat simplistic look at how typical authentication against Office 365 is performed. Figure 3-2 illustrates the
steps involved when a user tries to authenticate with Office 365 using a client like OWA. For the sake of simplicity, the
finer details are omitted, as we only want to understand the principles behind standard authentication.
 1. The user connects to the resource. For example, this might be OWA or SharePoint Online.
2. Because the client is unauthenticated, the Office 365 authentication service (in this case Azure Active Directory)
will send the client a “challenge”; the user is asked to provide his credentials.
3. The user (client) enters his credentials--a user name and password-- and sends them to the authentication service
which, in turn, will verify the credentials. If the user is a federated user, the authentication service will either
redirect the user to the AD FS endpoint or verify the credentials for the user with the AD FS infrastructure of the
user’s organization.
4. If the credentials are verified, the resource is “unlocked” and the user is granted access.

Figure 3-2: Traditional authentication flow

The last step is not entirely correct. Upon successful authentication, the client will get a logon token which it then can
use to request access to a specific resource (like OWA). This is the difference between authentication (getting a token)
and authorization (being allowed to access a resource). But for the sake of brevity, let’s just assume the user is
granted access to the resource after the authentication happened.

In this scenario, the password traveled “across the wire” which means that it potentially can be intercepted. Although
doable, this is not as easy as it sounds because the password does not travel over the internet in plain text. The
connection between the client, in this case Office 365, is secured using SSL. Often, the password itself is also hashed
within the SSL-protected connection, adding another layer of security. So, intercepting a password isn’t the biggest
risk.

An attacker who manages to steal a password, either by brute force guessing or through social engineering, might be
able to leverage the stolen password to access multiple sites. Users who pick weak passwords, write passwords down,
store them insecurely, or give them up to social engineering attempts aren’t necessarily to blame; although many
credential thefts are the result of user carelessness, it’s difficult to blame non-technical users for making these
mistakes when an average user must keep track of multiple credentials, each with its own length and strength
requirements. Although password management tools can help with remembering passwords, those tools are often
protected by a single password themselves.

Since knowing a user’s password allows an attacker to perfectly impersonate the user, poor password management
means that users are vulnerable to impersonation. To help fix this, we have several options. One is to make it harder
to guess or crack passwords; another is to harden systems, and train users, to reduce the chances that an attacker
can steal the password. A third option is to require the user to provide more than one type of authentication factor,
which is where the “multi-factor” part of MFA comes in.

The principle behind multi-factor authentication is that you add a second (or even third) factor from the “something
you have” or “something you are” categories. Typically, MFA’s implemented as a combination of a password with a
smartcard, one-time generated code, or biometric information such as a fingerprint, facial recognition, or an iris scan.
To gain access to a system, an attacker would have to have access to both the additional token and the user’s
password. Although adding an additional factor to the authentication is a huge leap forward in terms of security, it
does not take away all risks that entail the use of passwords.

Office 365 Multi-Factor Authentication

Office 365 includes simple multi-factor authentication (MFA) capabilities through a feature called Office 365 MFA.
Office 365 MFA is included with most Office 365 licenses and plans, except for Small Business and Dedicated plans.
Apart from Office 365 MFA, other solutions can provide multi-factor authentication for Office 365, including:

Azure MFA in the cloud.

Azure MFA on-premises.
Third-party (cloud) solution.

Office 365 MFA leverages the Azure MFA platform. Although Office 365 MFA and Azure MFA are very similar in terms
of functionality, the latter requires an additional license in the form of a standalone Azure MFA license, or as part of a
license bundle such as Azure Active Directory Premium or the Enterprise Mobility Suite. The biggest difference
between Azure MFA and Office 365 MFA is that the latter can only be used to secure access to workloads in Office
365, including:

Exchange Online.
SharePoint Online.
Skype for Business Online.
 Dynamics CRM Online.
Teams.
Office Pro Plus.
Project Online.

Azure MFA extends beyond these workloads and can also be used to protect other cloud and on-premises resources.
In addition, Azure MFA (in combination with an Azure Active Directory Premium license) includes reporting
capabilities allowing you to track suspicious activity, password resets, etc. Both Azure MFA and Office 365 support the
following additional authentication options:

Text Message (SMS). A one-time-code (6 digits) is sent to the user’s mobile device. This code must be entered
before the sign-in process can continue.
Phone call. The user receives a phone call, asking them to confirm that they are signing in by pressing the # key.
After confirming, the user is logged in automatically; no code is necessary.
Mobile App ( Azure Authenticator , available for Android and iOS)

Notification: the user receives a notification in the mobile app to ask them to confirm their identity by clicking
the Verify button. Once the user confirms, they are logged in automatically.
Verification Code: the mobile app generates a new code every 30 seconds. After the user enters the current code
(on screen), the sign-in process continues.
Authenticator app sign-in: Also known as “password-less sign-in”, in this mode the standard Microsoft sign-in
dialog will display a two-digit number and generate a request to the Authenticator app asking you to tap the
matching number on screen. This method lets users log in using their phone with no typing or password entry,
but it adds an extra step for users using AD FS. See the documentation for more details on how to set this up.

Additional third-party authentication methods , including those from RSA Data Security and Duo Security

In September 2017, Microsoft noted that only 0.73% of accounts with administrative access to Office 365 tenants had
MFA protection. Although some excuse can be offered for not using MFA to protect accounts used to run PowerShell
against Office 365 endpoints, that excuse is less valid now that all the key PowerShell modules support MFA. You
should secure all Office 365 administrative accounts with MFA. Microsoft is making it easier to do so by introducing a
baseline policy (currently in preview) that requires users with an admin role (currently global administrator,
SharePoint or Exchange administrator, conditional access administrator, or security administrator) to have MFA
enabled. Like other baseline policies in Azure AD, this policy is created automatically by Microsoft but must be
enabled if you want to use it. Interestingly, the current implementation allows you to specify that you want Microsoft
to activate the policy once it is out of preview and available in your tenant, or you can turn it on in preview mode.
Baseline policies are part of the Azure AD conditional access feature set, but don’t require the same Azure AD
premium license that the other conditional access features require.

To enable the baseline policy, you’ll need to go to the Azure Active Directory section of the Azure portal. In the
Security section of the left navigation rail, select Conditional access and you’ll see the list of existing conditional
access policies in your tenant. The current baseline policy is called “Baseline policy: Require MFA for admins
(Preview).” To enable it, just click the policy and chose the Use policy immediately option. That will immediately
force the use of MFA for any account that has one of the supported admin roles (global, SharePoint, Exchange,
conditional access, and security administrators are included). The policy will take effect immediately, and overrides
the MFA setting you may have applied to the accounts individually.

Azure Multi-Factor Authentication Server

Office 365 MFA and Azure MFA are the simplest ways to enjoy the benefits of multi-factor authentication without
having to worry about deploying or managing any added infrastructure. However, some situations may require you to
use an on-premises authentication server. This server is an application that is installed in the on-premises
environment and acts as a service broker between the on-premises environment and Azure Active Directory. It allows
you to leverage Azure's MFA platform and integrate it with your on-premises applications or AD FS server farm. The
integration with on-premises AD FS happens through the so-called MFA Adapter, a small piece of code that installs on
AD FS servers. It adds an additional authentication provider for multi-factor authentication. In turn, this
authentication provider is linked to the on-premises Authentication Server, which may, or may not be installed on the
federation server.

The features offered by Azure MFA in the cloud and through the Authentication Server are similar, with the main
exception being that the Authentication Server allows for additional scenarios and authentication options, such as
third-party hardware tokens, two-way communication via SMS, or PIN mode, which requires entering an additional
PIN before the identity can be verified. Another important difference is that Azure MFA allows you to preset the user’s
list of available authentication methods. Before the Authentication Server is ready to service MFA requests for Office
365, you must first complete a few configuration steps. These steps include:

Installing the Authentication Server.

Registering the Authentication Server with Azure AD.
Integrate the Azure Authentication Server with an on-premises federation server farm (AD FS). This requires
installing and configuring the Azure MFA Adapter on the federation servers.
Import identities from or synchronize with the on-premises Active Directory to enroll (enable) users for Multi-
Factor Authentication.

Although the Authentication Server can also be used to secure access to Office 365 (by integrating with an on-
premises federation server farm), most organizations choose to use the built-in, cloud-based Office 365 MFA
capabilities. As such, we will focus on the latter for the rest of this chapter.

How Office 365 MFA Works

When a user is enabled for MFA, the authentication process is slightly changed to accommodate the need to prompt
for a second authentication factor. Figure 3-3 illustrates the authentication flow for a user connecting to Office 365 for
a client such as OWA:

Figure 3-3: Office 365 MFA Authentication Flow

1. A user tries to sign in to Office 365 with an MFA-enabled account.

2. Either the user is redirected to Azure AD to provide credentials, or credentials are passed automatically onto
Azure AD.
3. If the user is a federated identity, Azure AD redirects the user to the organization’s federation endpoint to
validate the credentials.
4. If the credentials are successfully validated, the federation server will craft an authentication token and pass it
on to Azure AD. In this scenario, the user is a federated identity. If the user has a cloud account, steps 3-4 are
omitted and the primary authentication is completed by Azure AD.
5. Because the user’s account is enabled for MFA, Azure AD verifies the set of claims returned by the federation
server to verify that MFA has not already been done on-premises. Provided that no MFA has been performed,
Azure AD now invokes the Azure MFA service.
6. Azure MFA looks up the user’s information and challenges the user for the configured additional authentication
method. During the challenge period, the user will see an on-screen message telling them that the challenge has
been sent. For this example, let’s say the user receives a notification on his smartphone.
7. The user confirms the authentication request on his phone. This confirms his identity to the Azure MFA service. If
Azure MFA doesn’t receive a response within a timeout period (currently 60 seconds), the user can try to send
the authentication request again or switch to a different method (e.g. SMS instead of an app notification).
8. Once the user confirms the authentication request, Azure MFA confirms that the additional authentication was
successful.
9. The server applies any claims rules (described later) that may influence whether the use gets a token
10. Because both the initial authentication and secondary authentication completed successfully, Azure AD now
returns an access token to the user. The user application can then present the token to get access to the
requested resource.

Note: MFA does not require you to enter your credentials more often. The process only intervenes during the initial
authentication process by asking for an additional verification step.

Understanding app passwords

Not every client application supports the flow described above. As previously mentioned, clients that don’t use
Modern Authentication can’t use MFA. Since Modern Authentication shipped in 2016, Microsoft has steadily been
adding more support for it in its own applications, but not every Microsoft application fully supports it yet, much less
third-party applications. In such cases, either the application won’t be able to authenticate when MFA is required, or
some parts of the application may not work because those individual components cannot authenticate. For example,
the Skype for Business client for Windows can use Modern Authentication when communicating with the Skype
server, but can’t use Modern Authentication when making an EWS connection to get Exchange calendar data, so you’ll
see a persistent infobar telling you that you must enter your Exchange credentials.

Office 365 and Azure MFA provide a solution for this: app passwords are generated by the MFA subsystem upon user
request. These work in the same way as traditional passwords but aren’t related to the user’s existing password.
Applications that don’t support MFA can be configured with an app password instead of using the user’s true
password. App passwords don’t expire and cannot be recovered once created. When an app password is used, no
second verification option needs to be provided, which defeats the entire purpose of using multi-factor authentication
in the first place. On the other hand, an app password is the only way for a user to connect to Office 365 with non-
MFA capable devices or clients.

Unless the administrator has disabled the option, users can create their own additional app passwords through the
portal (as described later in the chapter). All app passwords are automatically generated (a user cannot choose his
own password). Paradoxically, app passwords are nearly impossible for a user to recall. So, if a client cannot
remember their app password, users are almost guaranteed to write down the password in order to remember them
for future use.

Enabling Accounts for Office 365 MFA

Enabling Office 365 accounts for multi-factor authentication is straightforward. The simplest way to do so is through
the Office 365 Admin Center. Login and navigate to Users and then Active Users . On the drop-down More menu,
select Multi-factor authentication setup. From the multi-factor authentication page, you can select users,
individually or in groups, and enable them for multi-factor authentication with a single click. Through the service
settings tab, an administrator can control additional options, such as:

Whether a user can generate app passwords.

What additional verification options are available. By default, all options are enabled (Mobile App
Verification/Notification, SMS, or Phone call).

If you have a license for the full version of Azure MFA, you get some very useful additional features, including the
ability to disable MFA when authentication requests originate from trusted IP addresses (so that MFA is only required
for users signing in from outside the corporate network) and for users to mark devices as trusted so they won’t be re-
prompted for authentication for a period of time.

An administrator can also use PowerShell module to configure MFA or to enable/disable MFA for users. In the
following example, we enable an account for MFA by running the Set-Msoluser cmdlet after setting the necessary
parameter values.

[PS] C:\> $MFA = New-Object –TypeName Microsoft.Online.Administration.StrongAuthenticationRequirement

$MFA.RelyingParty = "*"

$MFA.State = "Enabled"

$MFAOptions = @($MFA)

Set-MsolUser –UserPrincipalName Oisin.Johnston@Office365itpros.com -StrongAuthenticationRequirements $MFAOptions

It is also possible to pre-configure the authentication options for the user. In the following example, the same user is
enabled for MFA, and the preferred verification option is set to one-way SMS, meaning that the MFA service sends a
6-digit code via SMS to the user’s registered phone number (the mobile phone number defined for their account is the
default value). The user then inputs the code to complete the sign-in process. Other methods include
TwoWayVoiceMobile , where the user is called and asked to input a value on the phone keypad, and
PhoneAppNotification , where the user must respond to a notification sent to the Azure authenticator app running on
their mobile phone.

[PS] C:\> $MFA = New-Object –TypeName Microsoft.Online.Administration.StrongAuthenticationRequirement

$MFA.RelyingParty = "*"

$MFA.State = "Enabled"

$MFAOptions = @($MFA)

$MFAMethod = New-Object –TypeName Microsoft.Online.Administration.StrongAuthenticationMethod

$MFAMethod.IsDefault = $True

$MFAMethod.MethodType = "OneWaySMS"

$MFAMethods = @($MFAMethod)

Set-MsolUser –UserPrincipalName Oisin.Johnston@Office365itpros.com –StrongAuthenticationRequirements $MFAOptions –

StrongAuthenticationMethods $MFAMethods

Once a user is enabled, they must complete the MFA onboarding process by choosing an extra verification method the
next time they sign in (Figure 3-4 ). Depending on the selected verification method, the user may need to complete
some extra steps. For instance, if the mobile app is selected, the user must install the app on their smartphone and
scan a QR code to link the application to Azure AD, and by extension to the tenant. Once the added verification
method is configured, the user's enrollment is complete and the next time they sign into Office 365, they will use
multi-factor authentication.
Figure 3-4: Configuring the additional verification option(s)

Disable MFA for an Account

To disable MFA for an Office 365 account, we set the MFA options to null and write the value back to the account with
the Set-MsolUser cmdlet.

[PS] C:\> $MFAOptions = @()

Set-MsolUser -UserPrincipalName Oisin.Johnston@Office365itpros.com -StrongAuthenticationRequirements $MFAOptions

Listing MFA-Enabled Accounts

When managing MFA for multiple users, it is useful to know which users have already been enabled for MFA, users
have not yet completed the onboarding process, or the verification options configured. This information can easily be
retrieved using this PowerShell script to report MFA-enabled accounts. The information about the accounts is
reported in a CSV file.

[PS] C:\> $Report = @()

$i = 0

$Accounts = (Get-MsolUser -All | ? {$_.StrongAuthenticationMethods -ne $Null} | Sort DisplayName) ForEach ($Account in $Accounts) {

Write-Host "Processing" $Account.DisplayName

$i++

$Methods = $Account | Select -ExpandProperty StrongAuthenticationMethods $

MFA = $Account | Select -ExpandProperty StrongAuthenticationUserDetails

$State = $Account | Select -ExpandProperty StrongAuthenticationRequirements

$Methods | ForEach { If ($_.IsDefault -eq $True) {$Method = $_.MethodType}}

If ($State.State -ne $Null) {$MFAStatus = $State.State}

Else {$MFAStatus = "Disabled"}

$ReportLine = [PSCustomObject][Ordered]@{

User = $Account.DisplayName

UPN = $Account.UserPrincipalName

MFAMethod = $Method

MFAPhone = $MFA.PhoneNumber

MFAEmail = $MFA.Email

MFAStatus = $MFAStatus }

$Report += $ReportLine }

Write-Host $i "accounts are MFA-enabled"

$Report | Export-CSV -NoTypeInformation c:\temp\MFAUsers.CSV

The MFA status of the account can be:

Enforced : the user has completed the onboarding process and is (actively) using MFA.
Enabled : the user was enabled for MFA but has yet to configure the extra verification option(s).
Disabled : MFA is not enabled for the account or has been disabled in the past.

Tracking MFA Sign ins

Like any other sign-in to Azure Active Directory, all MFA-enabled sign-in appear in the Azure Active Directory sign-in
log . You can view the log interactively or download events to a CSV file and examine them afterwards in Excel or
Power BI. The MFA Required column in the CSV file tracks whether an account is MFA enabled or not, while the
status code captures the outcome of a sign-in attempt. Important codes are:

0 : Success.
50058 : An application tried to perform a silent sign-in (using an access token) to an MFA-enabled account but
couldn’t complete.
50076 : The user did not pass the MFA challenge because they did not respond in the allowed time.
50074 : The user did not pass the MFA challenge. For instance, they input an incorrect code. You can see the
authentication method used in the MFA Auth Method column.

Using MFA
Users sometimes need some time to get used to MFA, but since so many other cloud services and applications support
it (including Google’s applications, Facebook, Apple iCloud, and almost every online banking system), MFA quickly
becomes routine, needing minimal effort or thought. Personally, I like using the mobile app for authentication request.
It is very responsive and feels like the fastest way to complete the second verification step. Because it needs a
connection to the Internet, you might not always be able to use the app, for instance, when roaming in a foreign
country or in an airplane. In these cases, you can revert to another method, such as receiving a text message.

By default, the user's preferred additional verification option is used. If the user registered for multiple verification
options, they can select a different method when their preferred verification method is (temporarily) unavailable or
when the user doesn’t respond to a challenge within a period of time. For instance, when the mobile app is
unavailable because of a lack of mobile internet connectivity, a phone call or text message can be requested instead.
To register for added verification options, the users must access their Office 365 Settings (click the cogwheel from
an online app), select Security & Privacy , and then Additional security verification (Figure 3-5 ). From there,
they can add extra phone numbers, create app passwords, or change the preferred authentication method.
Figure 3-5: Changing MFA Verification Options for Office 365

Real world: MFA is a valuable security measure, but it is not a replacement for good password and account
management, and it has vulnerabilities of its own. For example, there have been a number of high-profile targeted
attacks where the attacker took control of the target’s mobile phone number by convincing the mobile carrier to
issue a new SIM so that the attacker received the MFA authentication code required to execute a password change
(here’s one example ). Even with MFA deployed and enforced, you should still be monitoring your users’ logon
activity for anomalies, and where possible, you should probably encourage the use of authentication apps or even
one-time MFA passwords over SMS MFA.

Pass-Through Authentication (PTA)

Pass-through authentication, if enabled, allows you to offload the authentication from Azure AD to your on-premises
Active Directory without the need to deploy AD FS or a third-party federation solution. As such, it can greatly simplify
your deployment if you are seeking to keep control of the actual authentication process.

For this to work, you must install and configure one (or more) on-premises connectors to communicate with Azure AD
to validate user authentication requests. The connectors can be installed as part of Azure AD Connect and are very
similar to Azure App Proxies. In fact, they share the same architecture; the PTA connector is a customized version of
the Azure App Proxy connector.

From a high-level perspective, here are the steps an authentication request goes through when PTA is enabled:

1. The client connects to an Office 365 service endpoint and is redirected to Azure AD for authentication.
2. The client connects to the Azure AD authentication endpoint and is prompted for credentials. If Modern
 Authentication is enabled, the client displays the forms-based Modern Authentication credential dialog.
3. After the user submits their credentials, and assuming PTA is enabled, Azure AD encrypts the credential data and
holds the authentication request in a virtual queue for validation.
4. The on-premises connector (which is part of the PTA deployment) makes an outbound connection to Azure AD
and retrieves the authentication request.
5. The connector decrypts the data from the request and validates the decrypted credentials against the on-
premises Active Directory. The result of this verification process is then communicated back to Azure AD.
6. If the credentials were verified successfully, a token is issued, or the multi-factor authentication flow is started.

In my testing, I found the feature to work well given its current limitations. For instance, you can only use pass-
through authentication with clients that support Modern Authentication, and there are several topologies where
Modern Authentication cannot be used to sign in to Skype for Business . In these scenarios, password hash
synchronization acts as a fallback .

Seamless SSO (Single-Sign On)

Microsoft have released a feature named "Seamless SSO" which aims to provide exactly what it describes, provided
that you’re using PHS and PTA. Given the term SSO also applies to other authentication options like federated
authentication, this name is a bit confusing.

Seamless SSO enables Azure AD to accept a Kerberos token from the on-premises Active Directory to authenticate the
user. At a high level, this is what happens when SSO is enabled:

1. The client connects to Office 365 and is redirected to Azure AD for authentication.
2. The client connects to the Azure AD authentication endpoint and is prompted for a Kerberos token.
3. The client turns to Active Directory and requests a Kerberos ticket for the computer account which is associated
with Azure AD.
4. Active Directory locates the computer account and encrypts the ticket using that account's computer secret.
5. The client sends back the ticket it received from Active Directory.
6. Azure AD decrypts the Kerberos ticket using the Kerberos decryption key which it received during setup of the
feature. If the ticket can successfully be decrypted, Azure AD will craft a token for the user (or the multi-factor
authentication flow is started when MFA is enabled).

For SSO to work, the following requirements must be met:

Modern Authentication must be enabled, and the client must support Modern Authentication.
The client must be domain-joined and able to directly communicate with an AD domain controller. This is
necessary for it to request a Kerberos ticket. Without direct access to a DC, SSO would fail and the process would
fall back to the regular authentication option (username/password). That means that seamless SSO can’t be used
with Macintoshes, mobile devices, or any other device that isn’t joined to an Active Directory-domain.
The Azure AD authentication endpoints must be added to the computer's browser intranet zone settings; by
default, browsers do not send Kerberos tickets to public endpoints.

Microsoft describes the SSO-feature to be "opportunistic". This means that SSO will be attempted, if enabled.
However, if anything goes wrong during the process, the authentication process will fall back to the default
authentication option.

Azure Conditional Access and Office 365

By default, all users with a valid license can access resources in Office 365; if they can reach the service endpoint and
log in, they can use the service normally. Sometimes though, an organization might want to limit who can access
resources, or perhaps control from which locations users can access those resources. Office 365 itself does not
provide such functionality. Instead, it leverages the advanced capabilities of Azure AD and the Enterprise Mobility
Suite.

Currently there are two types of conditional access policy: user-based and device-based. Although conditional access
policies are always assigned to a user, or group of users, the former option is a feature of Azure AD Premium and
refers to access rules that are based on attributes and characteristics of the user who’s trying to authenticate. Device-
based conditional access, on the other hand, are provided by Microsoft Intune and focuses more on specific device
characteristics (such as whether a device passes the Windows health attestation process). If needed, both can be
combined to achieve an even higher level of granularity.

There are numerous ways to restrict access to Office 365 resources using conditional access policies. You can restrict
who is allowed access to a resource, define which devices can be used to access resources, or control from what
locations an app or service can be used. You can also add restrictions based on characteristics of the user logon; for
example, if a user logs on from a unknown location, you can require them to authenticate with MFA even if the device
itself would normally be trusted.

Before diving into an example of a conditional access policy, it’s useful to know that each policy consists of one or
more conditions , to which one or more controls are linked as illustrated in Figure 3-6 below. Conditions may have
exceptions. Conditions you can apply to the policy include:

Users and Groups sets which users or groups the policy should apply to. You can choose to include or exclude
individual users or groups for the policy, or you can include or exclude all guest users or holders of specific Azure
AD directory roles.
 Cloud apps that define which applications the policy applies to. You can choose to enable the policy for all
applications or select the applications for which it should be valid. Applications include the various Office 365
workloads or supported third-party applications that you have configured to rely on Azure AD for authentication,
like Salesforce, Workday, Facebook, etc.
Conditions define characteristics of the authentication attempt which lead to a policy to be applied or not:

Device platform determines from which OS platform the authentication must be made, before an action is
considered. Options include: Windows (Phone), iOS and Android.
Locations specifies locations that will trigger the policy when the user logs in; locations are determined based
on the IP of the client making the request. You can define a set of known (trusted) corporate locations by IP
address range.
Client apps determine what application type (browser, ActiveSync, mobile client, etc.) should trigger the policy.
This condition set only applies to browser and Modern Authentication access; if you want to restrict access for
legacy applications or Exchange ActiveSync clients, you need to click the Advanced link in the Client apps
blade to get access to those conditions.
Device state (preview) lets you restrict access to devices that are managed through Intune .

On the controls side (which you access through the link labeled Grant ) you can choose from the following controls.
There’s also a radio button that allows to require any one of the actions selected or all of them:

Block Access : this is self-explanatory

Grant (Allow) access, with or without the following optional requirements:

Require multi-factor authentication.

Require device to be marked as compliant . For this option to work, the device must be enrolled to Intune so
that the device's health can be checked.
Require Hybrid Azure AD joined device verifies whether the computer is joined to the corporate Active
Directory domain (not Azure AD-domain joined).
Required approved client app lets you restrict whether or not applications must be accessed through an
approved (read “Intune-enabled”) client application instead of through a third-party application or a bare API.

Session : this control, currently in preview, lets you restrict what authenticated users can do within the context
of a specific application session. The controls available here are implemented by the applications themselves;
currently, SharePoint Online is the only application that honors this setting.

To illustrate the capabilities of the platform, consider the following scenario: an enterprise software company whose
name rhymes with “squad roe fleck” employs sales representatives, who travel often as part of their job. Sales reps
are allowed to bring their own devices, so there is a mix of iOS, Android and Windows devices in use. After a series of
unsuccessful phishing attacks, a new company policy was instituted, requiring that all external connections to Office
365 services from sales reps must use an additional authentication factor for access from outside the corporate
network. Here’s what’s required to implement the policy as a set of conditional access requirements:

We can break down the scenario into the following elements to create a new conditional access policy:

1. The policy should apply to all users who are part of the sales department. Because all sales representatives are
part of a group called "Sales", the policy can easily be applied to the latter.
2. Even though a conditional access policy can apply to many cloud applications (including non-Microsoft
applications), it must only be applied to the Office 365 workloads: Exchange Online, SharePoint Online, Skype for
Business Online, Office 365 Groups, and Microsoft Teams.
3. Only logon attempts from outside the corporate network should require an additional authentication factor.
4. Given that all sales representatives can choose their own devices, the policy should apply, regardless of the
device used to access resources.

To create a policy meeting these four requirements, login to the Azure AD Portal and navigate to Azure Active
Directory . From there, select Conditional Access .

Note: Before you can create a new policy, you should configure your known (trusted) locations. This can be done by
clicking the Named locations tab and defining the IP address ranges that belong to your corporate locations.
Without the information on known locations, the conditional access engine cannot determine if an authentication
request stems from inside or outside of the corporate network.

1. Click New policy and name the policy "Require additional auth for external sales" as illustrated in Figure 3-6 .
2. Under Assignments :

1. click Users and groups and select the Sales group. Don't forget to click Done to save your progress before
moving onto the Cloud apps blade.
2. Under the Cloud apps blade, select SharePoint Online, Exchange Online and Skype for Business Online, Outlook
Groups and Microsoft Teams.
3. Under Conditions , click Yes to enable this section. Then click Locations followed by Exclude. There, select All
trusted IPs .
 3. Under Access Controls ,

1. Click Grant , and then select Require multi-factor authentication and Require one of the selected
controls .

4. Under Enable policy , click On

Now, if someone from the sales department tries to authenticate to Exchange Online, no matter whether they access it
through the Outlook mobile app, desktop Outlook, OWA, or an Exchange ActiveSync client, they will be prompted for
additional authentication factors if the authentication is attempted from outside the known corporate locations.

Figure 3-6: Create a new Conditional Access Policy

Real World. It is good practice to enable multi-factor authentication for your users before creating a conditional
access policy that uses the Require multi-factor authentication option. That way, users can familiarize themselves
with the login behavior and you can ensure that they are properly enrolled for multi-factor authentication prior to
enabling this policy. If a user is not enabled for MFA, and such a policy is enabled, they would not be able to logon to
the in-scope applications until the registration is complete.

The above example illustrates one of many possible scenarios in which conditional access can be very useful. For more
information on conditional access, have a look here .

Restricting Access to a Single Tenant

Throughout this chapter, we discuss several ways of how you can restrict user access to Office 365 using a wide
variety of built-in features like custom claim rules or conditional access. However, none of these options address the
problem of a user authenticating against another Office 365 tenant. You might wonder why this is important or how
this is different from disallowing someone to authenticate to Office 365 by disabling their account. In the latter
scenario, the user is not able to login to the corporate Office 365 tenant, but that does not prevent them from
accessing other tenants, using another identity. From a data leakage standpoint, this is a potentially dangerous
situation as the user can logon to another tenant and copy corporate data to a repository on that tenant. The reason
why a user can login to various tenants is because most of the Office 365 endpoints (like login.microsoftonline.com)
are the same for all tenants, world-wide. Blocking the endpoint would not be a smart move, unless you want to block
access to Office 365 entirely. Here is an example to further illustrate the problem.

A user, Erica, works as a contractor for a government agency; she works at the agency but is an employee of a private
firm. She has accounts in two tenants: totallynotthefbi.onmicrosoft.com for the agency and
govtcontractor.onmicrosoft.com for her employer. When no tenant restrictions exist, Erica can login to either or both
tenants from her desk at the government agency, but there is nothing to prevent her from logging in into her
employer’s tenant while at work. To lower the risk of intentional or accidental data leakage, the security team at the
government agency wants to prevent Erica and her co-workers from logging in to their employer’s tenant from the
workplace.

As mentioned earlier, blocking the endpoint (for example. through a proxy server) would also prevent Erica from
signing into the agency’s own tenant, and thus prevent her from doing his job.

Of course, other ways exist to restrict access to Office 365. For instance, an administrator can define a set of internal
IP addresses from which a user can authenticate into the corporate tenant. Although this effectively prevents
someone from accessing the tenant outside the corporate network, it does not solve the problem described above.

To help organizations to control access to Office 36, Microsoft supports a feature known as “ tenant restriction .” The
way this feature works is quite simple. When a user authenticates with Azure AD, the authentication platform also
checks for a (HTTP) header called “Restrict-Access-To-Tenants.“ The value of this header holds the names of all the
tenants the user can access. You also need to add the “Restrict-Access-Context” header, which specifies the GUID of
your tenant, so the service knows which tenant to apply the restrictions to. If these headers hold the name of the
tenant the user is trying to access, the sign-in proceeds. If not, they receive a warning message.

For this feature to work, there must be (some sort of) a proxy server between the user and the internet. Basically,
anything that can add an HTTP header will suffice. If you want to make sure that control is always exerted over user
sign-ins, including outside the corporate network, users must always access the internet through a proxy server that
can inject the header. To ensure this happens, other counter-measures (such as ensuring the user cannot modify proxy
settings etc.) must be present. Otherwise, users can bypass the restriction themselves. In addition, the proxy server
should be accessible from outside the corporate network if you want to restrict access to a single tenant, regardless of
the user’s location. One way to achieve this it to use a “proxy-as-a-service” such as ZScaler.

The tenant restriction capability is not something you configure in Azure. Instead, it relies solely on your network
infrastructure to inject the header. Currently, the feature also only works with Modern Authentication, which is fine
for most of Microsoft’s own applications, but if you use other applications or built-in ActiveSync apps, you must block
access for those “legacy” apps if you want to maintain the restriction. You only need to apply the restriction to the
authentication process (only for the Azure AD endpoints). There is no need to force users to connect through a proxy
server for access applications such as Exchange Online.

Given the dependency on Modern Authentication, and because you cannot control how other tenants operate, if those
tenants still allow legacy authentication (basic authentication), users can bypass the restriction. Thus, the feature in
its current form only really is useful to control the potential for data loss if you fully control the endpoint, and thus
prevent the use of applications which do not use Modern Authentication. With that said, it is a fairly clunky way to
prevent data loss; it is much less flexible or capable than Azure Information Protection, although (unlike AIP) it
requires minimal configuration on the client side.

You can use the feature for free with Office 365, but you must buy an Azure AD Premium license if you want to restrict
access to other applications that rely on Azure AD for authentication.

Restricting access to a single tenant might or might not be useful. If preventing data leakage is a priority, anything
you can do to make it harder for people to share information in an unauthorized manner can be helpful. Although the
feature will most likely not prevent malicious users from leaking data, it can stop most regular users from
(accidentally) leaking sensitive data to another tenant. Of course, it is only a small cog in a much bigger picture as
many other Office 365 features like DLP policies (Chapter 22) and Azure Information Protection (Chapter 24) help to
safeguard your data.

Synchronizing with Azure Active

Directory
To access Office 365 workloads, a user must have a cloud identity. It does not matter if this identity is an online-only,
synchronized, or federated identity. However, when you have an existing on-premises Active Directory environment
and are planning to migrate some or all your workloads to Office 365, you probably do not want to create new Office
365 accounts for existing users. Creating new accounts creates the situation of having two accounts to manage for
each user in your organization and introduces the risk of differences and errors in the details of the two accounts, as
well as creating an unnecessary administrative burden.

Office 365 lets you use your existing on-premises Active Directory with Azure Active Directory through the process
known as directory synchronization (dirsync); in this process, some of the details and attributes of on-premises users,
groups, contacts, and other types of objects are synchronized from the on-premises Active Directory to Azure Active
Directory.

Directory synchronization can include the synchronization of password hashes (salted hashes ) to give end users a
“same sign-on” experience in which they use the same username and password to logon to both on-premises and
online services without storing the actual passwords in Office 365. Directory synchronization can also be needed for
certain features. For instance, if you want to use Active Directory Federation Services, or you are planning to
configure a hybrid Exchange connection, you must configure directory synchronization.

 Real World: Directory synchronization underpins the Staged and Hybrid migration scenarios for Office 365. When a
cutover migration is performed, directory synchronization cannot be used before or during the migration. However,
synchronization can be implemented once the migration is complete. In situations where third-party migration tools
are used, it is critical that the documentation provided by the vendor is carefully reviewed before any migration
activity commences. In many cases, the Microsoft directory synchronization tools can’t be implemented before or
when a third-party migration tool is in use, as the vendor might provide their own synchronization tools.

Because of the critical role directory synchronization plays in your ongoing identity management strategy, several
decisions must be made before you implement directory synchronization. These decisions include:

Are you synchronizing objects from a single Active Directory forest or from multiple forests?
What kind of authentication will you use? Should you use password hash synchronization or Active Directory
Federation Services (AD FS), or neither?
Should you synchronize every object from Active Directory or only a subset of objects?
Do you need to use a dedicated SQL instance or is the built-in SQL Express instance sufficient for your
environment?
Should the directory synchronization program be installed on an existing server, a dedicated server, or even in
Microsoft Azure?

All this leads to some potential for confusion when the time comes to implement directory synchronization for your
Office 365 deployment. We will now take a deeper look at Azure Active Directory, how directory synchronization
works, and the role that directory synchronization plays in an Office 365 hybrid deployment.

Note: Although we focus here on directory synchronization in the context of Office 365, directory synchronization
can also be used to enable federation and single sign-on for a wide range of other software as a service (SaaS)
applications from Microsoft and other vendors .

Microsoft Directory Synchronization Tools

Several directory synchronization tools are available from Microsoft that can be used with Office 365:

Azure Active Directory Connect (Azure AD Connect or AAD Connect) : This tool replaces both the original
DirSync tool and Azure AD Sync (AADSync) and contains all the features that were previously available along
with new functionality. All future feature development is planned for Azure AD Connect, making it the
recommended synchronization tool to use to connect on-premises and Office 365 directories. It is also the tool
that is offered as a download when you configure directory synchronization from the Office 365 Portal.
Forefront Identity Manager (FIM) 2010 R2 : Although the use cases for FIM are decreasing with every
update of Azure AD Connect, you may find a few border line cases that requires a very specific configuration that
cannot be done through Azure AD Connect. However, unless FIM is a specific requirement, it should not be used
because it’s out of mainstream support. Azure AD Connect uses a cut-down version of FIM as an underlying
service and provides a much simpler deployment and management experience. FIM is also not capable of
performing password hash synchronization or writeback.
Microsoft Identity Manager (MIM) 2016 : MIM is the successor to FIM and was released in July 2015.
Currently, it is recommended to use Azure AD Connect over MIM as the synchronization engine in Azure AD
Connect is newer and has more features geared specifically towards synchronization with Azure AD.

Directory Synchronization Technical Concepts

When describing the inner workings of directory synchronization tools there are several concepts that play a key role
in the entire process.

Connectors : Previously referred to as Management Agents, Connectors are the modules within Azure AD
Connect that connect to data sources such as the on-premises Active Directory and Azure Active Directory.
Connector space : Think of connector spaces as a cache that sits between a connected data source and the
metaverse. Any addition, changes or deletions are stored in the connector space until the next synchronization
operation occurs. The connector space does not contain the actual synchronized object, but rather a shadow copy
with the subset of the object's attributes that are marked to be synchronized. Each connector has its own
connector space and defines what objects and attributes are stored within the connector space.
Metaverse : The metaverse is the consolidated view within Azure AD Connect of all the joined identities from the
various connector spaces. It combines the identity information for an object and is stored in a SQL database.
Attribute flow : This is the process that copies data from one connected data source to another.
Source anchor: The sourceAnchor is the attribute of the on-premises object used to link it to an object in Azure
AD. By default, the value of the object's objectGUID attribute is used. The value of the ObjectGUID attribute is
stored as a base64-encoded string in the ImmutableID property of the corresponding object in Azure AD. As of
version 1.1.524.0 of Azure AD Connect, the tool uses the msDs-ConsistencyGuid attribute instead of the read-only
objectGUID attribute, if you have not populated the msDS-ConsistencyGuid attribute for other purposes. When
Azure AD Connect uses the msDS-ConsistencyGuid , it takes the value of objectGUID and writes it back into
msDs-ConsistencyGuid in binary form. Doing this enables you to manually update the reference later; for
instance, when you need to link the object to another (already existing) cloud object, or if you have multiple on-
premises forests and need to re-link the account from another forest to the matching object in the cloud after a
 user moves from one on-premises forest to another. Azure AD Connect stores the value in its metaverse as the
sourceAnchor attribute. The chosen attribute must not change during the life of the object or it will cause
synchronization problems and all sorts of unwanted effects on the Office 365 services. Figure 3-7 illustrates the
attribute flow for sourceAnchor . The objectGUID attribute is suitable except when a cross-forest move of the on-
premises object is going to occur. This is because during a cross-forest move, the objectGUID will change. There
are other scenarios which can cause the objectGUID to change. If you expect that the objectGUID for the object
might change during its lifetime, you can specify another attribute such as ExtensionCustomAttribute1 to be the
sourceAnchor .

Figure 3-7: The flow for the sourceAnchor attribute

The basic concepts outlined here apply regardless of the version of the directory synchronization tool in use, and you
are likely to encounter them in any directory synchronization project.

Synchronization Interval
Azure AD Connect v1.1 and later synchronizes Azure Active Directory with the on-premises directory every 30
minutes (previously the interval was every three hours). The interval implies that a change made to an object is
possibly only reflected in Azure Active Directory after a minimum of 30 minutes up to three hours, depending on
which version of the Directory Synchronization tool you use. An administrator can control various aspects of the
synchronization schedule like the interval between synchronizations, the type of synchronization, or he can force a
synchronization. This topic is explained later in this chapter.

In some cases, the period between change and effect will be longer. For example, when you enable a cloud-based
archive for an on-premises mailbox, the archive will be unavailable until Directory Synchronization has run twice: one
cycle to transit the change from the cloud to on-premises, and one cycle to synchronize back, which means a longer
delay could exist between enabling the archive and it being accessible in client interfaces.

Note: The synchronization delay only applies to synchronized objects, such as user accounts. Password hashes,
which are synchronized if the Password Hash Synchronization feature is enabled, are synchronized within two
minutes.

Supported synchronization topologies

Best practice for Active Directory has long been to keep the design as simple as possible, reflected by the
recommendation to use a single Active Directory forest whenever possible. However, this has not always been
possible, and some organizations still use a legacy infrastructure that is more complex than necessary. For instance,
mergers and acquisitions might force you to maintain multiple on-premises Active Directory forests. Some of these
directories might only contain user accounts, and others might also have Exchange deployed. Perhaps you still must
maintain support for legacy applications or meet specific legal and regulatory requirements, each warranting a
separate directory. Thankfully, Microsoft supports synchronizing multiple AD forests into a single Office 365 tenant. In
earlier versions of the synchronization tool, synchronizing multiple on-premises directories to Azure AD was very
painful. Although Azure AD Connect greatly simplifies things, by allowing you to easily add multiple on-premises
domains to be synchronized with Azure AD through a step-through wizard, they still recommend that you stick with a
single synchronized forest whenever possible. Using multiple forests only increases the chance of synchronization
errors, and lots of other issues resulting from those errors. The synchronization scenarios described in the following
sections are supported by Microsoft .

Single forest, Single Azure AD tenant

This scenario is the simplest and most common synchronization scenario. A single on-premises directory is
synchronized with Azure AD through a single directory synchronization server. Except for a staging server (more on
this later), you cannot use multiple directory synchronization servers to synchronize with Azure AD. Even though you
could theoretically configure multiple synchronization server to each only synchronize a subset of objects by
configuring exclusions, the risk of creating synchronization issues is not worth the trouble, so Microsoft doesn’t
support it.

Multiple forests, single Azure AD tenant

In this scenario, multiple on-premises directories are synchronized with a single Azure AD tenant. You can
synchronize by connecting multiple domains through a single synchronization server, but it is not supported to
connect multiple synchronization servers to a single tenant.

Single forest, multiple Azure AD tenants

In this scenario, a single on-premises forest (with multiple domains) is synchronized to multiple Azure AD tenants. For
example, consider a large multinational company that has separate domains for each of the countries where it
operates and wants to use separate Office 365 tenants. Because a 1:1 relationship exists between a synchronization
server and an Azure AD tenant, a separate server must be used for each Azure AD tenant. Additionally, each
synchronization server must be configured to exclude objects that are synchronized by the other servers to ensure
that objects are represented through all Azure AD tenants only once.

In addition to being able to only synchronize a subset of users to each Azure AD tenant, this synchronization scenario
also has other limitations. For instance, you cannot use the same UPN across various Azure AD tenants, because a
domain can only be registered in a single tenant. As such, you must ensure that the synchronized objects all use a
different UPN suffix, depending on the Azure AD tenant they are synchronized to. This limitation also carries forward
into Office 365, more specifically with regards to mail flow.

Object Matching for Existing Objects

The ImmutableID property plays a key role in the directory synchronization process as it is the attribute that links an
on-premises object to an object in Azure Active Directory. When you run directory synchronization for the first time,
the sourceAnchor attribute is created automatically and its value is stamped on the ImmutableID property for the
cloud object. However, by the time you start dirsync, you may already have created cloud objects in Azure Active
Directory. As one example, this is often the case after a cutover migration.

To match existing on-premises accounts to pre-existing objects in Azure Active Directory, directory synchronization
uses two different approaches: hard matching and soft matching .

Hard-matching Existing Azure Active Directory Objects

Hard-matching is when you link two objects by manually setting the ImmutableID on the cloud object. This requires
you to calculate the correct sourceAnchor value. When dirsync runs, it will automatically detect the relation between
the objectGUID and the ImmutableID and confirm the link between both objects in its metaverse through the
sourceAnchor .

Generally, you won’t want to do this if you don’t have to, because it requires manual or scripted action; soft-matching
(explained below) is an easier and simpler way to tie on-premises and cloud objects together.

If you need to use hard-matching, you can use PowerShell to extract the ImmutableID from the ObjectGUID (or any
other attribute) and then verify if it matches the object in Azure AD. In the following PowerShell example, the
ObjectGUID for a user called "Bruce Lane" is retrieved and then base64-encoded:

[PS] C:\> $objectGUID = Get-ADUser "Bruce Lane" | select ObjectGUID

[PS] C:\> [System.Convert]::ToBase64String($objectGUID.ObjectGUID.ToByteArray())

28yuIZMiVUuT2GJj6+qPDg==

To save you some typing, you can use a PowerShell script to generate the ImmutableID for an Active Directory user
object. Next, connect with PowerShell to Azure Active Directory and update the ImmutableID for the cloud-based
object that you wish to link to the on-premises object:

PS C:\ Set-MsOlUser –UserPrincipalName Bruce.lane@Office365ITPros.com –ImmutableId 28yuIZMiVUuT2GJj6+qPDg==

After you have manually configured the ImmutableID , you run the directory synchronization process. Once the
process completes, Azure AD Connect will link both objects together.

Soft-matching Existing Azure Active Directory Objects

Azure AD will automatically try to match objects together by looking for objects whose primary SMTP address or mail
attribute match. This is known as soft-matching . For this to work, the following requirements must be met:

The existing account in Office 365 must have an email address. It does not require a mailbox, but the
PrimarySmtpAddress or mail attribute must be configured.
There cannot be more than one object with the same primary SMTP address in the directory. This will cause the
soft-matching process to fail as Azure AD is unable to determine with which object it should match.
The object to match in Azure AD must not have a value for the ImmutableID property.

If Azure AD cannot match an object based on the SMTP address, it can attempt to match accounts based on the UPN
instead. By default, however, this feature is disabled, and customers must run the following PowerShell command in
Azure AD PowerShell to enable it:
PS C:\> Set-MsolDirSyncFeature -Feature EnableSoftMatchOnUpn -Enable $true

In the scenario of a cutover migration, both the on-premises and Azure AD object should already have a matching
SMTP address. This means that, once the migration completes, you can run directory synchronization and it should be
able to soft-match all existing objects with little to no effort, if you do not have duplicate SMTP addresses. Duplicate
addresses may exist when you have contacts in the on-premises Active Directory with the same email address as a
mailbox in Office 365. In this situation, you will receive a directory synchronization error message via email telling
you that duplicate objects exist, and that Azure AD Connect was unable to create or update the object.

If you only have objects in Azure AD and wish to link on-premises objects to them, copy the value of the email address
from the object in Azure AD onto the on-premises object. If you have Exchange installed on-premises you can use the
Exchange Management tools to do so. If not, the Active Directory Users and Computers console can be used to update
the E-mail field.

Preparing for Directory Synchronization

Before implementing directory synchronization, there are some system requirements that you will need to meet, as
well as other preparation tasks to help make sure that the on-premises Active Directory environment is ready to be
synchronized to Azure Active Directory.

The Active Directory forest must run in Windows Server 2003 forest functional mode or higher. The domain
controllers must also be running Windows Server 2003 SP1 or higher. It does not matter whether the domain
controllers are 32-bit or 64-bit. However, given that Windows Server 2003 is now out of support, it is
recommended to use at least a Windows Server 2008 R2 domain controller (although it is also about to become
unsupported), or preferably a Windows Server 2012 R2 or Windows Server 2016 domain controller. However, it's
important to remember that the different versions of domain controllers do not add or remove functionality from
directory synchronization tools.
Azure AD Connect can be installed on a server running Windows Server 2008 or later. Although Windows Server
2008 is supported, it is recommended to use a server running Windows Server 2008 R2 or later, because the
older operating system does not support using password hash synchronization. In some scenarios, you might
want to install the directory synchronization component on an existing server. This can be any server in your
environment from a domain controller to a file server. You cannot install Azure AD Connect or other Microsoft
directory synchronization tools on Windows client operating systems. Although installing Azure AD Connect on a
domain controller is supported, it is generally not recommended for security and performance reasons.
Make sure that the account(s) that you will use to configure directory synchronization have the appropriate
permissions. What permissions are required, and how to configure them, is explained later.
Review the synchronized object limit. By default, every Office 365 tenant allows you to synchronize up to 50,000
items . As soon as you register a custom domain in the tenant that limit is automatically raised to 300,000.
However, if you have a very large on-premises environment and you plan on synchronizing more than 300,000
objects, you must open a service request with Office 365 Support to raise the tenant limit. Synchronized objects
are not only user accounts, but also contacts and groups. Amongst other things, Microsoft's Readiness Checks
will verify whether you have more than 300,000 objects in your on-premises Active Directory.
Decide whether you want to use sync filtering. You can define filter rules, as described below, to synchronize only
a subset of your on-prem objects to Azure AD. Because you can enable sync filtering at any time, it’s not critical
that you decide this during the planning process.
Review on-premises objects and make sure they do not contain illegal characters or duplicate values. It is
important that no objects with duplicate or conflicting proxyAddresses attributes are synchronized to Azure AD
as this could disrupt the service for the user. This is why, by default, directory synchronization tools will not
synchronize objects with conflicting attribute values or unsupported characters. The list of characters that are
supported varies from attribute to attribute. A full list of unsupported characters is available here . Microsoft
provides a variety of tools which can review and remediate potential deployment blockers. These tools, and how
to use them, are explained in the section “Review and remediate directory synchronization blockers” later in the
chapter.
If you plan to set up single sign-on (SSO), you must also make sure that the User Principal Name (UPN) suffix for
the on-premises user objects is an Internet-routable domain such as yourdomain.com , for example
john.doe@yourdomain.com . This is because the UserPrincipalName attribute is used to sign in into Office 365
and to identify federated identities. To synchronize the value from the on-premises environment you must have
previously registered the domain in the Office 365 tenant. Given that you can only register public domains in
your Office 365 tenant that you can verify that you own by adding DNS records as directed by Microsoft, the on-
premises value must match a domain that has been registered in your tenant.

Directory synchronization and duplicate object values

Previously the synchronization tools could not synchronize objects that have specific duplicate or conflicting attribute
values. This would, for instance, be the case when synchronizing a user object with an email address or UPN that is
already assigned to another user in Office 365. Today, Azure AD Connect can handle two common cases: duplicate
UPNs and duplicate proxy addresses. By default, instead of failing to create or update the object, Azure AD
quarantines the conflicting attribute value and continues to create or update the object. What action is taken depends
on the type of conflicting attribute:

If the conflicting attribute is required to create the object (for example, User Principal Name), Azure AD creates
the object, but uses a placeholder value to replace the conflicting attribute. For instance, if the conflicting UPN is
tony.redmond@office365itprobook.com , the replacement value could be
tony.redmond6743@office365itprobook.com . The addition of the random 4-digit number removes the conflict
 and allows the object to be synchronized or created. An administrator can then revisit the object and manually
resolve the original conflict later.
If the conflicting attribute is not required to create the object (for example a secondary email proxy address), the
object is created or updated without the conflicting value.

When the Duplicate Attribute Resiliency feature is disabled, the synchronization error report and all subsequent
reports include the current, but also all previous, still outstanding, synchronization errors. This is because despite
previous unsuccessful synchronization attempts, the synchronization continues to try and synchronize all objects until
the conflict is resolved and the action succeeds. When the Duplicate Attribute Resiliency feature is enabled, objects
may be created successfully despite possible conflicts and therefore the synchronization errors only show up in the
report once. Because it is possible that an administrator misses the report, the list of conflicting objects and attributes
is available through Azure AD PowerShell:

[PS] C:\> Get-MsolDirSyncProvisioningError -ErrorCategory PropertyConflict

In large environments, the above command can potentially return a lot of results. If needed, you can reduce the
number of results by using filtering parameters like PropertyName and PropertyValue . The following example only
shows synchronization errors caused by a conflicting proxy address. Note that more filtering parameter and options
are available , and can help to further refine results.

[PS] C:\> Get-MsolDirSyncProvisioningError -ErrorCateogry PropertyConflict -PropertyName ProxyAddresses

Depending on the synchronization error, and conflicting attribute, various strategies exist to resolve the issue. Some
suggestions to tackle different conflict scenarios is in this article.

Review and remediate directory synchronization blockers

Microsoft offers a range of health, readiness, and connectivity checks an organization can use to assess if it's ready to
deploy Office 365. To access the tool, navigate to http://portal.office.com/tools . The following tools are available
there:

Office 365 health, readiness, and connectivity checks

Exchange Server Best Practice Analyzer (beta)
Office 365 Connectivity Analyzer
Exchange Client Performance Analyzer

The advanced Office 365 health, readiness and connectivity checks give more information about the organization's
readiness for Directory Synchronization. When you run the tool, a small application is downloaded from the portal to
inspect the on-premises environment. Note that the tool might take a while to run, especially in larger environments.

To remediate possible issues, you can use the IdFix tool (Figure 3-8 ), to bulk-edit conflicting or unsupported objects.
Figure 3-8 illustrates two unsupported objects. The attribute field shows which attribute contains conflicting or
unsupported values. The value field shows the current value and the update field contains a suggested value to
remediate the issue.

Figure 3-8: The IdFix tool

The Action drop-down menu allows you to select what action you want to perform on the object. If you select Edit
and then click Apply in the top-level menu bar, the on-premises objects will be updated with the value in the Update
field. Note that the value in the Update field is only a suggestion. As an administrator, you can put in any value you
like. The thought of bulk-editing on-premises objects might sound scary at first. In a way, it is. However, the IdFix tool
will only display non-critical objects. Nonetheless, you should make sure that any change you make does not have an
adverse effect, for example to a line-of-business application that references Active Directory attributes.

Real World: The IdFix tool covers the most common synchronization errors. However, there are some conditions
that can break the directory synchronization process but that IdFix doesn’t check or fix. For instance, it does not
check whether different accounts in the environment have a matching UPN and proxyAddresses . Imagine there are
two users in your organization with a similar name: Jane Doe and John Doe. Jane Doe's UPN is jdoe@contoso.com and
John Doe's UPN is DoeJ@contoso.com. However, for some reason, one of Jane Doe's email addresses (in the
proxyAddresses attribute) is DoeJ@contoso.com. Although potentially confusing, this is a fully supported on-premises
scenario. However, because of the importance of the UPN in Office 365, the directory synchronization process
perceives these two objects as a conflict, because the UPN of one user is on the proxyAddresses list of the other. This
scenario might not be very common, but because it is not reported in either tools from Microsoft, it's also very hard
to detect upfront. Another scenario that is not covered by IdFix is checking whether a synchronized attribute value is
too long. For instance, if you have an extensionAttribute that holds 1,200 characters, it will not be reported by IdFix,
nor will it be synchronized to Office 365 because the value is too long for Azure AD.
Sizing the dirsync server
Assigning too little resources to a directory synchronization server will not prevent it from synchronizing to Office
365, but it could significantly slow down synchronization up until the point that synchronizations cannot complete
within a reasonable period. If password synchronization is enabled, an underperforming synchronization server might
also cause passwords not to be synchronized to Office 365 in time, or with a significant delay, thus impacting the end-
user experience.

Microsoft publishes a list of minimum requirements , which are a great starting point. The amount of resource
required almost entirely depend on the number of objects that are synchronized to Office 365. Because the number of
objects is likely to be in flux all the time, make sure to account for a little growth.

Directory synchronization and SQL databases

A Microsoft SQL database is required to hold the objects manipulated by the directory synchronization tool. By
default, Azure AD Connect will install and configure a SQL Express LocalDB, a lightweight version of SQL Express
that installs very quickly. The LocalDB installation is valid for up to 100,000 total synchronized objects (not just
100,000 user objects). This isn’t a hard limit, but there is a hard limit of 10GB on the SQL Express database, which is
where the 100,000-object limit comes from. To remain officially supported by Microsoft, if you have more than
100,000 objects, you’re required to use a full Microsoft SQL Server database. That doesn’t mean that, if you have
100,005 objects, dirsync will break. However, the database for a metaverse containing more than 100,000 objects is
very likely to grow beyond the 10 GB database size limit of SQL Express. In larger environments, the use of a full
Microsoft SQL Server also provides the option of clustering the SQL instance for high availability. If you specify a
separate SQL Server, the account that is used to run the wizard must have permissions to create a database
(dbcreator) on the SQL server.

Installing and managing Azure AD Connect

Azure AD Connect is much more than just a synchronization tool; it can also take care of the installation and
configuration of an AD FS server and a Web Application Proxy server, and it can implement pass-through
authentication if you’ve chosen to use it. In addition, every installation of Azure AD Connect also configures the Azure
AD Connect Health Agent which interacts with the Azure AD Connect Health service and provides an administrator
with valuable insights into the synchronization process and the health of the synchronization components.

When installing Azure AD Connect, you have two installation options: Express and Custom. The Express installation is
the most basic configuration, and only installs the synchronization components with default settings. The Custom
installation allows for a much more granular configuration, including defining your own service account, using a SQL
database, etc. Aside from the additional configuration parameters, the Custom installation can also install, and
configure AD FS and Web Applications Proxy servers for you.

Overview of Express Installation of Azure AD Connect

By default, the Azure AD Connect installation wizard suggest the Express Settings option, which installs and
configures the following options:

Configure the directory synchronization engine to synchronize Azure AD with a single local Active Directory
forest. By default, all attributes are synchronized.
Enable password hash synchronization.
Enable Auto Upgrade; the feature ensures the tool maintains itself and automatically upgrades to the latest
version when available.

When you perform an express installation, the wizard guides you through a few basic steps, which include specifying
admin credentials to your Office 365 tenant and the on-premises Active Directory.

The on-premises and Office 365 admin accounts that you specify in the wizard are not used to run the synchronization
tool. Instead, the wizard uses these accounts to create the necessary service accounts. First, Azure AD Connect will
create a new service account in your Office 365 tenant. The format of the service account looks like
Sync_ServerName_Guid@domain.com and the account is granted limited administrative permissions to allow it to be
able to write objects and attribute changes into Azure AD. Similarly, an on-premises account is created in the default
Users container in Active Directory. This account is typically named MSOL_guid , as shown in Figure 3-9 :
Figure 3-9: Azure AD Connect service account

When the account is created, it gains appropriate permissions in your environment to execute the tasks which are
configured as part of the configuration Wizard.

Real World: If permissions inheritance is blocked somewhere in the domain tree, you must manually set the
permissions on the organizational unit that has inheritance disabled. If inheritance is blocked again at lower levels,
you must repeat this step for each OU that does not automatically inherit permissions.

Overview of Custom Installation of Azure AD Connect

As an alternative to the Express Settings installation, you can customize the installation of Azure AD Connect during
setup by clicking the Customize button instead. The following installation options are available:

Specify a different installation location. By default, the Azure AD Connect synchronization tool is installed in
"C:\Program Files\Microsoft Azure AD Sync".
Define whether a separate SQL database should be used and, if so, what credentials to use to access it.
Specify an (existing) service account. Unlike the Express installation option, the account that you specify will be
used to run the synchronization service afterwards. The account does not need high-level permissions and can be
a standard user account; it only requires read access to Active Directory for most of its functionality. However, if
support for a hybrid deployment, password hash synchronization, or any of the writeback features are required,
additional permissions must be granted to the service account so that it can perform the necessary tasks such as
updating the hybrid writeback attributes or reading the password hash. More detail on how to configure these
permissions is provided in the section “Writeback permissions ” later in the chapter.
Specify custom synchronization groups. These Active Directory groups control what administrative actions
someone can perform in the directory synchronization tool. By default, Azure AD Connect creates local groups on
the server, not in Active Directory.

The Azure AD Connect installation wizard also gives you the option to automatically install and configure Active
Directory Federation Services. On the User sign-in page, you can select the following options:

Password Hash Synchronization . This will enable the secure synchronization of on-premises user password
hashes to Azure AD.
Pass-through authentication, which enables the ability to validate credentials in the on-premises organization
through the pass-through authentication feature.
Federation with AD FS , which then guides you to configure one or more AD FS and Web Application Proxy
servers and configures federation with the domain selected in the wizard.
Federation with Ping Federate , which then walks you through the process of configuring federation with an
already-installed Ping Federate server in lieu of AD FS.
Enable single sign-on, which will configure the ability the single sign-on features as described in the Seamless
SSO section.
Do not configure , which does not configure any authentication options.

Note: You can also modify the user sign-in options with the Express Installation option by running the Azure AD
Connect wizard again after the initial configuration and selecting the Change user sign-in option.

When you select to install and configure Federation with AD FS, you must have at least two Windows Server 2012 R2
or Windows Server 2016 servers. One is used as the AD FS server; the other is installed as a Web Application Proxy.
You should also prepare a public SSL certificate that you would like to use for AD FS.

The wizard can configure a new AD FS server farm or add servers to an existing farm. You will notice that the wizard
is very similar to the manual setup of AD FS servers. During the installation, you are asked to provide the SSL
certificate for service communications, and you must specify a service account which is used to run the AD FS service.

Note: To install the AD FS and Web Application Proxy servers remotely, the target servers must have Remote
Management enabled. Additionally, if the Web Application Proxy server(s) is an untrusted, non-domain joined
computer, you must add the target host to the list of trusted hosts on the machine that is used to run the
configuration wizard. For more information, review this Microsoft article .

In addition to the above options, the custom installation also enables you to configure the following capabilities:

Object Filtering . Object filtering enables you to select what objects are synchronized to Office 365. The
different filtering options are explained in Object Filtering .
Multi-Forest synchronization . You can add multiple on-premises domains to synchronize to Office 365. For
each domain, Azure AD Connect creates a new connector. You can also specify how objects are matched between
the various domains. This is particularly useful when accounts are represented in more than a single forest. This
would for example be the case in a resource forest scenario: Users are represented once in the accounts forest,
 and a second time with a disabled account in the resource forest. More detailed information about multi-forest
synchronizations is provided in the section “Multi-Forest Directory Synchronization” later in the chapter.

For a detailed description of how to enable Directory Synchronization and how to install and configure Azure AD
Connect, you can refer to Bonus guide .

Staging Mode
In the last step of the Azure AD Connect configuration process, you can configure directory synchronization in Staging
Mode . If enabled, staging mode will prevent the directory synchronization tool from exporting data to either Azure
AD or the on-premises Active Directory; all of the matching and data collection steps are still performed but nothing
will be written either to Azure AD or the on-premises AD. This can be particularly useful in two scenarios: when doing
a swing upgrade from one version of Azure AD Connect to another or when you want to have a second directory
synchronization server configured as a warm standby, ready to take over in case the primary server experiences an
extended outage.

One can argue if a real need exists for a highly-available directory synchronization setup. By default, directory
synchronization happens every 30 minutes. If a dirsync outage lasts less than that time span, there is little to no
noticeable impact. Who and what functionality is impacted depends on what features are enabled in the
synchronization tool. For instance, if you have password hash synchronization enabled, no password changes will be
synchronized to Office 365 during the outage. The longer the outage takes, the more calls the helpdesk will get about
passwords or other user attributes being out of sync.

If the synchronization server cannot be recovered, the simple solution is to deploy a new directory synchronization
server. The installation of the synchronization tool, not including the time to build a new server from scratch, only
takes a few minutes. However, depending on the size of your organization, the initial synchronization can take several
hours. The more objects that are synchronized, the longer it takes. Depending on the situation, the time to wait for the
initial synchronization to complete might no longer be a viable option.

Prior to the introduction of Azure AD Connect, there was no built-in way to handle this situation. Now, staging mode
allows you to have a second synchronization server running in parallel with the active instance. In case you need to
activate the second server, all you must do on the staging server is rerun the setup wizard to disable staging mode. At
the next synchronization, the second server will now start functioning as the “real” dirsync server, writing changes to
both Azure AD and the on-prem environment. Because the synchronization server has been running in parallel with
the active instance, but not previously actively syncing, it will pick up synchronization where the formerly active
server left off.

The only caveat to this approach is that you must make sure that no two synchronization servers are active at the
same time, as this can yield unexpected results. With exception of the sync scheduler configuration (which is stored in
Active Directory), the configuration of both synchronization servers must be kept identical, manually. This means that
any filtering rules that are applied to the primary synchronization server must also be applied to the second server to
keep them consistent.

The staging mode option is also used during a swing migration. It allows you to install a second server and verify that
it is configured correctly by monitoring the synchronizations before exporting changes to Azure AD or the on-premises
Active Directory. Once the second server is setup correctly, you can disable the legacy synchronization server and
disable staging mode.

Synchronization Rules
The AAD Connect Synchronization Rules Editor allows you to make changes to attributes as they are synchronized
from Active Directory to Azure Active Directory. There are several dozen pre-defined sync rules that, together,
implement the standard AAD Connect sync functionality. Much like the registry, you can modify these rules to change
sync behavior—but you have to be cautious when doing so.

Why would you want to make these changes anyway? For one example, many large organizations use the last name,
first name format for the user DisplayName attribute to order the Exchange GAL by the last name. It can be argued
that Office 365 takes a more user-friendly approach to how applications refer to users and the default is to create user
accounts with the DisplayName formed by the first name, a space, and the last name. You can accomplish this
transformation with a rule (see this practical example ).

Microsoft’s documentation also describes the process of modifying the sync rules to allow the Azure AD UserType
attribute to be synced—this is the flag AAD uses to determine whether an account is a member or guest account.
Since on-prem AD lacks an equivalent attribute, you have to create a custom sync rule that copies an on-prem AD
attribute (such as extensionAttribute4 ) to the UserType value. For all sync rule changes, the basic process is fairly
simple:

1. Stop scheduled synchronizations in AAD Connect.

2. Create or modify the rule
3. Preview the rule on a single object. You don’t have to do this, of course, but it’s an excellent idea. Fix any errors
you find and verify that the target object is modified in the way that you expect.
4. Run the rule against all objects. Fix any errors that you find and check a selection of objects to verify that the
rule did what you thought it would.
5. Re-enable the synchronization scheduler.
Object filtering
Except for some system accounts, the default mode of directory synchronization will process every user, group, and
contact in your on-premises Active Directory to the cloud. Although having the accounts in Azure Active Directory will
not cost you money (since no licenses are consumed), having unnecessary accounts in Azure Active Directory can
make administration more difficult by cluttering the various views in the administrative interfaces. It’s also possible
that you might want to minimize risk and speed up directory synchronization by only synchronizing the minimum set
of objects you need to Azure Active Directory. This leads to the requirement to control which objects are synchronized
and which objects are excluded. Filtering can be used for this purpose.

Four methods can be used to apply filtering to your directory synchronization:

Domain-based – in a multi-domain forest you can select which domains should be synchronized with Azure
Active Directory. This is common in scenarios where all the users, contacts, and groups are in a single “account”
domain within the forest. In this situation, it is more efficient to only synchronize this domain, and ignore other
resource domains.
OU-based – this allows you to specify which OUs contain objects that will be synchronized to Azure Active
Directory. This method of filtering is a straightforward way to enable a slow, phased approach to directory
synchronization, in which objects are moved in small batches into an OU that is in scope for directory
synchronization.
Attribute-based – this approach allows you to specify which object attributes (for example, the department
attribute for users), should be used to determine which objects synchronize to Azure Active Directory. Attribute-
based filtering can be applied using either inbound or outbound synchronization rules. Inbound rules are
generally recommended as these will be preserved in an upgrade of Azure AD Connect, while outbound rules will
be overwritten during upgrades.
Group-based – this allows you to specify a group in the local Active Directory. Only member objects of this group
are synchronized to Azure AD. This option is only available when performing a custom installation of Azure AD
Connect. Note that you can only configure group-based whilst first running the setup wizard. Once you’ve
completed the setup, you cannot reconfigure group-based filtering using the wizard anymore.

Detailed steps for applying each type of filtering in Azure AD Connect are available on Microsoft's website . The
filtering options covered here are only available if you choose a custom installation. If you implement filtering,
understand that you are potentially changing the way users in Office 365 interact with on-premises recipients. This
could be the case in a hybrid deployment where you decide to only synchronize user objects that have a mailbox in
Office 365. The users in Office 365 may not see most or all the on-premises mailboxes in the Global Address List.

Real World: If filtering is enabled, you must tread carefully with moving on-premises objects in Active Directory. For
instance, if you excluded an organizational unit from synchronizing to Office 365, every object moved into that OU
will not be synchronized to Office 365. If an object which was previously synchronized with Office 365 is moved to
that OU, it will automatically be deleted in Office 365 at the next synchronization interval.

Password hash synchronization

When directory synchronization is implemented with password hash synchronization, users can logon to Office 365
services using the User Principal Name (UPN) and password of their on-premises Active Directory user account. This
is referred to as “same sign-on”, which is not to be confused with “single sign-on” even though they both can be
abbreviated to SSO.

The concept of synchronizing passwords tends to raise immediate concerns within an organization, as people
inevitably assume that real passwords are being transmitted over the Internet and stored on Microsoft servers. The
reality is that password hash synchronization is a secure process, and the passwords themselves are not transmitted
or stored. Instead, a password hash is used. Despite this, some organizations may still object to the use of password
hash synchronization, so it’s important to understand what is being synchronized.

On-premises Active Directory stores passwords as hashed values that are said to be irreversible . In other words, a
password hash can’t be used to determine a user’s plain text password. The directory synchronization tool extracts
the password hash from Active Directory, hashes it a second time, and then transmits the hash over a secure HTTPS
channel to Azure Active Directory.

When password hash synchronization is enabled, the password complexity and expiration policies of on-premises
Active Directory override the policies set in Office 365. When a password is changed on-premises the new password is
synced to Azure Active Directory. This process usually occurs in under two minutes. Existing logged on sessions for
Office 365 services are not interrupted by a password change on-premises. The new password only takes effect for
any subsequent logon attempts.

If an on-premises password expires, the expired password will continue to work in Office 365. This is because when
password sync is enabled, account passwords in Office 365 are set to not expire. For this reason, you should not rely
on expiring or changed passwords to prevent a user from logging on to Office 365. Instead you should disable the on-
premises user account and either force a directory synchronization, or block sign-in for the Office 365 user account
through its settings in the Office 365 Admin center.

Although many of the attributes of synchronized user accounts can’t be modified using the Office 365 portal, it is still
possible to reset the password of a synchronized user in Office 365; that is if the administrator has enabled the Self-
Service Password Reset feature. When the user then changes the password in Office 365, both the on-premises and
online passwords will be out of sync until the on-premises password is updated again after which that new password
is synchronized to Azure Active Directory. To avoid the problem of disconnected passwords, you should implement
Password writeback, which will also update the password in the on-premises directory when changed using the Office
365 Self-Service Password Reset tools.

Real World: The Azure AD Connect server must be available for password changes to synchronize with Azure AD. If
the server is unavailable, the password change will not be synchronized to Azure AD nor will the password update be
queued for future processing. This means that if Azure AD Connect is unavailable and a user changes his password,
the password will be out of sync with Azure AD which can lead to potentially confusing situations and additional calls
to the help desk.

Password writeback and self-service password reset

You can avoid the issue of a password being out of sync after a password reset in Office 365 by enabling the password
writeback feature. This feature will write changed passwords back into the on-premises Active Directory after they
are changed in Azure AD. The ability to write back passwords is integrated with the self-service password reset
(SSPR) ability in Azure AD, which is only available when you purchase licenses for Azure Active Directory Premium or
the Enterprise Mobility Suite.

This is how password writeback works:

1. The user clicks a link in the Office 365 or Azure UI to request a password change.
2. The Azure AD password change service verifies whether the writeback service is running. If it isn't, the user
won’t be able to change her password.
3. Provided that the password writeback service is available, the user enters the old and new passwords.
4. The selected password is encrypted with a special key that was created during the setup of the writeback feature
for that specific tenant.
5. The encrypted password is sent over HTTPS to a tenant-specific service bus endpoint which is used to
communicate with the on-premises password writeback service. Communications on this bus are protected by a
shared password that was created during the setup of the password writeback feature and is only known to your
organization and Azure AD.
6. The writeback feature looks for the user account in the on-premises Active Directory. To find a match, the
sourceAnchor is used to look up the user in the connector space. From there, the object is traced back through
the metaverse to Active Directory.
7. If the user account is found, the password is reset. If the reset is successful, the user is notified.
8. If the password reset operation fails, an error is returned to the user. One of the reasons the password reset
might fail is when it does not meet the on-premises complexity requirements.

Directory synchronization with password hash synchronization is relatively simple to deploy without requiring
significant infrastructure and is suitable for organizations of many varied sizes. The password writeback features can
also be deployed without using password hash synchronization; you can also deploy password writeback alongside AD
FS.

Real World: You might expect Azure AD Connect to always connect to the nearest domain controller (DC). Sadly,
this is not always true. If the connection is made to a DC in a different site or over a slower connection, a variety of
problems might occur like delayed password synchronization or slow synchronization performance. Depending on
what your Active Directory topology looks like, you might have to force Azure AD Connect to communicate only with
specific DCs, preferably in the same AD site and as near to the synchronization server as possible in order to
minimize latency. How to do this, is explained in the section “Restricting sync to specific Domain Controllers” in this
chapter.

Configuring Password Writeback

Password writeback can be enabled as part of the Express or and Custom installation modes for AAD Connect, or
through PowerShell. To enable the feature using PowerShell, you must know the name of the connector to Azure AD.
To find the name of the connector, run the following PowerShell command from the Azure AD Connect server:

[PS] C:\> Import-Module ADSync

[PS] C:\> Get-ADSyncConnector | ?{$_.SubType -like "*Azure Active Directory*"} | Select Name

Then, to enable the feature, use the following command. Replace <Connector Name> by the result from the previous
command. The feature can just as easily be disabled, by replacing $true with $false in the example below:

[PS] C:\> Set-ADSyncAADPasswordResetConfiguration –Connector "<Connector Name>" –Enable $True

Before the password writeback can start writing back passwords, you must grant the synchronization service account
the necessary writeback permissions. The easiest way to do this is via PowerShell. The process is very similar to
configuring other writeback permissions, as explained in the section “Writeback permissions” in this chapter.

Run the following code from a server that has access to the dsacls.exe tool. The account that is used to run the code
must have sufficient permissions in Active Directory to grant these permissions:

#Get Domain Root

$DomainRoot = ([ADSI]"LDAP://RootDSE").rootDomainNamingContext

#Service Account Name

$accountName = "Exchangelab\svc-AADC"

#Configure Password Writeback

Write-Host "-------------------------------------------" -ForegroundColor Yellow

Write-Host "Configuring Password Writeback Permissions" -ForegroundColor Yellow

Write-Host "-------------------------------------------" -ForegroundColor Yellow

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":CA;Reset Password;user`""

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":CA;Change Password;user`""

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":WP;lockoutTime;user`""

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":WP;pwdLastSet;user`""

Configuring Self-Service Password Reset

The password writeback feature also works when an administrator changes the password for a user from the Azure
portal, but it is most useful when combined with the self-service password reset capability in Office 365 so that users
can update their own password without having to worry about the passwords getting out of sync.

Unlike most other features, password reset is not managed from the Office 365 portal. The link in the portal will
redirect you to the Azure management portal. Login to the Office 365 Admin center and navigate to Settings and
then Security and Privacy . Now click Azure AD admin center under Let your people reset their own password
. This will automatically redirect you to the right section in the Azure portal.

In the Azure portal, click Configure , and scroll down to the user password reset policy section. Click Yes next to
Users enabled for password reset . This will reveal several new options which allow you control over various
aspects of the password management features in Office 365/Azure AD, such as whether users are required to provide
multiple verification methods (to verify their identity), and if so, what options they should use. The option to turn
on/off the password writeback feature is towards the bottom of this section. If you previously configured the writeback
capability in Azure AD Connect, you will notice that the status changes to Configured . Once you have configured the
password reset capabilities to your liking, do not forget to click Save !

Using self-service password reset

Before users can use the self-service reset option, they must configure additional verification options. This process is
almost identical to when a user is enabled for MFA. Unfortunately, one does not take care of the options for the other.
As such, if you enable users for one feature first, and then for another, they will have to go through the verification
process twice.

Once the verification process is completed, there are several ways a user can change his password. One way is to go
through the Office 365 portal, click Settings (Cog Wheel) and select Password . This will take you to the page where
you can change the password. Alternatively, if the user forgot their password, they can click the “Can't access your
account ” link on the Office 365 portal after having entered a wrong password once. This brings them to the online
password reset portal where they are guided through a few steps to verify their identity based on the verification
options they provided. If the user is successfully verified, the password can be reset.

Optional AAD Connect features

Some of the optional features, such as password synchronization, support for a hybrid deployment, or password
writeback were already covered. In addition, Azure AD Connect also includes other functionality to unlock a variety of
extra capabilities in Office 365, including:

Azure AD app and attribute filtering . By default, all default attributes are synchronized to Office 365. This
 feature allows you to control what attributes are synchronized to Office 365, by either selecting which workloads
you will use in Office 365, or by selecting which attributes you (do not) want to synchronize. Mandatory
attributes, required to make the workload in Office 365 function properly, cannot be filtered.
Device writeback . Devices that are registered to a user’s account in Azure AD are synchronized back into the
on-premises directory, enabling you to use conditional access policies to shield access to on-premises applications
through integration with AD FS.
Group writeback . This feature is currently still in preview. Group writeback enables on-premises mailboxes to
interact with Office 365 Groups. Without Group writeback, Office 365 Groups do not show up in the GAL and on-
premises mailboxes are not able to send email to them unless you manually create corresponding contact objects
for each Office 365 Group. The topic of dealing with Office 365 Groups in hybrid environments is covered in the
companion volume.
Directory extension attribute sync . Although most attributes are synchronized by default, some organizations
might have extended the on-premises directory with extra, non-standard, attributes. Through this feature, those
attributes can also be synchronized to Azure AD. Additional attributes are not consumed by Office 365 workloads
and are only available to the Microsoft Graph.

Writeback permissions
The permissions required for writeback can be granted using the Delegation of Control Wizard for Active Directory
domains where synchronized objects will be located. In a Hybrid deployment, more permissions are required for
AADSync to writeback attributes in the on-premises Active Directory. A full list of the permissions and the steps for
configuring them is available on MSDN.

PowerShell is a good (and faster) alternative to the wizard. The following example uses dsacls.exe to configure
writeback permissions for the hybrid attributes for a service account called svc-AADC.

#This script configures permissions documented on:

#https://msdn.microsoft.com/en-us/library/azure/dn757602.aspx

#Get Domain Root

$DomainRoot = ([ADSI]"LDAP://RootDSE").rootDomainNamingContext

#Service Account Name (Change to your local AAD service account)

$accountName = "Exchangelab\svc-AADC2"

#Enable Hybrid Configuration Writeback

#Configure Contact Writeback

Write-Host "--------------------------------------" -ForegroundColor Yellow

Write-Host "Configure Hybrid Contact Object Permissions" -ForegroundColor Yellow

Write-Host "--------------------------------------" -ForegroundColor Yellow

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":WP;proxyAddresses;Contact`""

#Configure Group Writeback

Write-Host "--------------------------------------" -ForegroundColor Yellow

Write-Host "Configuring Hybrid Group Object Permissions" -ForegroundColor Yellow

Write-Host "--------------------------------------" -ForegroundColor Yellow

Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":WP;proxyAddresses;Group`""

#Configure User Writeback

Write-Host "--------------------------------------" -ForegroundColor Yellow

Write-Host "Configuring Hybrid User Object Permissions" -ForegroundColor Yellow

Write-Host "--------------------------------------" -ForegroundColor Yellow



Invoke-Expression "dsacls.exe '$DomainRoot' /I:S /G `"`"$accountName`":WP;proxyAddresses;User`"

`"`"$accountName`":WP;msExchArchiveStatus`" `"`"$accountName`":WP;msExchBlockedSendersHash`"
`"`"$accountName`":WP;msExchSafeRecipientsHash`" `"`"$accountName`":WP;msExchSafeSendersHash`"
`"`"$accountName`":WP;msExchUCVoiceMailSettings`" `"`"$accountName`":WP;msExchUserHoldPolicies`""

For other writeback features, Azure AD Connect includes a new PowerShell module which contains cmdlets that
configure the feature, along with the necessary permissions. At time of writing, the following cmdlets are available:

Initialize-ADSyncDeviceWriteBack to configure Device Writeback.

Initialize-ADSyncDomainJoinedComputerSync configures the ability to writeback Azure AD-joined computers
back into the on-premises directory.
Initialize-ADSyncGroupWriteBack to configure permissions for Group writeback.
Initialize-ADSyncNGCKeysWriteBack is used to write back key material from Azure AD-Joined computers into the
on-premises directory. This feature will require Windows Server 2016 AD.
Initialize-ADSyncUserWriteBack to configure permissions for User writeback. This feature used to be in preview,
but was removed from Azure AD Connect. It is expected to turn up again in the future.

Before you can use any of the above cmdlets, you must first import the additional ADSyncPrep module that is
available on the Azure AD Connect server. To do this, you must open an administrative PowerShell session:

[PS] C:\> Import-Module 'C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep\AdSyncPrep.psm1'

Then, you can continue using the cmdlets. For example, to configure permissions for the Group writeback feature:

[PS] C:\> Initialize-ADSyncGroupWriteBack –AdConnectorAccount "account"

–GroupWriteBackContainerDN "OU=Groups,DC=Domain,DC=Com"

Note: The account used to implement writeback must have write permissions to the OU where objects are written
back to.

Multi-forest directory synchronization

Multi-forest Active Directory environments create some additional complexity when it comes to synchronizing with
Azure Active Directory. There are two general approaches that you can take:

Consolidate the user accounts from all forests into a single "account" forest, and synchronize that forest with
Azure Active Directory
Synchronize multiple forests with Azure Active Directory by choosing a compatible directory synchronization
tool.

Real World: Another approach is to use “shadow accounts”, which involves syncing or manually provisioning the
user accounts from different forests into a single forest from which they will be synced to Azure Active Directory.
However, this approach is complex and requires the deployment of a synchronization or provisioning tool such as FIM
or custom scripting to manage the shadow accounts. Azure Active Directory Connect (Azure AD Connect) does not
provide that functionality.

When using Azure AD Connect in a multi-forest scenario there is a requirement that the synchronized objects must
only exist in one of the Active Directory forests. If a user has enabled accounts in multiple forests, those accounts
must have unique attributes (the User Principal Name and email address in particular) to enable the synchronization
tool to view them as separate objects. Similarly, if you have multiple Exchange organizations that you are planning to
configure in a Hybrid deployment with a single Office 365 tenant to configure in a single hybrid configuration then
FIM 2010 R2 or Azure AD Connect must be used.

Managing and monitoring directory synchronization

After you implement directory synchronization, you naturally need to manage and monitor it on an ongoing basis.

Checking the synchronization interval

Earlier versions of the synchronization tool relied on a scheduled task to perform the synchronizations. In Azure AD
Connect v1.1 and later, the scheduled task is replaced by a built-in synchronization scheduler. The scheduler is only
available through PowerShell. To check the current settings, run the following command from the directory
synchronization server:

[PS] C:\> Import-Module ADSync



[PS] C:\> Get-ADSyncScheduler

AllowedSyncCycleInterval : 00:30:00

CurrentlyEffectiveSyncCycleInterval : 00:30:00

CustomizedSyncCycleInterval :

NextSyncCyclePolicyType : Initial

NextSyncCycleStartTimeInUTC : 2/28/2016 4:47:40 PM

PurgeRunHistoryInterval : 7.00:00:00

SyncCycleEnabled : False

MaintenanceEnabled : False

StagingModeEnabled : False

AllowedSyncCycleInterval specifies the minimum time between (automatic) synchronizations Azure AD will allow.
Note that this is a fixed value and cannot be changed.
CurrentlyEffectiveSyncCycleInterval denotes the current interval. If a custom schedule was configured, the
CustomizedSyncCycleInterval will contain the currently used interval.
NextSyncCyclePolicyType shows the type of the next synchronization. This can be either Delta or Initial . Delta
means only changes will be picked up on. Initial means the next synchronization will be a full import and sync,
which also includes changes.
NextSyncCycleStartTimeinUTC . As the name implies, it displays the time the next synchronization is scheduled
to start (converted to UTC timing).
PurgeRunHistoryInterval controls how long the run history should be kept. The default is 7 days. The run history
can be useful for troubleshooting purposes, but also consumes disk space!
SyncCycleEnable indicates whether the scheduler is running, and if it's active what action it currently is
executing.
MaintenanceEnabled shows if maintenance mode is enabled.
StagingModeEnabled shows if this synchronization server is configured in Staging Mode.

Forcing synchronization
Depending on the version of Azure AD Connect, the default synchronization interval is 30 minutes to 3 hours.
Sometimes an administrator might need to manually trigger a synchronization sooner, for example if an urgent change
need to be synchronized.

How to force a synchronization depends on the version of the synchronization tool in use. In earlier versions of Azure
AD Connect, you started a scheduled task called "Azure AD Sync Scheduler" that was created automatically during
the installation of the tool. For Azure AD Connect 1.1 and later, to start a new delta synchronization cycle, run this
PowerShell command on the Azure AD Connect server:

[PS] C:\> Start-ADSyncSyncCycle –PolicyType Delta

If you need to trigger a full synchronization, replace Delta with Initial in the command shown above.

Monitoring synchronizations
Once directory synchronization has been setup and the initial synchronization completes the synchronization process
will be run again every 30 minutes. The history of synchronization attempts is called the “run history”, and is stored
within the SQL database used by the tool. Monitoring the run history is an effective way to keep track on your
directory synchronization deployment. The run history is exposed through the Synchronization Service Manager on
the directory synchronization server, which provides the administrative interface for managing the tool, and it’s also
abstracted and displayed in the Office 365 Admin center if you have the DirSync Status card displayed (Figure 3-10 ).
This is the fastest way to quickly check whether dirsync is basically healthy, as it always shows you whether the
service has recently seen a dirsync attempt and if password hash sync is enabled and successful.
Figure 3-10: The DirSync Status card in the Office 365 Admin center

If you want more detail, you can always use the Synchronization Service Manager, which exposes far more controls
than an administrator needs. You should always use caution when viewing information in this tool. A good rule to stick
to is “look but don’t touch”, unless you are making a supported change using a Microsoft documented procedure. The
Synchronization Service Manager is found in the default installation directory, for example C:\Program
Files\Microsoft Azure AD Sync\UIShell\miisclient.exe . After the tool is open, you can see the run history from the
Operations tab (Figure 3-11 ).

Multiple entries are logged for each synchronization cycle. Depending on the number of domains and whether you are
running a hybrid deployment or not, you will see different entries per cycle. Each step within a synchronization cycle
has its own status that shows the overall result of the step. Of course, "success " is what you are hoping to see, but
often other statuses like "completed-export-errors " can be observed too. Those are the type of events you look for
to start troubleshooting.

The Synchronization Service Manager is not the only way to expose this information. As mentioned earlier, the run
history information is stored in the SQL database. A useful script is available that uses PowerShell to query the SQL
database and return relevant information on DirSync such as the last synchronization attempt. The script has been
subsequently updated to support Azure AD Connect as well. Be sure to check both scripts out as they can help you to
monitor your directory synchronization implementation.

Figure 3-11: Viewing synchronization operations

Triggering full password synchronization

When password sync is first enabled, it will sweep through the on-premises Active Directory and bulk-update all the
user passwords in Office 365 so that they are in sync with their on-premises counterpart. Generally, it is not required
to repeat this process. However, some situations might require you to force a new full password synchronization (not
to be confused with a full synchronization). To (re-)synchronize all passwords, run the following PowerShell commands
from the Azure AD Connect Server:

[PS] C:\> $onPremConName = "Robichaux.net"

$onPremCon = Get-ADSyncConnector –Name $onPremConName

$cloudConName = "cajunlab.onmicrosoft.com - AAD"

$pwSyncSettings = New-Object Microsoft.IdentityManagement.PowerShell.ObjectModel.ConfigurationParameter

"Microsoft.Synchronize.ForceFullPasswordSync", String, ConnectorGlobal, $null, $null, $null

$pwSyncSettings.Value = 1

$onPremCon.GlobalParameters.Remove($pwSyncSettings.Name)

$onPremCon.GlobalParameters.Add($pwSyncSettings)

$onPremCon = Add-AdSyncConnector -Connector $onPremCon

Set-ADSyncAADPasswordSyncConfiguration –SourceConnector $onPremConName –TargetConnector $cloudConName –Enable $false

Set-ADSyncAADPasswordSyncConfiguration –SourceConnector $onPremConName –TargetConnector $cloudConName –Enable $true

After completing these commands, you should see multiple events with ID 650 in the Application event log. Each of
these events indicates a batch of users whose passwords are being synchronized to Office 365. Note that it might take
a while for this process to complete, depending on the size of your environment.

Restricting sync to specific domain controllers

Consider the following scenario: you have multiple Active Directory sites across several geographically dispersed
locations all over the world. Unsurprisingly, some of these locations have better connectivity than others and you
might not want Azure AD Connect to connect to domain controllers in locations with a slow or high latency connection
at the risk of slowing down the entire synchronization process.

When Azure AD Connect connects to a new forest, it uses DNS to locate domain controllers it needs to connect to.
Without additional configuration, it is very difficult to control or to know exactly which domain controllers Azure AD
Connect connects to. By default, Azure AD Connect will try to connect to domain controllers within the same site first.
However, that does not necessarily apply to remote forests as there is no way for the synchronization engine to know
which site in the remote forest is closest. Additionally, I have seen cases where Azure AD Connect did not select a
domain controller within the same site.

There is no way to control the behavior during the installation process. However, once Azure AD Connect is installed,
you will find that it is relatively easy to define a (static) list of domain controllers that it should connect to.

First, open the Synchronization Service Manager on your synchronization server. This executable (miisclient.exe) is
typically located in C:\Program Files\Microsoft Azure AD Sync\UIShell . Then, navigate to Connectors and locate
the connector, specific for your on-premises domain (forest). Now, right-click the connector and click Properties . In
the properties window (Figure 3-12 ), navigate to Configure Directory Partitions (1) and make sure to check the
box next to Only use preferred domain controllers (2) :

Figure 3-12: Configuring a static list of domain controllers in Azure AD Connect

In the Configure Preferred DCs window, add the domain controllers you want AAD Connect to interface with. You
can order the domain controllers preference by moving them up/down the list. Once you have completed the list, click
OK to confirm the changes. Azure AD Connect will now only talk to the domain controllers you have specified.

Upgrading Azure AD Connect

New builds of the directory synchronization tools are released as new features are added and bugs are fixed.
The in-place upgrade is the easiest way to upgrade. After launching the installer, Azure AD Connect will detect an
existing synchronization tool installed and offer you to upgrade it to Azure AD Connect, provided you do not have
more than 50,000 objects. If you haven’t already upgraded AADSync to Azure AD Connect, you should do so
immediately. In the process, remember that:

Customized settings such as the synchronization schedule will be reset to the default configuration
Any changes that you made to pre-configured synchronization rules will be lost
Any pre-configured synchronization rules that you deleted will be recreated
Custom inbound synchronization rules that you created will be retained
Custom outbound synchronization rules that you created will be lost

With these caveats in mind, you should:

Make as few customizations to the synchronization engine/rules as possible

Export your custom synchronization rules using the Synchronization Rules Editor before running any upgrade
Document all other customizations to pre-configured rules so that you can reapply them after an upgrade

To perform an in-place upgrade, run the Azure AD Connect installer from the existing synchronization server. The
setup automatically detects the pre-existing synchronization tool and offers you to upgrade it to Azure AD Connect,
keeping the limitations in mind.

Synchronization is disabled during the upgrade, which usually takes anywhere between 10 to 30 minutes depending
on the size of the existing metaverse. During the upgrade process, you cannot make any configuration changes. The
credentials that you must specify to connect to Azure AD and the on-premises Active Directory must match with what
was used before. After the upgrade, the Azure AD Connect shortcut will be available from the desktop. From there,
you can launch the configuration wizard to make additional changes to the installation.

A swing migration requires a little more work. This approach is recommended if you have more than 50,000
synchronized objects, if you have a complex dirsync configuration with lots of customizations or you have a FIM
deployment with the Office 365 connector and wish to replace it with Azure AD Connect.

The steps to perform a swing migration are:

1. Install Azure AD Connect on a new server. It is important that you do not start synchronization after the
installation.
2. Start the Azure AD Connect configuration wizard and enable staging mode. This will prevent the tool to export
any unwanted changes to Azure AD or the on-premises Active Directory.
3. Re-apply customizations like object filtering to the Azure AD Connect installation.
4. Start synchronization on the new Azure AD Connect server, without disabling staging mode . This allows you to
verify what objects it will synchronize with Azure AD and the on-premises Active Directory once staging mode is
disabled. You can use tools such as csexport and csexportanalyzer to compare the results from the new Azure AD
Connect installation with the existing synchronization tool.
5. Once you are happy with the outcome (i.e. the expected set of objects are synchronized), you can decommission
the old synchronization server and disable staging mode on the Azure AD Connect server.

Azure AD Connect automatic upgrades

With the release of Azure AD Connect v1.1, Microsoft announced a new feature called Automatic Upgrade. Because of
the frequent updates to Azure AD Connect, it can be cumbersome having to manually update the installation each
time an updated version is available. Automatic Upgrade replaces the need to manually update Azure AD Connect by
enabling each AAD Connect server to automatically pull updates when Microsoft releases them, in the same way that
Windows Update works. Automatic Upgrade is enabled by default when the following conditions are met:

Azure AD Connect was configured by an Express installation and no custom service account is used
There are no more than 100,000 objects in the metaverse
The SQL Express LocalDB database is used

To view if Automatic Upgrade is enabled, run this PowerShell command on the Azure AD Connect server:

[PS] C:\> Get-ADSyncAutoUpgrade

Enabled

The cmdlet can return three values: Enabled, Disabled or Suspended. Both Enabled and Disabled clearly show their
meaning. Suspended means that the system determined that it's no longer eligible for automatic upgrades. For
instance, this is the case when Azure AD Connect was reconfigured with custom settings.

If your system is enabled for automatic upgrades, but you don't want to automatically install new versions, you can
run the following command to manually disable it:

[PS] C:\> Set-AdSyncAutoUpgrade –AutoUpgradeState Disabled

Real World: Even when Auto Upgrade is enabled, you might not see the latest version installed. This is because
Microsoft also must enable customers for automatic upgrade in the service. It might take a while before all customers
have been enabled for automatic updates (in the service). Until then, you can still upgrade manually.

Monitoring dirsync and AD FS with Azure AD Connect Health

When a synchronization server is (temporarily) unavailable, there is no immediate impact on the environment except
if password synchronization or password writeback have been configured. As such, synchronization failures or
conflicts can go unnoticed very easily, especially because monitoring of the service is often an afterthought.

Until recently, only a few third-party solutions monitor the directory synchronization server and the synchronization
process. Azure AD Connect Health provides an alternative solution. It allows customers to monitor both the
synchronization process and its federation servers (AD FS).

The agent that uploads diagnostic information to Azure is automatically installed with Azure AD Connect. As a result,
no other configuration is required to start using the service. However, you must have an Azure Active Directory
Premium license for the user(s) accessing the dashboard (Figure 3-13 ). To access the health dashboard, navigate to
https://aka.ms/aadconnecthealth . From there, the administrator can navigate through different widgets that give
insights into various aspects of the synchronization process, like detailed information about the latest synchronization
attempts, connection latency, the number of synchronized objects, and synchronization errors. If needed, the
administrator can also configure notifications so that an email is sent, every time a problem is detected.

Figure 3-13: Azure AD Connect Health Dashboard

To monitor AD FS servers, the Azure AD Connect Health agent must first be installed on the federation servers and
Audit logging must be enabled. Once the servers have been configured to report into the Health Service, you will be
able to find a variety of information on the federation servers and the authentication process. The latter is especially
useful because it allows you to track suspicious activity such as failed logons etc.

If you do not have Azure Active Directory Premium licenses, you can still monitor the synchronization process, albeit
in a more basic way as the new Office 365 Admin portal includes a section on the synchronization process status in its
dashboard.

Active Directory Federation Services

and Office 365
Configuring single sign-on for Office 365 is done in two steps. First, the AD FS infrastructure must be configured.
Then, you enable one or more domains for federation. Active Directory Federation Services is a built-in role of
Windows Server 2008 R2, 2012, 2012 R2 and 2016. Windows Server 2008 R2 was released with version 1.1, which is
not supported by Office 365. If you want to deploy your AD FS farm on Windows Server 2008 R2, you must first
configure version 2.0, which is available for free from Microsoft as a separate download.

From an authentication perspective, there is no difference whether you run AD FS on Windows Server 2008 R2 or
later editions of Windows Server. However, from a functionality standpoint, AD FS in Windows Server 2012 R2 and
later offers some benefits, including:
 Enhanced security because a dependency on IIS no longer exists
Simplified deployment through a new UI wizard which also supports integrating with a separate SQL server
(instead of the built-in database)
Remote installation and configuration
Enhanced AD FS sign-in experience through extensive customization and branding capabilities
Built-in support for multi-factor and other authentication types which makes it easier to configure conditional
access (for example, location-based)
Faster and easier creation of custom claim rules
Easier password management (for Office 365 users)

In the following sections, we describe how to configure AD FS with Office 365. Although Azure AD Connect can take
care of most of these tasks for you, going through the motions manually provides you with a better understanding of
how everything comes together, making troubleshooting easier if you ever run into issues.

Real world : In August 2018, the team at Okta (which, coincidentally, makes a product which competes with AD FS)
identified a security vulnerability in AD FS when used with MFA. An attacker who can capture one user’s password
and MFA authentication code can use the same MFA code as a second factor when logging in as any other user in the
organization. Microsoft patched it quickly (see the relevant KB articles ), and because the patch is included in the
monthly security rollup, anyone who’s regularly patching their AD FS servers will get the patch. The specific
vulnerability here sounds horrifying but in practice is not quite as bad as press reports make out. An attacker who
wants to compromise Alice’s account using this vulnerability must have the user name and password for Alice’s
account and the user name, password, and MFA code for Bob’s account. While this is certainly possible, it is fairly
unlikely.

Configuring AD FS
The following example assumes that you use a Windows Server 2016 machine. You can enable the built-in AD FS role
using the Server Manager UI or through PowerShell:

[PS] C:\> Install-WindowsFeature ADFS-Federation

Success Restart Needed Exit Code Feature Result

------- -------------- --------- --------------

True No Success {Active Directory Federation Services}

After restarting the server, you can launch the AD FS configuration wizard (Figure 3-14 ) from Server Manager. The
wizard will guide you through a series of steps and questions to configure the server.

1. Welcome : allows you to choose to create a new farm or add servers to an existing farm.
2. Connect to AD DS : requires you to enter credentials for an account that is member of the Enterprise
Administrators security group. This account is only used to create the AD FS configuration container in Active
Directory during setup.
3. Specify Service Properties : here you specify the SSL certificate which the AD FS server should use to secure
communications using HTTPS. The certificate must be issued by a public, trusted Certification Authority and the
subject name of the certificate must match the host name that will be used to publish the AD FS farm onto the
Internet.
4. Specify Service Account : this page allows you to specify a service account which will be used to run the AD FS
service on the AD FS servers. The account should be a regular domain user account and does not require any
elevated permissions. All servers in a farm use the same account and Windows will manage its password. You can
also specify a Group Managed Service Account, if your on-premises Active Directory supports it.
5. Specify Databas e: on this page, you can choose what database type should be used to store the configuration
data.
Figure 3-14: The AD FS Configuration Wizard

As explained before, the AD FS deployment can be made more resilient by configuring multiple servers in a single
farm. A load balancing solution such as Windows Network Load Balancing or an external load balancer must then be
used to distribute incoming requests across the different servers in the farm. It is recommended to deploy at least two
federation servers for redundancy. A single AD FS server can handle the connections created by approximately 15,000
users. Beyond that, you should add servers into the farm to increase capacity up to the expected workload. Of course,
these are just arbitrary numbers. If you want to properly size your AD FS environment, use Microsoft’s ADFS Capacity
Planning Guide for AD FS 2.0 . The advice also applies to more recent versions of AD FS.

Validating the AD FS Installation

In prior versions of AD FS, you could navigate to https://<servicename>/adfs/ls/IdpInitiatedSignon.aspx once
configuring the server. You could then use the page to test if the AD FS server functioned properly (i.e.: talk to Active
Directory and authenticate the user). In Windows Server 2016 this feature is disabled by default. To access the page,
you must first run the following PowerShell command from the AD FS server, after which you can test the
functionality:

[PS] C:\> Set-AdfsProperties -EnableIdPInitiatedSignonPage $true

If Windows Integrated Authentication is enabled, the web page might automatically sign you in (SSO) and not display
the authentication form. Note that once you confirm that AD FS works as expected, you should probably disable the
feature again. The only reason to leave it enabled is if you want to use AD FS to provide SSO for third-party
applications that allow users to initiate sign in from their pages. For example, you can configure Amazon Web
Services, BambooHR, or the Roadmunk or Aha! product planning tools in this mode.

If you can successfully authenticate, it means the AD FS servers can successfully talk to Active Directory, validate user
credentials, and produce valid authentication tokens. At this point, however, you cannot accept logon request from
Office 365. For that, you must first configure a trust relationship with Office 365 so that the service knows where to
send logon request, and your AD FS farm is willing to accept them.

Enabling domains for federation

By default, when you add and validate a custom domain to Office 365, it will be marked as a “managed” domain,
meaning that identities in that domain must be authenticated directly by the service. In order to use AD FS, you must
federate your domains. You cannot mark a domain as federated when you create it, and there is no GUI to configure
federation. You must federate domains using PowerShell. For a detailed walk-through of how to add and verify
domains to an Office 365 tenant, refer to the bonus guide available at http://exchangeserverpro.com/office-365-
bonuses .

Because authentication for Office 365 configuration is handled by Azure AD, you must make a remote PowerShell
connection to Azure AD first. This can only be done after you install the following components:

Microsoft Online Services Sign-In Assistant

PowerShell module for Azure Active Directory
After these components have been installed, open PowerShell and run the following commands to connect to Azure AD
and convert the tenant domain to a federated domain. The first command is only required if you are running on a
Windows Server 2008 R2 computer:

[PS] C:\> Import-Module MsOnline

[PS] C:\> Connect-MsolService –Credential (Get-Credential)

[PS] C:\> Convert-MsolDomainToFederated –DomainName "o365itpros.com"

Successfully updated 'o365itpros.com domain.

Running the Convert-MsolDomainToFederated cmdlet should result in a message that the domain was updated
successfully. Note that running this command requires the use of the 'legacy' Azure AD PowerShell module. This is
because the newer version (using the *-AzureAD* prefixes) does not support this functionality yet.

Warning. Running the Convert-MsolDomainToFederated command will immediately change the authentication
method for that domain. Don’t run it until you’ve tested your AD FS configuration and are ready to put it into
production.

The PowerShell example assumes that the code is executed from one of the AD FS servers in your environment. If you
are running the Azure AD PowerShell module from another non-AD FS server or management workstation, you must
execute the following command prior to converting the domain. This will ensure that the AD FS configuration
information can be read during the configuration of the domain federation. In this example, the administrator is
prompted for credentials. Note the account used must have administrative access to the AD FS servers.

[PS] C:\> Set-MsolADFSContext –Computername <ADFS Server> -ADFSUserCredentials (Get-Credential)

When Convert-MsolDomainToFederated runs, a trust relationship is created between the on-premises federation
server farm and the authentication platform in Azure. As part of creating this trust, specific information from the on-
premises AD FS environment is uploaded to Azure. To view which information was stored during the process, you can
run the Get-MsolDomainFederationSettings and Get-MsolFederationProperty cmdlets. The information includes:

Certificate details including information about the current token signing certificate (TokenSigningCertificate )
and the future token signing certificate (NextTokenSigningCertificate ). The token signing certificate is used by
the AD FS server to digitally sign the tokens it generates. Office 365 uses the public key of the certificate to
verify and validate the token before accepting is as a proof of authentication.
The federation endpoint information includes hostname(s) of the on-premises federation endpoint. These
different endpoints are important to the AD FS operations as they allow to differentiate between the different
methods of authentication (active, passive, etc.).

The NextTokenSigningCertificate value is used only when the automatic certificate rollover feature in AD FS is used.
The automatic rollover will ensure that the new (future) token signing certificate is automatically used shortly before
the current certificate expires. This step prevents an outage in case the administrator forgets to renew the certificate.
There is a catch though. The trust only maintains information on the current and a single future certificate. As a
result, after each certificate rollover, the domain federation information in Office 365 must be updated using the
Update-MsolFederatedDomain cmdlet.

Real world: To prevent AD FS outages due to the rollover of a certificate, an administrator can use the Microsoft
Office 365 Federation Metadata Update Automation Install Tool . This tool creates a scheduled task that will take
care of updating the federation metadata as described above. Please note that the tool has only been tested with AD
FS 2.0, but it should also work for AD FS 3.0 environments. Alternatively, you can also configure the AD FS
certificate with a longer duration so that it does not have to be updated as often. See the this helpful text on the topic
. Note that other applications (such as Dynamics CRM) may have their own certificate rollover requirements when
used with AD FS.

Validating the AD FS Configuration

Once you have run the Convert-MsolDomainToFederated command, users should be able to login to Office 365
through your federation servers. You can easily test the authentication by signing in to Office 365 and verifying that
you are redirected to your federation service endpoint, and that you can successfully authenticate using your on-
premises credentials.

 Note: To test federated authentication, you must have at least one user from your on-premises directory
synchronized with Azure AD and the synchronized user's UPN suffix must match the federated domain.

Alternatively, you can use Microsoft's Remote Connectivity Analyzer , which includes a section specifically created to
test authentication to Office 365 through federation servers. The test can be found under the Office 365-tab and is
called Office 365 Single Sign-On test. To check if and for which domains federation has been enabled, you can look at
the Domains section in the Office 365 portal, or connect to Azure AD PowerShell and run the following command:

[PS] C:\> Get-AzureADDomain | ?{$_.AuthenticationType -eq "Federated"}

Name AvailabilityStatus AuthenticationType

---- ------------------ ------------------

Office365itpros.com Federated

To get a list of all federated identities (users who will authenticate through the federation solution), run the following
commands. Note that these commands might take a while to complete in larger environments:

[PS] C:\> $federatedDomains = Get-AzureADDomain | ?{$_.AuthenticationType -eq "Federated"}

[PS] C:\> foreach($domain in $federatedDomains){ Get-AzureADUser | ?{$_.UserPrincipalName –like "*@"+$($domain.Name)} | Select

DisplayName,UserPrincipalName}

DisplayName UserPrincipalName

----------- -----------------

Tony Redmond tredmond@office365itpros.com

Paul Cunningham pcunningham@office365itpros.com

Michael Van Horenbeeck mvanhorenbeeck@office365itpros.com

Federating Sub-domains
Sometimes a customer might want to enable federation only for some of its users. This is common in the education
market, where a large university might have 50,000 or more student accounts in Office 365, but where it is
undesirable to have those students leverage federation because it could generate a heavy load on their AD FS
infrastructure. Another scenario is when an organization seeks to segregate control over domain authentication.
Controlling the authentication per subdomain allows you to specify a different AD FS endpoint for each domain. For
example, federation requests for europe.domain.com can be controlled by an AD FS server farm in Europe while
requests for us.domain.com are handled by a server farm in the US.

When a domain is federated, all its sub-domains are automatically federated with it. However, it is possible to
selectively enable federated authentication for sub-domains by executing the following steps in order:

1. Register the child domain(s) in the Office 365 tenant

2. Register parent domain in the Office 365 tenant
3. Configure authentication per (sub-)domain

It is crucial that you register the child domains before registering the parent domain. If your tenant already has
domains configured, you can use the following PowerShell command connected to Azure AD to determine if a sub-
domain was added before or after the parent domain:

PS C:\> Get-MsolDomain -DomainName childdomain.parent.com | Format-List *

ExtensionData : System.Runtime.Serialization.ExtensionDataObject

Authentication : Federated

Capabilities : None

IsDefault : False

IsInitial : False

Name : childdomain.parent.com

RootDomain : parent.com
Status : Verified

VerificationMethod : DnsRecord

If the RootDomain property points to a parent domain, it means that the child domain and parent domain are linked.
As such, if you modify the authentication type for the parent domain, it automatically changes for the child domain as
well. Unfortunately, there is no uncomplicated way to switch to the scenario where you can control the child domain
separately from the parent domain. If you find yourself in that situation, you should open a ticket with Microsoft
support and seek their help. If the RootDomain property is empty, you can control the authentication type of the
parent and child domain separately. To convert the authentication type for one or more subdomains, without affecting
the parent domain or vice-versa, you must use the SupportMultipleDomain switch when executing the Convert-
MsolDomainToFederated cmdlet as illustrated below:

PS C:\> Convert-MsolDomainToFederated -DomainName childdomain.parent.com -SupportMultipleDomain

Successfully updated 'child1.parent.com' domain

The above approach works perfectly if you only have a single domain tree, like child.domain1.com,
child2.domain1.com, child3.domain1.com etc. In larger, more complex, environments where use multiple domain trees
are used, things become a little trickier if you want to use the same AD FS infrastructure for all domains; in some
conditions authentication will fail as soon as you add another domain that belongs to a different domain tree.

The AD FS configuration database

AD FS servers use a configuration database to hold a set of parameters and data to control server operation. By
default, the configuration database is stored in the Windows Internal Database (WID) that comes with Windows
Server 2008 and later. Alternatively, the configuration data can also be stored in a separate Microsoft SQL Server
database. A Windows Server 2016 farm based on the WID is limited to a maximum of 30 servers , provided there are
no more than 100 trust relationships (Relying Party Trusts) which is a lot more than you need to support only Office
365. However, if there is a need to scale out beyond these limitations, you should configure AD FS to use an external
SQL database. In addition to being more scalable, using a SQL database allows you to enable additional security
features such as token replay detection or token artifact resolution. Both features can be very useful to protect other
applications that leverage AD FS, but they do not provide much value for Office 365.

The token replay detection feature enables AD FS servers to detect when a token request is re-used, by storing
additional information about each successful authentication in the SQL database. The feature is particularly useful in
a scenario where multiple people share a common workstation (for example, a kiosk) and a malicious user tries to
leverage the previously logged on user's history to resubmit the same authentication request. The feature is intended
to protect the application that is being accessed, in case the application itself is not capable of detecting replayed
tokens. The authentication process in Office 365, on the other hand, is designed to detect the re-use of tokens, almost
nullifying the additional value of the Token Replay detection.

The same is true for artifact resolution. For this feature to provide any value, the application (Office 365 in this case)
needs to specifically take advantage of the feature. However, Office 365 does not leverage artifact resolution, even if
it's enabled. Using the internal database is a lot simpler and cheaper than SQL Server and there are other, perhaps
better, ways to mitigate identity theft such as enabling multi-factor authentication which also protects an account
beyond AD FS alone.

If you choose to use the WID during the AD FS configuration, the first server that is joined to the farm will
automatically create the database. This server then continues to assume the role of ‘Primary AD FS Server’. Because
of the importance of the configuration database, all servers that are subsequently added to the farm will replicate the
contents of the configuration database from the primary federation server to make sure that a local copy is available.
This approach ensures that each server can function independently and increases the tolerance against failures of a
single server in the farm. After the initial replication, each “secondary” federation server will poll the primary
federation server every five minutes to check for updates and – if necessary – synchronize them.

In the WID scenario, configuration changes can only be made from the AD FS Management Console on the primary
federation server. If for some reason that server is unavailable, the other servers will continue to service requests, but
you cannot make any changes to the configuration until the primary server is brought back online. If necessary, a
secondary federation server can be promoted to become the primary server as shown in the following example:

[PS] C:\> Set-AdfsSyncProperties –Role PrimaryComputer

Next, all other servers in the farm must be reconfigured to now synchronize from the new primary federation server:

[PS] C:\> Set-AdfsSyncProperties –Role SecondaryComputer

–PrimaryComputerName <FQDN of primary federation server>

Switching from WID to SQL

Changing from the Internal Database to using a SQL database is not an easy task. Instead, it is better to ensure you
choose the correct database from the start. However, sometimes requirements change, or you only find out later that
you need more than 100 Relying Party Trusts, for example. If you find yourself in that scenario, you have two options:

1. Create a new federation server farm and configure it to use SQL by specifying the SQL database information
 during the setup.
2. Migrate the WID to SQL. From a high-level, this process includes the following steps:

1. Stop the AD FS Windows service on the primary AD FS server.

2. Use the SQL Management Tools to connect to the Windows Internal Database and detach the AD FS databases.
3. Copy the database files to the SQL server and attach the databases. Before you do so, ensure that the AD FS
service account has a login on the SQL server.
4. Grant the AD FS service account appropriate permissions to the recently-attached AD FS databases.
5. Point the AD FS server farm to the SQL database instead of the WID, and then restart the AD FS service on the
primary federation server.
6. Stop the AD FS Windows service on secondary federation servers in the farm and update the AD FS properties to
point to the SQL database, one by one. On each server, restart the AD FS Windows service.

AD FS proxy servers
Another component to consider deploying is the AD FS Proxy server or alternatively, for AD FS in Windows Server
2012 R2 and later, the Web Application Proxy (WAP). Although AD FS proxy servers aren't required, it is highly
recommended to deploy them to increase security of the AD FS deployment and unlock additional features. Like the
AD FS servers, you can deploy one or more proxy servers in a load balanced proxy server farm for redundancy. That
farm is then configured to connect to the internal AD FS farm.

A good reason to deploy proxy servers is to avoid exposing the AD FS servers directly to the Internet. The AD FS
proxy server does not have to be a domain-joined server, so it acts as a barrier between clients and the AD FS servers.
The proxy accepts token requests from users and then passes the information securely to the internal AD FS servers;
if the authentication succeeds. For this reason, proxies are frequently deployed in a perimeter network. It also
receives tokens from the internal AD FS servers and passes them back to the user. Deploying a proxy does not change
how AD FS works or the hostname that is used to publish the farm onto the Internet. However, it does allow additional
functionality, such as making a distinction between authentication requests coming from inside or outside the
organization's network.

In Windows Server 2008, 2008 R2 and 2012, the AD FS proxy came as part of the additional AD FS 2.0 download. In
Windows Server 2012 R2 and later, Microsoft replaced the dedicated AD FS Proxy server role with the Web
Application Proxy, which is part of the Remote Access service. Some load balancer vendors such as F5 or Citrix offer
alternatives for the AD FS Proxy servers. Instead of deploying additional Windows Servers, a service within their
solution can take on the function of the AD FS Proxy. Of course, it is always a clever idea to check with Microsoft if the
solution you are considering is supported or not.

The process to deploy an AD FS Proxy Server or Web Application Proxy is very similar to how a regular AD FS server
is deployed. The following example guides you through the setup of a Web Application Proxy on Windows Server 2016.
The Web Application Proxy Role Service is part of the Remote Access Server Role and can be installed using the
Server Manager UI or through PowerShell:

[PS] C:\> Install-WindowsFeature web-application-proxy

Success Restart Needed Exit Code Feature Result

------- -------------- --------- --------------

True No Success Remote Access, Web Application Proxy}

In addition to the Web Application Proxy, it is recommended that you also install the Remote Access Management
Tools to provide access to the Web Application Proxy wizard. After the tools have successfully been installed, but
before running the Web Application Proxy Wizard, you must import the communications certificate on the server. This
certificate is selected in the wizard and is used to secure communications with clients. The certificate should, at the
very least, include the hostname of the federation endpoint. For instance, adfs.office365itpros.com. When the
certificate is imported, open the Remote Access Management Console and click on Web Application Proxy in the
navigation pane. Then click, Run the Web Application Proxy Configuration Wizard from the results pane and
follow the steps on screen:

1. Welcome : informs you that you are about to installe a Web Application Proxy.
2. Federation Server : requires you to enter the federation service name and credentials for an account that is a
local administrator on the federation servers. This account is only used to create the AD FS Proxy trust with the
federation servers.
3. AD FS Proxy Certificate : here you specify the SSL certificate which the Web Application Proxy server should
use to secure communications (HTTPS). The certificate must be issued by a public, trusted Certification
Authority and the subject name of the certificate must match the host name that will be used to publish the AD
FS farm onto the Internet.

If the wizard completed successfully, you should be able to observe a series of events in the AD FS Admin Event log.
First an event ID 391 will be logged, indicating that a trust was successfully established with the federation service,
followed by an event 245, and 252 denoting that the Web Application Proxy was able to retrieve and update its
endpoint information. Finally, an Event ID 198 indicates the federation proxy started successfully.
Extranet Lockout
Because the Web Application Proxy role allows you to differentiate internal from external access, you can also benefit
from the built-in extranet lockout policy, which further hardens your deployment. The feature functions as a protection
against brute force guesses, but also as a barrier against DDOS attacks.

This feature is a little misnamed. When the maximum number of failed logon attempts is reached, the account itself
isn’t locked out; instead, the AD FS server will stop processing authentication attempts for that account for the
duration specified. This choice was intentional to avoid blocking the user from accessing resources internally.

By default, the extranet lockout policy is disabled, and it can only be enabled through PowerShell using the Set-
ADFSProperties cmdlet. When configuring the extranet lockout policy, the following options are available:

ExtranetLockoutThreshold : this parameter defines the maximum number of failed logons before an account is
temporarily locked out.
ExtranetObservationWindow : this value serves multiple purposes. It defines the time span during which the AD
FS servers keep track of failed logons and also defines how long federation servers will stop processing requests
for the affected account once the maximum number of failed logons has been reached.

In the following PowerShell example, the extranet lockout feature is enabled, and the threshold for failed logon
attempts is set to 10 failed attempts in 20 minutes:

[PS] C:\> Set-AdfsProperties –EnableExtranetLockout $true

–ExtraNetLockoutThreshold 10 –ExtranetObservationWindow (New-Timespan –Minutes 20)

Changing the token lifetime

In a default Microsoft AD FS implementation, the token returned as part of the authentication process is valid for up
to 8 hours. The SSOLifeTime attribute controls the duration of the token validity and can be viewed by running the
Get-ADFSProperties cmdlet on one of the AD FS servers in the farm. This token lifetime has nothing to do with the
lifetime of tokens issued by Office 365 once a user logs on; it’s possible for a user to have both an expired AD FS
token and a valid Office 365 token at the same time.

Once a connection is authenticated, the same token can be reused to login to Office 365 services within the specified
timeframe. However, each service or application can impose different timings. Unfortunately, the documentation on
what the exact timings are per application is non-existent. As such, you might find that closing and re-opening Outlook
will trigger the authentication process entirely again whilst doing the same with OWA will allow you to continue to
access other Office 365 workloads using the same token you received previously. For an end user, the difference might
not be visible at all. Even if the token is still valid, the authentication platform in Office 365 will still redirect the
client’s request to the organization’s federation endpoint where – instead of re-authenticating the request – the
existing token is simply re-used when redirecting the user back to Office 365.

Restricting access to Office 365 through AD FS

By default, when AD FS is configured for Office 365, all users in the environment can receive an AD FS token for
Office 365. Although a user can authenticate, it doesn't necessarily mean that each user will have access to Office
365-based services; if you don't assign a license to a user in Office 365, they won't be able to use any services. One
symptom of this is when users log in to the Office 365 portal and see none of the service icons, just the interface for
changing their individual account settings.

Sometimes an organization might want to limit access to Office 365 using a variety of conditions, like a user's group
membership or possibly the location from where a user is trying to authenticate inside the corporate network or
outside (from the Internet). To allow such granularity, AD FS can be configured with something called custom claim
rules , sometimes also referred to as Client Access Policies. More specifically, these claim rules are configured within
the issuance authorization rule set which controls what accounts can receive a token for the relying party, that being
Office 365. Without going into the very specifics of how claim rules work, you can regard them as condition-based
access policies: if a specific condition is met, a user is either allowed or denied access. Claim rules can be created
manually by writing the code for the claim rule using the appropriate claim rule language, or they can be constructed
using the built-in wizard in the AD FS Management Console.

A claim rule can contain one or more conditions. A condition consists of an incoming claim and a value that can be
used to validate that claim. For instance: email address, User Principal Name, group membership, and so on. The
administrator can then choose to allow or deny access if a value matches.

Restricting access based on the location of a client is more difficult. In this scenario, an AD FS Proxy (or Web
Application Proxy) is required. Even when AD FS Proxies are deployed, users on the internal network usually connect
directly to the AD FS servers while external users (not connected to the corporate network) connect through the AD
FS Proxy servers. Because of the difference in the connection flow, external users can now be differentiated from
internal users, for example using a new, custom, claim called X-MS-Proxy . For this to work, however, some more
configuration on the AD FS servers may be required. Older versions of AD FS require that an administrator configure
the additional claim X-MS-Proxy so that AD FS is aware that such a claim exists. Next, the existing relying party trust
with Office 365 must be updated with a new rule which will deny (or allow) a token to be created if the claim is found
in the token request.
Using the X-MS-Proxy claim is not the only way to block external traffic. Although using the claim seems perfectly
valid, it does not work in scenarios where the active authentication flow is being used, such as when Outlook connects
to Exchange Online (without Modern Authentication). Even though a user might be residing inside the corporate
network, the connection to AD FS is made by Microsoft's authentication platform, which is outside the organization's
network. In this case, another claim could be configured: X-MS-Forwarded-Client-IP . Using a regex expression which
matches the corporate's internal IP address ranges to the IP address returned by the claim, a user can be denied
access if the claim does not match. Keep in mind that when users connect through a VPN, if the IP address they’re
issued is on the corporate network, they’ll still be able to log in to services where the claims rule grants access based
on being on the corporate network.

Different claim rules and conditions can be used to create a set of Authorization rules that allow or deny access. To
help achieve this purpose, a Microsoft article describes how to prevent access to Office 365 for external connections.

AD FS and Modern Authentication. Some limitations still apply when using Modern Authentication. For instance,
if you want to restrict access to Office 365 based on the client’s location, or you need to restrict access to a certain
type of applications, it is worth reading through your options here .

AD FS and modern authentication

As described earlier, Modern Authentication changes the way clients authenticate with Office 365, and how they
communicate with federation servers. When you start using Modern Authentication, you might notice that you aren't
always getting a full SSO experience. Instead, clients are presented with the forms-based authentication window, even
when using a domain-joined computer from within the corporate network.

Although the Office client will first attempt to authenticate using Windows Authentication, it attempts to do so on the
WS-Trust 1.3 endpoint of the AD FS server farm, which is not enabled by default. Because that attempt inevitably
fails, Office automatically fails back to forms-based authentication. This annoyance can be avoided easily, by enabling
the WS-Trust 1.3 endpoint on the federation server farm. The quickest way to do this, is to run the following
PowerShell command on one of the federation servers. Note that before the endpoint is available, the AD FS service
on all federation servers in the farm must be restarted.

[PS] C:\> Enable-AdfsEndpoint -TargetAddressPath "/adfs/services/trust/13/windowstransport"

WARNING: PS0038: This action requires a restart of the AD FS Windows Service. If you have deployed a federation server farm, restart
the service on every server in the farm.

AD FS auditing
By default, older AD FS versions don’t keep track of successful and failed authentication attempts. Yet, this
information can be extremely useful for a variety of reasons. If not for general reporting purposes, it allows you to
keep track of failed authentication and, thus, detect potentially suspicious activity. Most reporting solutions for AD FS
plug in on top of the built-in auditing capabilities. As a best practice, enabling auditing from the start gives you a
history you can work with. To enable auditing, you must first configure the AD FS farm to log successful and failed
authentication attempts. The easiest way is to use PowerShell to apply the change to one of the AD FS servers in the
farm:

[PS] C:\> Set-AdfsProperties -LogLevel $((Get-AdfsProperties).LogLevel

+ "SuccessAudits" + "FailureAudits")

To confirm that the additional events are logged, use the following command and verify that both SuccessAudits and
FailureAudits are shown:

[PS] C:\> Get-ADFSProperties | Select –ExpandProperty LogLevel

Errors

FailureAudits

Information

Verbose

SuccessAudits

Warnings

Next, edit the local security policy on the AD FS servers to allow auditing. This can either be done locally on each of
the federation servers, or through Group Policy. Using the Group Policy Management Console, or local Policy editor,
navigate to the following setting: Computer Configuration > Windows Settings > Security Settings > Local
Policies > Audit Policy . There, change the Audit object access to include Success and Failure . Alternatively,
you can also modify the local security policy on each federation server using the following command. Note that,
depending on your Group Policy configuration, these settings might be overwritten at the next Group Policy refresh:

[PS] C:\> auditpol.exe /set /subcategory:"Application Generated" /failure:enable /success:enable

The command was successfully executed.

Once auditing has been enabled, the federation servers will start logging additional information in the Security event
log of the federation servers. Each authentication attempt (successful or failed) is accompanied by several events. For
successful authentications, Events with ID 299 and 500 are most interesting. The most useful information for failed
authentications is provided in events with ID 4625.

Troubleshooting AD FS authentication
One of Murphy's laws says that when something can go wrong, it will. Although one can only hope it never happens,
the chances are that sooner or later you are faced with a problem in the AD FS infrastructure. Depending on the
problem, the solution might be simple and straightforward. For instance, a server goes down because of a hardware
issue, or because an underlying dependency failed.

However, when the federation servers are working properly, but the actual authentication fails, troubleshooting is a
lot tougher, especially when you’re using Office 365. This is mainly because you do not have control over the entire
authentication process. Large pieces of the puzzle are hosted and managed by Microsoft, and you have no visibility
into their infrastructure. As such, it is harder to identify the issue, or to know whether the issues is due to a problem
at Microsoft's end, or your federation servers.

In order to troubleshoot some of these issues , you might need to trace an authentication attempt in order to figure
out what exactly is happening to the request. For this, you will need to enable AD FS Debug Logging first. To enable
debug logging, open the Event Viewer on a federation server, click View and select Show Analytic and Debug logs .
In the crimson channel, under the Applications and Services Logs a new folder called AD FS Tracing will appear.
In that folder, there is a Debug event log. By default, this log is disabled. To start logging, right click the log and
select Enable Log . Once the log is enabled, the federation server will start spawning a ton, mostly informational,
events for each authentication. Each of these events contain a piece of the authentication puzzle and might reveal
information such as the values of (some of) the claims passed through claims pipeline.

Real World. Because of the multitude of events that are generated for each authentication attempt it is quite
difficult to filter out relevant information from the noise. If you have multiple federation servers, it is easier to
(temporarily) single out a federation server for troubleshooting purposes. That way, you can force only specific
authentications to go through the server you have enabled tracing on. Another way to quickly search for information
in the AD FS Tracing log is to use the Activity ID. The is a unique ID which identifies the authentication attempt and
by extension all events associated with it. Typically, the Activity ID is displayed on screen when the failure happens.

Password hash synchronization as a backup

Organizations sometime substitute AD FS with password hash synchronization. Although many organizations have
deployed AD FS because it was the only way to achieve true single sign-on, the simplicity of password hash
synchronization makes it a suitable alternative for many organizations (not to mention that PTA now exists and is
much easier to deploy than AD FS).

Today, password hash sync is often deployed alongside AD FS as a backup mechanism in case the on-premises AD FS
infrastructure has an extended outage. The ability to switch from AD FS to Password Synchronization is not automatic
and can take up to two hours to complete. During that time, and depending on the size of your organization, some
users might be able to authenticate, others might not. During all my testing, I've seen the process complete in under a
few minutes. However, those test environments were all limited in size. Your mileage may vary!

When AD FS is still available, switching from AD FS to password synchronization is done by issuing the Convert-
MsolDomainToStandard cmdlet while connected to Azure AD. This will update both Azure Active Directory and the AD
FS server farm. However, in case of an AD FS outage the on-premises AD FS server farm cannot be updated, and you
should use the Set-MsolDomainAuthentication cmdlet instead. This will only update Azure AD and removes the need
for AD FS to be available. When performing the switch, you do not explicitly configure password synchronization;
instead you are (temporarily) "un-federating" your domain so that it uses Microsoft's authentication system again. If
password synchronization was setup previously, it is used automatically after the next synchronization by AAD
Connect. If you have not yet setup Password Synchronization, all your users will be assigned a new password in Office
365.

Real World. As a best practice, it is recommended to configure password synchronization, even if you are using AD
FS for SSO. Imagine having to distribute new passwords to users in a large environment when they don’t have access
to email. Wouldn’t it be much easier if they could continue to use their existing password instead?
While you can depend on different tools, it is better to make sure that your AD FS infrastructure is solid and highly
available. This ensures that users have always access to Office 365-based services, even when a single AD FS server is
experiencing issues. For instance, Microsoft Azure can be used to extend the on-premises AD FS deployment. Instead
of using a second datacenter, the on-premises network can be extended into Azure where additional AD FS servers are
deployed. This hybrid approach is ideal for organizations who do not have access to a second datacenter but need site
resiliency.

AD FS and password expiry notifications in Office 365

When passwords are set to expire, users in an on-premises environment typically receive notifications telling them
their password is about to expire or to prompt them to change the password when it expires. The same is true for
users in Office 365 that have a cloud-only account. Federated users in Office 365, however, do not get a notification in
Office 365. This can be a problem for remote users that are never in the office, or do not use a corporate device. This
is because Office 365, by default, does not know when the on-premises password is about to expire.

Although AD FS supports providing information about password expiration to Office 365, it is not enabled by default,
and you must configure an additional Issuance Transformation rule so that the information is sent to Office 365. Using
the Claims Rule wizard, add the following Issuance Transformation rule to the Office 365 Relying Party Trust:

c1:[Type == “http://schemas.microsoft.com/ws/2012/01/passwordexpirationtime“]

=> issue(store = “_PasswordExpiryStore”, types = (“http://schemas.microsoft.com/ws/2012/01/passwordexpirationtime“,

“http://schemas.microsoft.com/ws/2012/01/passwordexpirationdays“, “http://schemas.microsoft.com/ws/2012/01/passwordchangeurl“), query
= “{0};”, param = c1.Value);

The newly-created claim will ensure Office 365 gets the following additional information:

The exact time when the password expires

The number of days remaining before the password expires
The endpoint (URL) where the password can be changed

Before the latter can work, you must first enable AD FS to allow password changes. This is explained in the next
section.

Enabling password updates through AD FS

AD FS in Windows Server 2012 R2 and later versions support updating expired passwords, but by default the feature
is disabled and only works for devices registered with AD FS through workplace-join. The ability to update an expired
password is not the same as a password reset. The latter is something that the user can do at any given time, while
updating an expired password only happens when it is expired. Resetting passwords is typically done by an
administrator or the users themselves through a password reset portal like the Azure Self-Service Password Reset
portal. If you have configured password writeback, the password can also be changed (reset) in the on-premises
directory. To enable the ability to update an expired password, you must complete the following two steps. If you do
not want unregistered devices to have to option to update passwords, you can skip the first step.

To enable the feature, you must enable the AD FS endpoint used for password updates. Do this by running the
following PowerShell command on one of the federation servers. Afterwards, you should restart the AD FS Windows
services on each of the federation servers in the farm.

[PS] C:\> Enable-AdfsEndpoint "/adfs/portal/updatepassword/"

WARNING: PS0038: This action requires a restart of the AD FS Windows Service. If you have deployed a federation server farm, restart
the service on every server in the farm.

If you have deployed Web Application Proxies, you should also run the following command before restarting the AD FS
windows service on each federation server. Once the feature is successfully enabled, users will be prompted to change
their password through the AD FS portal.

[PS] C:\> Set-AdfsEndpoint "/adfs/portal/updatepassword/" –Proxy $True

Enabling "Persistent SSO"

Those who have used Office 365 without AD FS must have noticed the Keep me signed in checkbox on the Office 365
portal login page. By default, this functionality called "Persistent SSO", is disabled. To make the option appear on your
AD FS forms-based login page, run the following PowerShell command from the AD FS server:

[PS] C:\> Set-AdfsProperties -EnableKmsi:$true

When enabled, and upon the first authentication, the client receives a cookie which is valid for 24 hours. This cookie
enables the client to login across multiple browsers and sessions for as long as the cookie is valid. After 24 hours, the
user must re-authenticate. Without the feature enabled, the authentication cookie is only valid for the active session.
Connecting to LinkedIn
Nearly 600 million people have a LinkedIn account. Ever since Microsoft bought LinkedIn in June 2016, the two
companies have looked for ways to co-operate, including the ability to connect LinkedIn to Azure Active Directory.
This initiative was announced at the Ignite conference in September 2017, but the need to work through some privacy
concerns and to ensure that any connection protected personal data pushed general availability out to September
2018.

By default, all Office 365 tenants except those in France and Germany have the LinkedIn connection enabled. Tenants
of the sovereign clouds do not support the LinkedIn connection, so you can’t use the feature if your tenant is in the
Black Forest, China, or U.S. Government clouds. Enabling the connection only allows users to decide if they want to
connect their Office 365 account to their LinkedIn account. No data is ever exchanged unless authorized by the user
to whom the data belongs.

To control the connection, go to the User Settings section of the Azure Active Directory portal. You can then set the
connection to allow all users to connect, some users to connect, or no users to connect. If you decide to restrict the
connection to a selected set of users, you must enter each account into the portal individually as you cannot input a
group and have the portal add all the members.

If the tenant allows a user to access LinkedIn, a LinkedIn logo appears in the people cards displayed by browser
applications such as OWA and SharePoint Online. Before any data can be fetched from LinkedIn, the user must
connect to their LinkedIn account to approve sharing the data with Office 365. This is done the first time a user tries
to use LinkedIn from Office 365. After a connection is set up, clicking the LinkedIn logo in a people card causes Office
365 to try to fetch data about the person from LinkedIn using information like their full name (display name) and
email address to find a match. The search can happen for both internal and external recipients and result in three
outcomes:

The person is a LinkedIn contact: You see the private profile for the person as shared with their LinkedIn
contacts (Figure 3-15 ).
The person has a LinkedIn account but is not a contact: You see the public profile of the person and can send
them a LinkedIn invitation to connect.
LinkedIn can’t find a match: Either the person doesn’t have a LinkedIn account or several matching LinkedIn
accounts are found. In this case, you can choose the best match.

Figure 3-15: Viewing LinkedIn profile information for a contact

The important thing to remember about the LinkedIn connection is that users control it. Although the tenant decides
if the connection can be used, users decide if they want to connect their Office 365 account with their LinkedIn
account.

Managing a Tenant
Of course, knowing how to deal with identities, authentication, and directory synchronization is all very well, but it is
worth absolutely nothing if we manage these identities within Office 365. We now progress to consider how to manage
an Office 365 tenant.
Chapter 4: Managing Office 365
Paul Robichaux

There are many ways to manage Office 365 tenants. As the Office 365 suite has grown, different products have
adopted different workflows. Depending on the task you're trying to perform, the functionality of the tools Microsoft
provides may lead you towards one approach or another. Sometimes there's more than one way to perform the same
task; in those cases, each approach will have its pros and cons. For example, the web-based admin interfaces are
simple to use and make it easy to perform one-time tasks such as adding a domain name to the Office 365 tenant.
However, for bulk administrative tasks such as changing the department attribute for multiple users, PowerShell is
more efficient.

Understanding how to build and execute PowerShell commands is an essential skill for IT professionals working with
Office 365. If you have not already embraced PowerShell, then the move to Office 365 is an opportunity for you to do
so. Even a simple administrative change applied to a single user is faster using PowerShell given the time needed to
open a web browser, login to the correct administrative portal, navigate to the right place to select the user record
with a series of mouse clicks, and make the change. Many of the more complex tasks you'll need to perform in Office
365, such as bulk changes to user licenses, office addresses, or department names, can quickly be performed with a
line or two of PowerShell but would need a great deal of point-and-click drudgery if done through the web portal.

It's important to remember that in a hybrid environment, changes to users, groups and other objects that synchronize
from on-premises Active Directory to the cloud must be made in the on-premises Active Directory, which is the source
of authority. For example, to change a user’s surname you must make the change using the on-premises Active
Directory management tools and then allow directory synchronization to synchronize the change into Azure Active
Directory. The Office 365 portal, and PowerShell, will both warn you when you try to change an attribute in the cloud
that is synchronized from the on-premises AD. More detail about directory synchronization is Chapter 3.

Because there are so many individual services in Office 365 (including, to name a few, Azure Active Directory,
Exchange Online, SharePoint Online, and Teams), Microsoft hasn't really tried to unify administration into a single
portal or a single PowerShell module, although they are making some moves to do so in the Microsoft 365 product. It
wouldn't make sense to do so because the workloads are so different; a single unified toolset would be complex, slow,
and confusing to use, and would limit the ability of the product groups that develop each individual service to optimize
and improve their own admin experiences.

Part of learning to manage Office 365 is becoming comfortable with a sometimes-confusing variety of web-based
admin portals, PowerShell modules, and endpoints that we can use to administer Office 365. In the next section, we'll
look at how to connect to each Office 365 service for management tasks.

Cloud versus on-premises management

Any seasoned professional experienced at managing on-premises infrastructures should already be aware that some
level of control and visibility is lost when a company migrates workload to the cloud. Office 365 is sold as “Software as
a Service” (SaaS), which transfers the responsibility for running the service to Microsoft. You give up access to
anything relating to underlying hardware, networking, and software. You don’t get to select the make and model of
servers used to run the service, nor do you get to choose the specifications and sizing of those servers.

Storage for Office 365 applications is managed by Microsoft, both in terms of quotas assigned to individual
applications and users and overall storage performance. At most you may need to help your end users with managing
their mailboxes and sites within the quota limits that Office 365 imposes, through a combination of user education and
policy controls (e.g. retention policies). Microsoft also manages the network and Internet connectivity for the service.
You only need to ensure that adequate bandwidth is provisioned for your organization’s Internet connectivity to
reliably connect to Office 365. Similarly, you must manage any firewalls or other network devices such as web proxies
that affect connectivity of your users to Office 365.

The underlying operating system and all the software running on Office 365 servers is also out of your hands.
Microsoft surfaces a lot of configuration options for you that apply to the users in your own tenant, as well as some
reporting tools such as message tracing. However, you simply do not get access to Windows event log data,
performance log data, or any of the other log types that Windows, and the applications generate. Similarly, you do not
get access to stop or restart services on the servers or perform server reboots. On the positive side, you also don’t
need to perform any Windows patching or upgrades for Exchange, SharePoint, and other applications.

As you can see a lot is taken out of your hands when you migrate to Office 365. Some IT professionals view this as a
loss of “control” and are fearful of the impact that is will have on their careers. Another way to look at it is that you
are allowing the same people who develop the product to run it for you within a defined set of parameters. All the
time that you previously spent monitoring, managing and fixing server hardware, operating systems, and application
installations is now available for you to spend on helping deploy Office 365 features to your end users, align the
configuration of Office 365 with your organization’s policies, and act as a broker between your company and
Microsoft as a service provider. On the bright side, when something goes wrong it’s now Microsoft being woken at
3am by a pager instead of you.
Office 365 Administrative Interfaces
The Office 365 admin center , also sometimes referred to as the Office 365 portal (Figure 4-1 ), is used to configure
items shared across the Office 365 applications, such as domain names, billing details, and license assignments. Some
frequently-used administrative functions are also in the Office 365 admin center. For example, you can create a new
cloud-only user account complete with a mailbox without opening the Exchange admin center. Users with Global
administrator, Service administrator, Billing administrator, or application-specific administrator rights will have access
to the Admin center. In May 2018, Microsoft rebranded the Office 365 admin center to be the Microsoft 365 admin
center to reflect the growing closeness between Office 365 and the wider Microsoft 365 suite. The functionality of the
Office 365 elements doesn’t change, but Microsoft 365 elements now share the common portal if you license that
suite.

Figure 4-1: The Office 365 Administration Center

You can customize the home page of the admin Office 365 Admin Center to display specific tiles and the order and
layout for the tiles (which Microsoft calls “cards”). Click on the Customize your home link at the top of the page.
The default set of cards you see include cards for installing Office 365 Professional Plus software, reviewing your
billing, and working with support incidents. One customization that is useful for tenants that synchronize Azure Active
Directory with an on-premises Active Directory is to add a card showing the directory sync status to your home page.
Clicking the card reveals other information about your directory sync status, including the version of the directory
synchronization client (Azure AD Connect) installed in the on-premises environment (Figure 4-2 ).
Figure 4-2: Directory synchronization status available in the Office 365 admin center

The navigation pane on the left side of the Office 365 portal has expandable menus for different categories of tasks.
The tasks that are available throughout the menu are those that Microsoft thinks will be most commonly performed by
Office 365 administrators. In some cases, a complex administrative task is provided with a user-friendly wizard in the
Office 365 portal, while the workload-specific admin portal for that task will have a more complicated process
involved. In other cases, admin tasks initiated from the Office 365 portal simply launch a wizard from one of the
workload-specific admin portals. Administering Office 365 will require you to use a mix of the native Office 365 admin
center functionality and the workload-specific admin centers, and Microsoft frequently adds new functionality in the
Office 365 admin center, and otherwise rearranges things, to keep us all on our toes.

An unprivileged user who tries to log into the Office 365 Admin center will just get an error message. It used to be
that going to https://portal.office365.com would work for any user, allowing them only to see the navigation pane
sections and items that their permissions allowed them to, but Microsoft has changed the service so that URL
redirects users to a summary dashboard showing their available applications and a selection of recently-used
documents. Now an ordinary end user who logs in, for example, will see the Office 365 applications and services they
have access to, but they won’t see the “Admin” application icon that a a user with Global Administrator rights would.
Anything that a user doesn’t have permissions to view or perform changes in will be hidden from them.

In the Users section, you can manage user account, contacts, guest users, and deleted users, and initiate data
migration processes such as uploading PST files or importing email from other platforms.
The Groups section allows you to create and manage Office 365 Groups and Exchange distribution lists, as well
as shared mailboxes. See Chapter 11 for more detail about Office 365 Groups.
The Resources section is where you can manage room and equipment mailboxes and create new SharePoint
Online sites. There is also an external link to create a public website via one of Microsoft's web hosting partners.
Microsoft used to provide public website capabilities in SharePoint Online, but has deprecated that feature and is
in the process of winding down existing public sites for customers.
In the Billing section you can manage subscriptions, buy and allocate licenses, and update payment details. This
is also where added services are bought separately or as bundled products. For example, if your organization
uses Office 365 Enterprise E3 you can buy Office 365 Cloud App Security Management (CAS) as a standalone
license or buy the full Office 365 Enterprise E5 license which includes CAS as well as other features.
In the Support section, you can also submit and monitor the progress of service requests for problems that you
have encountered with your tenant. Any open Customer Lockbox requests will appear in this section.
The Settings section contains several tools for controlling the configuration of Office 365 services. For example,
you can enable or disable individual services such as Cortana or Microsoft Teams, or apply restrictions to
services such as preventing external access to Office 365 Groups or Teams. Security and privacy settings for the
organization are also in this section, including the password policy for cloud identities, Customer Lockbox
configuration, and sharing controls. The Settings section also contains domain setup, which is also located in the
Setup section of the admin portal. The organization profile is also configured in this section, which includes
company and contact details, and the options for configuring Targeted Release (previously known as First
Release) for advanced deployment of new features.
In the Setup section, you can review your purchased licenses and assign licenses in a similar fashion to the
Billing section. Domains can be configured in this section as well, using the exact same interface as the Settings
section. The Setup section also includes a launchpad for data migration processes (the “Migrate user mailboxes”
hyperlink), which are the same operations that can be started from the Users section of the portal.
The Reports section presents a dashboard view of the usage of individual Office 365 services such as Exchange
Online, OneDrive, and Skype for Business. A series of detailed reports are also available here. The Reports
section also gives access to security & compliance reports such as mailbox auditing, email spam and malware
protection, and data loss prevention policy matches. More information about reporting is in Chapter 21.
The Health section is where you can access the Service Health Dashboard (SHD) to see the current health status
of the Office 365 workloads in addition to details of outages, service history for the previous 30 days, and
planned maintenance. The Health section also includes access to Message Center, where notifications for new
and changed features in your tenant are published by Microsoft, and a summary of your directory
synchronization status identical to the one available in the dirsync status card. The SHD and Message Center are
 discussed later in this chapter.
The Admin centers section has shortcuts to other administration consoles for services that you are licensed to
use, such as Exchange Online, Teams and Skype, SharePoint Online, and OneDrive for Business. Some of these
consoles are nearly identical to their on-premises forebears, while others have been changed to reflect the
specific features available in the service.

The management interfaces for Exchange Online and SharePoint Online are dealt with in Chapters 5 and 8. Let's take
a closer look at the other workload-specific admin portals and PowerShell interfaces.

Managing Skype for Business Online and Microsoft Teams

The Skype for Business Online Admin Center has historically been where you manage your Skype users, external
communications settings, voice services, and online meetings. When Microsoft introduced Teams, they kicked off the
process to integrate Skype for Business and Teams, meaning that administrators had to separately manage settings
for two related products that have very little infrastructure in common. Initially you could use the Skype for Business
portal to manage Skype users and settings and PowerShell to manage Teams users and settings. In mid-2018
Microsoft began rolling out a new combined portal to manage Teams and Skype for Business together. Most of the
administrative organization-wide and tenant-specific settings are now available in the new portal . You can also
navigate there by going to the Services & add-ins section under the Settings section of the left-hand navigation bar of
the Office 365 Admin Center, then clicking Microsoft Teams , then clicking the infobar link to the new portal. More
information about using the Teams and Skype for Business Online Admin Center is in Chapter 13, with Chapter 16
covering policies for Teams meetings.

You can also manage Skype for Business Online or Teams using PowerShell by installing the appropriate PowerShell
module. Note that the cmdlets to set Teams administrative settings are in the Skype for Business Online PowerShell
module instead of the Teams module.

Installing PowerShell for Skype for Business Online

After you download Skype for Business Online PowerShell Module , t o connect to Skype for Business Online, run the
following PowerShell commands:

[PS] C:\> $credential = Get-Credential

[PS] C:\> $skypesession = New-CsOnlineSession -Credential $credential

[PS] C:\> Import-PSSession $skypesession

If your admin account uses multi-factor authentication you can use the -Username parameter instead of passing a
credential object so that the ADAL (modern authentication) prompt will appear.

[PS] C:\> New-CSOnlineSession -Username admin@eoffice365itpros.onmicrosoft.com

Skype for Business Online cmdlets are prefixed with Cs or CsOnline , for example Get-CsOnlineUser . You can see the
full list of available cmdlets by running the following command.

[PS] C:\> Get-Command -Noun Cs*

Some non-Skype cmdlets will also be returned, such as Export-CSV , but you will be able to differentiate them by
looking at the Source column of the output.

Installing PowerShell for Microsoft Teams

The Teams and Skype for Business Online Admin Center is the administrative portal for Teams. To manage Teams with
PowerShell, there are several sets of PowerShell cmdlets you should be familiar with:

The cmdlets in the Skype for Business Online PowerShell module to manage messaging and meeting policies (e.g.
Get/Set-CsTeamsInteropPolicy ).
The Teams PowerShell module is still in preview, but it is now capable of managing most elements of teams.
If you need to manage the properties of the Office 365 Groups used by Teams, use the -UnifiedGroup cmdlets
included in the Exchange Online module.
If you need to manage the properties of the Azure Active Directory group, you can use the cmdlets in the Azure
AD module.

The current version of the Teams PowerShell module is available in the PowerShell Gallery. You can install the module
with the following command.

[PS] C:\> Install-Module -Name MicrosoftTeams

To connect to Microsoft Teams in your tenant, run the following command.

[PS] C:\> Connect-MicrosoftTeams

To see a list of available cmdlets, run the following command.

[PS] C:\> Get-Command -Module MicrosoftTeams

Further examples of using the PowerShell module for Teams are available in Chapter 14.

Managing the Security & Compliance Center

The Office 365 Security & Compliance Center (Figure 4-3 ) brings together compliance features from across the
entire service with a specific focus on those that apply to multiple workloads rather than being specific to an
application. The basis for the initiative is solid because when the need for compliance arises, the focus is usually on
the data that organizations need to discover, recover, or retain instead of the application that manages the data. The
URL is https://protection.office.com . The Security & Compliance Center is under active development to allow it to
deliver equivalent or better functionality that works across all Office 365 workloads to what is available for individual
workloads today. For this reason, what we show or describe on these pages might differ to what you see inside your
tenant.

Like the Office 365 Admin Center, you can customize the appearance of the Security & Compliance Center by moving
the widgets around to suit the needs of your tenant; the Customize button at the top of the page will allow you to
make changes.

Figure 4-3: The Security & Compliance Center

The following sections form the Security & Compliance Center; all are accessible through the left navigation bar:

Alerts - Manage alert policies and view security alerts for the tenant. Alert policies allow you to set conditions
that match activity for which you would like to be alerted. There are dozens of alert conditions available covering
a wide range of possible security concerns such as malware detection, file and folder sharing, DLP policy
matches, permissions changes, and many more. The Alerts section also includes a link to the Office 365 Cloud
App Security (CAS) console. Chapter 21 covers activity alerts, alert policies, and CAS in more detail.
Permissions - Assign permissions to users to allow them access to compliance functionality and data. Although
the permissions look like the Exchange RBAC management groups and roles, they only apply within the Security
& Compliance Center. Users who hold the Global administrator right will have access to the Security &
Compliance Center. You can use the Permissions tab to grant selective access to other users.
Classifications - Manage the labels that will made visible to end users in their applications so that they can
apply labels to their content to control retention or deletion of the content. You can also manage label policies to
automatically assign labels to content. Chapter 19 covers Office 365 labels and the policies used to distribute
labels to Office 365 workloads.
Data loss prevention - Manage DLP policies for content stored or transmitted through Exchange Online,
OneDrive and SharePoint (Chapter 22). You can also manage device security policies for Office 365 MDM, as
covered in Chapter 18.
Data governance - Manage the import of data to Office 365 data from PST files, on-premises SharePoint
servers, file shares, and external services such as social media sites. You can also enable and disable archive
mailboxes for Exchange Online, create retention policies to manage the preservation and deletion of content, and
configure supervision policies to check email that may need manual review to ensure compliance with policies.
Chapter 19 covers data governance in more detail.
Threat management - Manage a range of security policies that apply to Office 365, including email security
policies for anti-spam, anti-malware, outbound spam, DKIM, safe attachments, and safe links. See Chapter 17 for
a description of how these features work.
Mail flow – Allows you to see a dashboard view summarizing mail flow for the organization, including queue
 lengths, the percentages of traffic using Transport Layer Security (TLS) for SMTP, and so on, or to perform
message traces. This is merely another entry point to the mail flow dashboard described in Chapter 17.
Data privacy – for now, this section only has a link to the GDPR dashboard, a landing point for features to help
Office 365 tenants deal with the requirements of the European Union’s General Data Protection Regulations
(GDPR). For example, you can create and process Data Subject Requests (see Chapter 20) and view analytics
about the use of labels (Chapter 19) through this dashboard.
Search and Investigation - Perform content searches to search for information across multiple Office 365
sources, audit log searches, and manage eDiscovery cases. Chapter 20 covers content searches and eDiscovery
cases
Reports - Create, view, and download reports linked to security and compliance topics in a dashboard
customized for the tenant. The exact set of reports revealed in the dashboard depends on the functionality
licensed by the tenant and can include, DLP, Advanced Threat Protection (ATP), and anti-malware reports. You
can schedule reports for Office 365 to run and distribute via email to whoever you need to get the information.
Chapter 21 covers Reports.
Service Assurance - Access to information about how well your Office 365 tenant meets various aspects of
regulatory requirements. Included are third-party independent reports describing how Office 365 meets
regulatory and security standards for specific industries and geographic regions (such as ISO reports). Microsoft
also publishes white papers to help tenants meet compliance and trust requirements and give details about how
audits of Office 365 prove its compliance with global information security standards and regulations (such as ISO
27001:2013), including the date of the last audit.

Surprise alerts are surprising: Sometimes the alerts you receive due to alerting policies in Office 365 can surprise
you. One example: Microsoft recently made a change to the back-end services that automatically grants a previously-
undocumented service account ( BOXServiceAccount@yourTenant.prod.outlook.com ) the View-Only Organization
Management right. It’s good to receive these alerts (and review them, ideally using a good set of filters) because you
want to know when an unexpected role grant has been made, but Microsoft really should do a better job of notifying
administrators of plans for such changes instead of documenting them after the fact.

Some features accessed through the Security & Compliance Center are only available if the tenant buys the necessary
licenses. For example, Advanced eDiscovery (accessed through Search and Investigation ) is only available if you
have an E5 license or have bought the necessary Office 365 Advanced Compliance add-on for the E3 plan. Another
example is that Device management is only available if you subscribe to Microsoft’s Intune service. Finally, if you
have an E5 plan or have licensed the Office 365 Cloud App Security add-on (to connect to Microsoft Cloud App
Security ) , you will also see Manage advanced alerts listed in the Alerts section.

Confidentiality of Service Assurance Reports : Many of the reports available through the Service Assurance
section are Microsoft Confidential Information and covered by the contracted terms and conditions for Online
Services (the current terms and conditions are available online ), which means that customers cannot disclose or
distribute the information to third parties. The intention of providing this information is to allow customer compliance
officers to map their requirements against the results recorded for Office 365 in third party audits and other
assessments against international and industry standards. Microsoft is ahead of other cloud vendor in terms of the
volume and depth of information it releases about Office 365 compliance with standards.

You can manage the Security & Compliance Center with PowerShell without installing any other modules. However,
before you can work with Security & Compliance Center functionality, you must connect to a different endpoint to that
used for Exchange Online or SharePoint Online. This can be a little confusing, especially because many cmdlets with
the same name (but slightly different functionality) are used in Exchange Online and the Security & Compliance
Center. For example, if you execute the Get-RoleGroup cmdlet against Exchange Online, you see the RBAC role groups
defined for Exchange administration. If you execute the version of the Get-RoleGroup cmdlet against the Security &
Compliance Center, you see a completely different set of role groups: the ones defined for RBAC access to security
and compliance features.

I have the following commands in my PowerShell profile to create a session that connects to the Security &
Compliance Center. Any time I need to use PowerShell cmdlets to work with data used by Security & Compliance
Center functionality, I invoke the ConnectCC function.

function ConnectCC

# Helper function to connect to the Office 365 Security & Compliance Center and use their set # of PowerShell cmdlets

$global:O365Cred = Get-Credential

$global:CCSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri

https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $O365Cred -Authentication Basic –
AllowRedirection

Import-PSSession $CCSession –AllowClobber -DisableNameChecking

}

The reason why a different PowerShell endpoint is used is that the features found in the Security & Compliance
Center are designed to work independently across multiple Office 365 workloads. The endpoint connects to a different
service than those that serve either Exchange Online or SharePoint Online. Although some of the cmdlets used with
the Security & Compliance Center are loaded into a session when you connect to Exchange Online, these cmdlets are
usually read-only or might return less information. For this reason, it is best to create a dedicated PowerShell session
to work with the Security & Compliance Center.

The AllowClobber and DisableNameChecking parameters are used because some of the cmdlets imported when you
connect to the Security & Compliance Center might already exist in the PowerShell session, especially if you have
already connected to Exchange Online. AllowClobber permits the import of cmdlets that already exist.
DisableNameChecking suppresses the warning message I’d otherwise see when duplicate cmdlet names are
encountered.

The Security & Compliance Center has a range of cmdlets for different tasks such as managing DLP, preservation
policies, and eDiscovery cases. You can see the full list of cmdlets on TechNet . The Security & Compliance Center is
covered in more detail in Chapters 19 through 22.

Managing Other Office 365 Services

There are many other services in Office 365 besides the “big four”: Exchange, Skype for Business, SharePoint, and
OneDrive for Business. Microsoft has been adding new services at a rapid pace since Office 365 was first introduced.

Yammer is an enterprise social network for enterprises and teams. The Yammer admin portal includes options to
activate your Yammer network, customize the appearance, manage users and policies, and perform community
management tasks. The URL for the Yammer admin portal will vary based on your tenant domain, but you can use the
general admin URL of https://www.yammer.com/office365/admin to access the portal, which then redirects to your
specific tenant URL automatically.

Flow is a cloud-based service to create automated workflows connecting different applications and services. For
example, you can create a flow that triggers when a mailbox receives new mail to automatically save email
attachments to a OneDrive folder. Flow works as both an individual service, allowing users to create personal flows
that improve their productivity, as well as an enterprise application for automating business processes. The Flow
admin center allows Office 365 administrators to configure data loss prevention policies (DLP) for Flow. The DLP
capabilities are in the free pricing tier of Flow, included with Office 365 Business, Education, and Enterprise plans.
The paid plans for Flow have extra management capabilities such as the ability to control which services users can
include in flows and how they can connect on-premises data gateways. The Flow management portal is at
https://admin.flow.microsoft.com/home .

Dynamics 365, the latest version of Microsoft’s venerable customer relationship management (CRM) system that
competes with Salesforce, has its own link in the Admin centers section. However, the link will only take you to an
error page if you click it while logged in to an account that doesn’t have admin permissions on your Dynamics 365
instance.

PowerApps is a cloud-based service to create custom business apps without having to learn a programming language.
The apps are essentially forms connected to a data source that users can run (including on mobile devices) to enable
productivity on the go. You can create apps that connect to data sources such as SharePoint, Dynamics, or Salesforce.
Like Flow, PowerApps has an admin portal that allows customers on the free pricing tier to configure DLP policies. For
paid customers, added admin controls become available for managing user environments, which are a way to separate
apps that need different security levels or audiences. The PowerApps admin portal is at
https://admin.powerapps.com/home .

Integrated Apps : The Integrated Apps setting in the Services & Add-ins section of the Office 365 Admin Center
controls whether users can allow applications to access their Office 365 data, such as their calendar. By default, the
setting is on. You can also change the setting with PowerShell:

[PS] C:\> Set-MsolCompanySettings -UsersPermissionToUserConsentToAppEnabled $False

Because some malware exploits work by gaining access to user data ( here is an example that encrypts user data for
ransom), you might prefer to turn the setting off. It is worth emphasizing again that unless users are aware of threats
and sensitive to granting access to their data, the possibility always exists that they will succumb to an attack if they
are targeted. Microsoft has released two PowerShell scripts to help tenant administrators understand what OAuth
consent grants and app permissions have been given by users. The scripts create a set of CSV files holding details of
access granted by users to applications and their publishers.

Managing Other Microsoft Cloud Services

The Azure management portal administers other Microsoft cloud services for your tenant, or for separate Azure
subscriptions that your organization also has. If you access Azure AD administration from the Office 365 Admin
Center, it takes you to the Azure Active Directory blade in the Azure portal. You can navigate from there to the entry
point for the Azure portal to manage the other Azure services that your tenant users, such as Azure Information
Protection, and manage Intune mobile device and application policies.

The most common operations required to operate Office 365 involve user management: creating new users, modifying
their properties, and managing their permissions. As you learned in Chapter 3, Office 365 supports standalone and
hybrid identities in a variety of configurations. The thing that all identity types have in common is that the Office 365
service relies on Azure Active Directory, so it’s helpful to have some familiarity with the Azure AD management tools.

When you open the Azure Active Directory blade, you’ll see that it has categories for managing users, groups, roles,
devices, Azure AD Connect, and password reset (along with several other categories). It also contains links to controls
for conditional access policies, reports for risky logins and suspicious sign-ins, and troubleshooting tools. A complete
exploration of this section is far outside the scope of this book, but it is well worth your time to explore the settings
here, as there are many features and settings that can improve the security and functionality of your Office 365
environment that you may not be familiar with.

Using PowerShell to Manage Azure Active Directory

Many Office 365 tasks such as user and licensing management, adding and removing domain names, and managing
company information in PowerShell can be performed using the Azure AD PowerShell module (referred to by
Microsoft as Azure AD PowerShell V2), which can be installed using PowerShellGet.

PowerShellGet is part of Windows Management Framework (WPF) 5.0, which is essentially PowerShell 5.0. Windows
10 and Windows Server 2012 R2 or later ship with PowerShell 5.0 installed by default, which means that
PowerShellGet is already available. For earlier operating systems, you'll either need to upgrade to WMF 5.0, or install
PowerShellGet for PowerShell 3.0 or 4.0. Some applications such as Exchange Server are sensitive to changes in the
version of WMF installed on the system, so you should not upgrade WMF until you've verified that all your installed
software will continue to work.

To install the AzureAD module on a workstation, run the Install-Module cmdlet.

[PS] C:\> Install-Module AzureAD -Repository PSGallery

After installing the Azure AD module, connect to your Office 365 tenant by running the Connect-AzureAD cmdlet, and
then entering your admin credentials when prompted. The Azure AD module supports the use of multi-factor
authentication (MFA). To explore the available cmdlets in the Azure AD module, run the following command.

[PS] C:\> Get-Command -Module AzureAD

The Graph API underpins the Azure AD module, so it should come as no surprise that elements of that API influence
how the cmdlets behave. For example, where the Exchange Online cmdlets allow you to find mail-enabled recipients
using distinguished names, display names, email address, or alias, the Azure AD cmdlets use object identifiers
(GUIDs). Fortunately, Exchange stores details of the object identifiers for its recipients, meaning that you can retrieve
and use the identifiers to work with Azure AD. For instance, we can retrieve the membership of a distribution group
from Azure AD as follows:

[PS] C:\> Get-AzureADGroupMember -ObjectId (Get-DistributionGroup -Identity O365BookTeam).ExternalDirectoryObjectId

Another difference that you will probably meet is how the Azure AD cmdlets find or filter objects. The syntax used is
ODATA and the simplest example is using the SearchString parameter to find an object. In this case, we look for any
accounts whose display name starts with the string “Paul”:

[PS] C:\> Get-AzureADUser -SearchString "Paul"

It is good practice to use filters to restrict the amount of data returned to just the set needed. You also need filters to
create dynamic Office 365 Groups because the filter tells Azure AD how to build the group membership. In this case,
we want to find all the users who belong to the sales department. Note that unlike the filters used to find mailboxes,
the equals operator is not prefixed with a hyphen. That’s because this is ODATA syntax. Make sure that you use “eq”
and not “EQ” as all the Boolean operators are case sensitive, including “true” and “false.”

[PS] C:\> Get-AzureADUser -Filter "Department eq 'Sales'"

Another example is a filter to find guest accounts in the tenant. These are accounts created to allow people outside
the tenant to access resources belonging to the tenant, such as Office 365 Groups or Teams:

[PS] C:\> Get-AzureADUser -Filter "UserType eq 'Guest'"

Finally, filters can use several properties:

[PS] C:\> Get-AzureADUser -Filter "Department eq 'Sales' and Country eq 'US' and AccountEnabled eq true"

Azure AD also supports the startswith filter to check a value against the start of properties. Be careful here as the
operator and the property it checks against both need to be in lowercase. In this example, we want to find users who
come from countries like Austria or Australia:

[PS] C:\> Get-AzureADUser -Filter "startswith(country,'Aus')"

Like all scripting and programming languages, it takes time for your muscle memory to absorb the unique quirks of
syntax.
Directory Synchronization Status
The directory sync status for the tenant is available in the output of the Get-AzureADTenantDetail cmdlet. This is
useful information to keep an eye on if you infrequently use the Office 365 admin center. For example, you can run a
scheduled PowerShell script or use a monitoring tool to keep watch on the CompanyLastDirSyncTime property to
ensure that directory synchronization has not stalled.

[PS] C:\> Get-AzureADTenantDetail | Select *DirSync*

CompanyLastDirSyncTime DirSyncEnabled

---------------------- --------------

5/26/2018 8:04:16 PM True

Preview Version
A preview version of the Azure AD PowerShell module is also available, named AzureADPreview . The preview build
usually has some extra cmdlets or functionality that are not quite ready for general usage. The new cmdlets show up
in the production Azure AD module after thorough testing by Microsoft and people who download and use the preview
module. The preview build is a good opportunity for IT pros to begin developing and testing PowerShell scripts for
new capabilities before they arrive in production. In some cases, the AzureADPreview module might be necessary to
perform production administrative tasks.

The MSOnline PowerShell Module

The older MSOnline module supports Windows 7 and later client operating systems and Windows Server 2008 R2 and
later. Although the Azure AD module is now generally available and intended to replace the older MS Online module,
the MSOnline module is still available and supported for use. If you're writing new scripts today, it is best to use
cmdlets from the Azure AD module instead of the old module. If you have an existing library of scripts, some work will
be necessary to redevelop them to use the Azure AD module. Most of the MSOnline module cmdlets have equivalent
cmdlets in the Azure AD module, but the usage is often different, using different parameters or handling of the
returned results. The upshot is that the transition to the new cmdlet set is unfortunately not a simple case of running
a find and replace operation across all your scripts.

You can install the MSOnline module from the PowerShell Gallery. For example:

[PS] C:\> Install-Module -Name MSOnline -Repository PSGallery

To use the MS Online module for an administrative task, connect to your Office 365 tenant by running the Connect-
MsolService cmdlet. The module prompts you for your account credentials. As with the Azure AD module, the latest
version of the MS Online module also supports MFA.

[PS] C:\> Connect-MsolService

To explore the available cmdlets in the MSOnline module run the following command.

[PS] C:\> Get-Command -Module MSOnline

PowerShell Gallery
Many of the PowerShell modules you use to manage different aspects of Office 365 can be downloaded and installed
from the PowerShell Gallery. By default, the gallery is considered an untrusted repository (like any other internet
source), so you will be promoted to authorize an installation. If you want to make the PowerShell Gallery a trusted
resource, connect to PowerShell as an administrator and run the command:

[PS] C:\> Set-PSRepository -Name PSGallery -InstallationPolicy Trusted

This command must be run on each workstation you use to install PowerShell modules. To see a list of installed
modules, use the command:

[PS] C:\> Get-InstalledModule

Connecting to Office 365 Endpoints

With so many different PowerShell management endpoints to connect to, you could easily accumulate a half dozen or
more PowerShell functions in your profile to accommodate them all. Fortunately, you can save some time thanks to
MVP Michel de Rooij who has written a PowerShell script to connect to each service, including:

Azure Active Directory.

Azure Rights Management.
Exchange Online.
Skype for Business Online.
Exchange Online Protection.
 Security & Compliance Center.
SharePoint Online.
Microsoft Teams.

The endpoints are liable to change as Office 365 evolves, so you must be prepared to update the script to stay abreast
of changes and to tailor its functionality to meet your needs. The script supports connecting to the endpoints with
modules that support multi-factor authentication.

Linux, Mac OS, and Office 365 : You can use PowerShell V6 (or later) to connect to Exchange Online and the
Security & Compliance Center from a Linux or Mac OS X workstation. This is possible because these endpoints
support remote sessions. The PowerShell modules for other parts of Office 365, like SharePoint Online or Skype for
Business, need Windows assemblies that do not exist on Linux, so you cannot connect PowerShell to these services.
However, the Azure PowerShell modules, which rely only on PowerShell Core, will run just fine on Mac OS X and
Linux both. Microsoft documents the process and, if you follow it, you’ll find that it works quite well for managing
Azure resources.

Figuring Out What PowerShell Cmdlets to Use

With so many different PowerShell interfaces available for the different Office 365 workloads, it can become confusing
for administrators to learn which cmdlets carry out various tasks. Some choices are easy as components like the
Security & Compliance Center, SharePoint Online, and Skype for Business Online have clear purposes. Some choices
are more complicated. For instance, these three cmdlets all return some of the same information about a mailbox,
such as its display name:

Get-AzureADUser (from the Azure AD PowerShell module).

Get-User (from Exchange Online).
Get-Mailbox (from Exchange Online).

Depending on what task you’re trying to perform, you might also need to use all the above cmdlets plus Get-MsolUser
or Get-CsOnlineUser ! Each cmdlet returns some information that isn’t returned by the others. For example, Get-
AzureADUser and Get-MsolUser both return license information but cannot return the spectrum of mail-related
attributes for a user that Get-Mailbox delivers. However, only Get-MsolUser returns information about the Office 365
MFA status of a user account. There’s currently no single cmdlet that will return every property associated with a
single Office 365 account.

Over time by looking up external resources such as TechNet and various blogs on the internet, and from
experimentation and testing, you will develop an understanding of which cmdlets are best suited for different tasks.
As a rule, use the cmdlets that most related to the nature of the change or information you are working with, for
example:

Use the Azure AD cmdlets (such as Get-AzureADUser ) to manage users in Azure Active Directory, assign
licenses, and manage tenant setup and configuration items that you would normally administer in the Office 365
admin center. However, keep in mind that not all administrative controls have PowerShell cmdlets available.
If you’re working with a user whose identity is managed on-premises and synchronized or federated to Office
365, you’ll need to manage most of that user’s account settings using the on-premises PowerShell cmdlets.
Use the workload-specific cmdlets (such as Get-Mailbox or Get-User for Exchange Online) to perform the tasks
that you would normally perform in those admin portals. However, also keep in mind that some common tasks,
such as creating a shared mailbox, also appear in the Office 365 admin center, even though they specifically
relate to Exchange Online.

Don’t Limit Your PowerShell Output : You might notice that sometimes PowerShell limits the output of values and
uses an ellipsis (…) to show that some added information is available. The $FormatEnumerationLimit preference
variable, which has a default value of 4, controls how many items PowerShell displays when a property has multiple
items. If you want to be sure to see all available items, set $FormatEnumerationLimit to -1 (minus 1). Be aware that
changing this value might affect the appearance of some of your reports.

Checking for a Valid Session

If you write PowerShell scripts to interact with Office 365 applications, you must ensure that connections exist to the
different endpoints needed by the cmdlets called in the script. This code snippet is an example of how to check that
the current PowerShell session is connected to Exchange Online. If no error is detected, the code continues, otherwise
it stops with an error.

[PS] C:\> Try

{$Session = Get-PSSession -InstanceId (Get-OrganizationConfig).RunspaceId.Guid

-ErrorAction Stop }
Catch

{Write-Error "No active Exchange Online session detected, please connect to E XO first"

-ErrorAction Stop }

Other applications have their own PowerShell modules which must be loaded into a session before you can call the
cmdlets. You can check that the module is available before calling any cmdlets. For example, this code checks that the
Teams module is available before running the Get-Team cmdlet.

[PS] C:\> If (Get-Module | ? {$_.Name -Like "*Teams*"}) {Get-Team}

Office 365 administration apps

Microsoft includes an Office 365 tenant management app (Figure 4-4 ) for each of the major mobile operating
systems, as well as a Universal Windows Platform (UWP) version that runs on Windows 10 and the sadly defunct
Windows Mobile. The management apps can perform common administrative tasks from a PC or mobile device
including editing or removing users, resetting passwords, turning email forwarding on or off, and viewing service
health information. The feature set varies between the apps, but the mobile apps can create new users and assign
licenses, allowing an administrator to provision a new user without being near their own PC. However, the
management apps do not support managing any aspect of a hybrid configuration.

Figure 4-4

: Views of the Office 365 Admin App for iOS

The Windows 10 app is a useful way to give an administrative interface to support staff in your organizations that they
can log in to with their Office 365 admin credentials and perform basic tasks, without needing them to use a separate
web browser. In addition, the Windows 10 app can generate notifications to alert support staff or relevant new
information, such as Service Health Dashboard updates, without requiring them to be logged in to the Office 365
admin center.

Administrative roles
Office 365 (and most of its workloads, notably Exchange Online) and Azure AD both offer customers a variety of
administrative roles that you can assign to users who need to perform management tasks. Much like on-premises
environments, there is no single “admin” level permission that who needs to perform some management tasks for your
Office 365 tenant should have. Instead a set of pre-configured administrative roles and groups is available to allow the
assignment of limited but necessary permissions to administrators to do their job. This model is called role-based
access control (RBAC). Keep in mind that the Office 365 and workload-specific roles are different from the Azure and
Azure AD roles that you may wish to assign; for more information on Azure AD role management, see Microsoft’s
documentation .

Role assignment and management are complex; in late November 2018 Microsoft accidentally shipped a bug that
would overwrite any Azure AD role assignments made for a user if you assigned Office 365-specific roles to the same
user. It pays to be thoughtful and to keep good records of the role assignments and definitions you use to ensure that
you always know what roles users should have so you can compare them against what the service says they do have.

Office 365 administrative roles

To assign an admin role for Office 365 to a user, select and edit their account in the Office 365 Admin Center. You can
then assign an administrative role to the user by clicking the Edit link next to Roles . By default, no user account
receives an administrative role, except for the account that Office 365 creates when a company signs up for a new
tenant, which is assigned the Global administrator role.

Billing administrator: Most organizations have a person or team with finance responsibilities that needs to
have visibility of the monthly or annual billing for the company’s Office 365 tenant. Billing administrators can
manage, buy, and cancel licenses and subscriptions, and can manage support tickets and see the service health
dashboard. If you buy an Office 365 tenant through a reseller (Microsoft partner), you cannot assign the Billing
administrator as the reseller manages the role.
Customer Lockbox access approver : The customer lockbox feature (part of Office 365 E5) allows tenants to
control access to certain data (Exchange Online, SharePoint Online, and OneDrive for Business, and some Skype
for Business Online data) by Microsoft support engineers when they need that access to investigate and fix a
problem. Lockbox requests last a maximum of four hours. This role allows non-privileged staff to approve the
access requests made by Microsoft. Customer lockbox access is always logged.
Dynamics 365 service administrator : Users with this role can perform almost any task in the Dynamics 365
admin portal. Dynamics service administrators can perform backups and restores in Dynamics, configure and
manage multiple instances, and create support requests.
Exchange administrator : Users with this role use the Exchange Online admin center to manage mailboxes,
groups, anti-spam policies, and access activity reports in the Office 365 admin portal. An Exchange Administrator
becomes an Organization Management role group member in Exchange Online, which is a high privilege role
group. Exchange Online has a more granular permissions model known as Role Based Access Control (RBAC),
which you can use to assign least privilege permissions instead. RBAC is discussed later in this chapter.
License administrator : Users assigned this role can manage licenses for Office 365 accounts. The purpose of
the role is to avoid the need to have a global administrator be involved in the day-to-day tasks of assigning and
removing licenses.
Message Center reader: Users assigned this role can access information in the Office 365 message center. The
role might be given to someone who handles change management within an organization so that they get a
heads-up about changes due to be deployed to the tenant.
Password administrator : Delegating responsibility for password resets can relieve some of the support burden
from IT teams. Many processes created to reset user passwords involve the need for an authorized person to
check the identity of the user before any change is made to their password, for example by having them walk up
to their desk and show a company identification badge, Once the user is recognized, the authorized person sends
the request to the help desk. Giving that authorized person the rights to reset the password themselves can save
time. However, there is no granularity when it comes to assigning this admin role in the Office 365 admin portal.
Password admins can reset any other user or password admin’s password, but not other types of admin users.
They could reset the password for someone in a completely different location, or someone whose identity is
unverified. As such it is important to be careful to only assign this role to trusted individuals.
Power BI service administrator : Users with this role can perform administration tasks in the Power BI admin
portal, including viewing usage data and setting controls and limits on Power BI usage.
Reports reader: Users with this role can access all the reports available in the Office 365 Admin Center. They
can also use the Power BI Usage Analytics pack for Office 365 and any data fetched through the Microsoft Graph-
based API for tenant-specific or ISV-provided reports.
Service administrator : This role grants the ability to manage service requests and to access the service health
dashboard. You should assign the Service administrator role to users who are members of the Exchange
administrator, Skype for Business administrator, or SharePoint administrator groups, so that they can raise
support tickets for those services. The Service administrator role also gives the holder read-only access to
information about groups, users, and licenses, which means that it is a good role to assign to people who want to
know about license usage within a tenant.
SharePoint administrator : This role grants the ability to manage document storage in SharePoint Online,
which also affects OneDrive for Business, as well as creating and managing site collections, and managing user
profiles. SharePoint administrators can assign other users with administrative permissions within SharePoint
Online for site collections and term stores. They can also access activity reports in the Office 365 admin portal.
Skype for Business administrator : Users with this role can perform almost any task in the Skype for Business
admin center and can also access activity reports in the Office 365 admin center. Skype for Business has its own
workload-specific admin permissions as well, so it is not necessary to assign the Skype for Business
Administrator role if you only want to grant a lower level of administrative privilege. Skype for Business
permissions are discussed later in this chapter.
User management administrator : This role can manage users, groups, passwords, service requests, and see
the service health dashboard. This admin role is ideal for a help desk or low-level support person. Although a
User management admin role holder can remove other user accounts, they cannot remove the account belonging
to a Global administrator nor can they reset passwords for Billing admins, Global admins, or Service admins.

You can assign a user to the Global administrator role or assign them to one or more Customized administrator
roles (Figure 4-5 ). When you assign a user to one of these roles you must give an alternative email address for them
to use when they need to recover a lost password for their admin account. Global admins get administrative access to
all features within Office 365, including individual services such as Exchange Online and Skype for Business.
Naturally, every organization will have at least one Global admin. This account is based in the cloud and will remain
permanently available unless you delete it. You should keep at least one cloud-based Global admin account available
so that you can log in even if AD FS or dirsync is broken for your tenant. Keeping multiple accounts also gives you
redundancy in case the one person with administrator permissions isn’t available when you need them.

The familiar security “principle of least privilege” dictates that you should only grant other administrators whatever
limited admin access is necessary for them to do their job. You can assign the following admin roles through the Office
365 admin portal:
The list above isn’t exhaustive; you can list the full set of known roles and the people who hold these roles with
PowerShell:

[PS] C:\> Get-AzureADDirectoryRole | %{$Role = $_.DisplayName; Get-AzureADDirectoryRoleMember -ObjectId $_.ObjectId} | Format-Table

@{Name="Role"; Expression = {$Role}}, DisplayName, UserPrincipalName -AutoSize

Note that in the results you will see several service accounts used by Office 365 alongside user accounts. For
example, the Office 365 Portal holds the Adhoc License Administrator role while a service like Secure Score will hold
both the Directory Readers and Directory Writers roles.

You can assign Office 365 admin roles to licensed or unlicensed users. An unlicensed user does not have access to any
of the licensed features of Office 365, such as an Exchange Online mailbox or being able to install the Office 365
ProPlus software on their computer. However, they can log in to admin portals and use PowerShell to perform
management tasks. Administrators who perform content searches and want to preview search results need an
Exchange Online mailbox to be able to see the results.

Figure 4-5: Assigning administrative roles to a user account

Note: Office 365 uses the alternative email address for account recovery if the password for the account is lost. In
some cases, the alternative email address is needed when the administrator is unable to access their Office 365
account or mailbox at all. Therefore, it is best to use an email address that is independent of the Office 365 tenant.
This raises some obvious security concerns, because someone could use the alternative email address to reset the
password for the administrator account. You should ensure that the alternative email address is well protected by a
strong password and multi-factor authentication. Services such as Outlook.com or Gmail can offer the necessary level
of security.
Exchange Online administrative roles
Exchange Online has its own administrative roles or management role groups (Figure 4-6 ), some of which link to
Office 365 admin roles and others that are independent. You can see the Exchange Online admin roles by logging in to
the Exchange admin center and navigating to the Permissions section.

The three Office 365 admin roles that inherit admin rights within Exchange Online are:

Global admins
Password admin
Exchange administrator (the workload-specific admin role)

Office 365 Global admins automatically have Organization Management rights in Exchange Online. Users assigned
the Global admin role join in an Exchange Online role group called TenantAdmins_xxxxx, where the last five
characters in the group name are unique to your tenant. The TenantAdmins group is nested in the Organization
Management role group and is displayed as Company Administrators . Organization Management is a powerful
admin role that has access to manage all the features of Exchange Online, so you can see why the Global admin role
should only be assigned to selected administrators in your organization.

Figure 4-6: Exchange Online RBAC role groups

Office 365 Exchange administrators also automatically have Organization Management rights in Exchange Online.
The role group used for this purpose is ExchangeServiceAdmins_xxxxx and is displayed as Exchange Service
Administrator when nested in the Organization Management role group. From an Exchange Online perspective, this
is the same level of admin rights that a Global admin receives, but an Exchange service administrator is not granted
any other admin rights within Office 365 itself, such as managing billing, subscriptions, or domain names.

Office 365 Password admins are automatically granted View-Only Organization Management rights in Exchange
Online. Users assigned the Password admin role join an Exchange Online role group called HelpdeskAdmins_xxxxx .
The HelpdeskAdmins group is nested in the View-Only Organization Management role group and is displayed as
Helpdesk Administrator . This role group can view the configuration and recipient details within Exchange Online
but can’t make any modifications, other than resetting user passwords.

Note: You might notice in the Exchange admin center that a View-Only Organization Management role group
member can create distribution groups and manage distribution groups that they have created. This is due to the
default role assignment policy for users in Exchange Online which permits anyone to create and manage their own
distribution groups.

Aside from the Exchange Online admin roles inherited from Office 365 admin roles, several other Exchange Online
role groups also exist for granularly assigning rights to different users within your organization.
 Compliance Management – members of this role group can manage Data Loss Prevention, Information Rights
Management, and Retention. In addition, members of this role group can view audit logs as well as all
configuration and recipient attributes in the organization. Managing features such as DLP and IRM is often part
of a more general security and information compliance role in an organization, not necessarily a duty performed
solely by Exchange administrators, and this role group allows those users to be assigned the admin rights they
need without having broader Exchange admin rights. Note that a separate set of roles govern access to the
Security & Compliance Center, which is discussed later in this chapter.
Discovery Management - members of this role group can perform discovery searches of mailbox data in
Exchange Online, export search results to PST file, and configure legal hold on mailboxes. This role is well suited
for users in Human Resources, Legal, or Security departments that need to investigate breaches of corporate
policies or perform searches for information that is required in court cases.
Help Desk – members of this role group can view and manage the same individual recipient attributes that the
users can view and manage for themselves. For example, users can login to Outlook Web App and change
personal details such as their phone number and street address, or update their own password. Help Desk role
group members can therefore also change the user’s phone number, street address, or reset their password. This
role group is suitable for low level support staff, or for service accounts that automatically synchronize attributes
such as phone numbers and street addresses from other systems such as a HR database.
Hygiene Management – members of this role group have view-only access to Exchange configuration and
recipients, and they have permission to manage some aspects of the transport system, mostly settings for spam
and anti-malware filtering. You’ll normally assign this role to people who manage your anti-malware appliances
or services, and perhaps to select members of your organization’s security team.
Install Apps – members of this role group have permission to install Office applications: not the desktop
applications you may be thinking of, but applications that integrate with Office 365 itself. Examples of these apps
include Microsoft FindTime and the Message Header Analyzer.
Organization Management – members of this role group have access to manage all features of Exchange
Online, except for the rights that the Discovery Management role group allows. As discussed earlier, Office 365
Global admins and Exchange service administrators are automatically made members of this role group. You can
also assign membership of this directly in Exchange Online, but you may find it better to assign the Exchange
service administrator role in Office 365 instead as this will force the addition of an alternative email address for
lost password recovery.
Recipient Management – members of this role group have access to manage recipient objects in Exchange
Online. Recipient objects include mailbox users, shared mailboxes, resources mailboxes, distribution lists,
contacts, and other mail-enabled objects. Recipient Management members can also perform mailbox migrations,
message traces, and reset passwords. This role group is ideal for general day to day administration and is often
assigned to first and second level support teams. However, it is also suitable for general use by higher level
administrators who want to separate their admin accounts into low and high privilege use. For example, a Global
admin or Exchange service administrator may have a separate admin account that is only granted Recipient
Management rights. They can use the low privilege account for general administrative tasks, running report
scripts, and so on, and only log on to the higher privilege account for the less frequent tasks that require those
higher admin rights.
Records Management – members of this role group are granted rights to perform compliance-related tasks
such as configuring or viewing audit logs, configuring journaling, performing message traces, managing
retention policies, and configuring transport rules.
Security Administrator and Security Reader – although you will see these role groups in the Exchange admin
center, they only appear here because the role groups are used across multiple Microsoft products (including
Azure Information Protection and the Office 365 Security & Compliance center). The Security Reader role group
grants read-only access to Azure AD, Azure Identity Protection, Azure Privileged Identity Management, and all
audit logs and sign-in reports. The Security Administrator role group grants the same access as Security Reader,
plus the ability to configure security services. You won’t manage membership in these role groups yourself;
instead, you’ll assign access to the services, and when you do, the role group membership will be automatically
updated.
UM Management – members of this role group can manage the Unified Messaging settings for the organization,
as well as configure UM properties on mailboxes. This role group is useful for organizations that are using the
Unified Messaging features of Exchange Online for telephony integration.
View-Only Organization Management – this role group gives read-only access to all recipient properties and
configuration settings for the organization. This role group is ideal for anyone who needs broad visibility of the
organization without the rights to make changes. One scenario where this level of access is useful is for reporting
scripts and tools that need to be able to read a wide range of information about the organization.

As you can see there are a wide range of pre-configured administrative roles in Exchange Online to suit different
requirements. At the very least you should ensure that you are granting only the minimum administrative rights
required for people to perform their roles. It is all too easy to grant someone full admin rights (Organization
Management) and hope nothing goes wrong when they work with Office 365, but that is far from best practice.

Security & Compliance Center administrative roles

The Security & Compliance Center supports a set of permissions defined in RBAC role groups that allow the
segregation of responsibilities across different sets of users. The role groups are like those used by Exchange Online
except that these role groups only apply to the functionality available through the Security & Compliance Center and
have no connection to the role groups used by Exchange Online, even though some of them share the same name.

The Security & Compliance Center role groups define a set of roles that can be assigned to administrative users to
control what the user can do. Users assigned compliance permissions can use those permissions to view data from any
of the sources accessible to content searches. As functionality moves to the Security & Compliance Center from other
parts of Office 365, it is likely that Microsoft will define new roles and give more control about the scoping of
permissions for security and compliance activities.

To see the definitive version of the available roles and to assign or remove roles to users, go to the Permissions
section of the Security & Compliance Center.

Mailflow Administrator – introduced in mid-2018, this role group grants its members access to the mail flow
dashboard. Users who are members of the Global admin or Exchange admin role groups will already have access
to this dashboard, but if you grant membership in Mailflow Administrator to users they will only have access to
the dashboard, not to other Exchange or Office 365 management features.
Organization Management - members of this role group can perform the same tasks as Compliance
Administrators. They can also configure permissions for the Security & Compliance Center and configure audit
logging for the organization. Global admins are automatically added to this role group.
Compliance Administrator - members of this role group are typically IT administrators who are responsible for
configuring security and compliance settings and policies, such as mobile device management, DLP, and
preservation. Compliance administrators can also manage settings for reports in the Security & Compliance
Center.
Reviewer - members of this role group can view the list of e-discovery cases that currently exist. Case managers
can assign specific documents in an eDiscovery case for a reviewer to analyze or use a limited set of the analysis
features in Office 365 Advanced eDiscovery. Members of this group can see only the documents that are assigned
to them.
Records Management – members of this role group can manage retention and compliance features related to
how long user data is stored and what policies are applied and enforced to archive or expire it.
Supervisory Review - members of this role group can create supervisory review policies for the organization,
which determines the type of content that will be subject to supervisory review, and who will be performing the
reviews.
Service Assurance User – members of this role group can access the Service assurance section in the Security
& Compliance Center, where they’ll find reports, documents, and audit reports related to how Microsoft handles
customer data in Office 365.
Security Administrator - membership of this role group is not intended to be managed by you through the
Security & Compliance Center. This role group may include Microsoft Support or external partners.
Security Reader - like the Security Administrator role, this role group is not intended to be managed by you. It
gives read-only access to security and compliance features.
eDiscovery Manager - members of this role group can perform searches across Office 365 workloads, apply
holds to Exchange Online mailboxes, SharePoint Online sites, and OneDrive for Business libraries. An eDiscovery
Manager has access to specific eDiscovery cases. When you grant a user the eDiscovery Manager permission,
you can also grant them eDiscovery Administrator privileges, which allows the user to view and edit all
eDiscovery cases for the organization.

Like any other role assignment, the tenant administrator should review what accounts hold which roles on a regular
basis to ensure that users do not retain privileged access for longer than required. The eDiscovery Manager role
group is particularly powerful because of the two sub-roles that it contains. eDiscovery Managers have access to all
their assigned cases. eDiscovery Administrators have access to all cases and can assign themselves to a case.
Although dependent on the volume of eDiscovery workload within the tenant, the usual situation is to have only a few
eDiscovery Administrators and more eDiscovery Managers. However, in a small tenant, the same people might
comprise the two groups.

Teams administrative roles

In September 2018, Microsoft added four new administrative roles that are specific to the Teams product. As of
October 2018, the only way that you can assign these roles to users is to use the Azure Active Directory portal .
Assigning these roles allows users to undertake specific management tasks for Teams using an RBAC model very
similar to that used elsewhere in Office 365; users holding a role see sections of the Teams and Skype for Business
Online Admin Center appropriate for the role they hold. The current set of roles are:

Teams Service Administrator : This role can perform every action available in the Teams and Skype for
Business Admin Center. Anyone assigned the role can also run the equivalent PowerShell cmdlets. Accounts
holding this role are also able to create Office 365 Groups even if a policy to limit creation exists in the tenant.

Teams Communications Administrator : Anyone assigned this role can manage the meetings and voice
settings for Teams, including the ability to troubleshoot call quality problems. People with this role can access the
Users and Meetings sections of the dashboard, but don’t see sections like Org-wide settings and Messaging
policies. This role is typically given to those responsible for taking care of the voice and audio infrastructure used
by a tenant to support Teams meetings and calls. The knowledge and experience necessary to debug complex
telephony issues, especially in multinational tenants, often lies outside the ability of the average Office 365
administrator. See Chapter 16 for more information about Teams voice and audio communications.
Teams Communications Support Engineer: This role is intended for people who use Call Analytics to monitor
and address issues in call quality. People with this role can see full call data for all users, but they have no access
to policies, org-wide settings, or meeting configuration.
Teams Communications Support Specialist: This role gives access to users to perform basic troubleshooting
for calls and is intended for first-level support staff.

For more information about Teams administrative roles, see this page .
Privileged accounts
Best practice in the Windows community is to perform administrative actions with administrator accounts rather than
assigning the necessary permissions to user accounts. This is done to restrict permissions to a limited set of accounts
rather than allowing for a proliferation to user accounts, sometimes for doubtful reasons. It also ensures that you do
not have to perform regular reviews of highly permissioned accounts to remove permissions from accounts that have
no need for them. Inside Office 365, very few administrators have elevated permissions and great care is taken to
ensure that permissions are only granted when absolutely needed and for a minimum period.

You can certainly bring the practice of using dedicated accounts for administrative tasks forward into Office 365.
However, given that Office 365 is a very different environment that hosts many different workloads and the need for
permissioned access is reduced because there are fewer administrative tasks to perform (no server management, for
instance), perhaps this is a good time to consider whether a better approach could be taken.

Privileged Identity Management

Microsoft offers two features to manage privileged access to user information. The first is Azure Active Directory
Privileged Identity Management (PIM), a framework for management of privileged access to applications that depend
on Azure Active Directory, such as Office 365. Using PIM, you can manage temporary elevation of user accounts to
grant administrative privileges, such as Global Administrator. A user can be permanently eligible to elevate their
permissions, or alternatively you can require the user to request approval each time they require elevated
permissions. PIM will elevate the user's permissions for the time needed, then revoke the permissions afterwards.
Added controls such as enforcing multi-factor authentication (MFA) are also available in PIM, as well as an audit trail
and activity report. PIM is available for customers who have Azure AD Premium P2 edition, which is included with the
Enterprise Mobility + Security E5 license or can be licensed separately.

Privileged Access Management

The second feature is Privileged Access Management (PAM), which is now generally available for Exchange Online.
Other workloads will be added over time. PAM works on the basis that administrators create requests for
authorization when they want to perform privileged tasks. Policies controlling individual cmdlets or RBAC roles or role
groups state whether requests receive automatic approval or need manual review. These requests are routed to a set
of approvers (a mail-enabled security group). Anyone in the group can approve a request through the Microsoft 365
Admin Center or with PowerShell. Once approved, the requester can execute the task for as long as the approval still
is valid (the default is four hours). PAM is licensed with Office 365 E5 or the Advanced governance add-in.

In addition to considering who has permissions, you should also take steps to check the use of permissions through
auditing. Office 365 collects events for administrative actions in the Office 365 audit log (Chapter 21). You can use the
audit log search in the Security & Compliance Center to examine these events and export them for later analysis
should the need arise.

Office 365 administrative units

Administrative units (AUs) are containers for user objects, like organizational units on-premises. AUs allow you to
group your users into logical units, then apply policies or permissions to the AU instead of to individual users. For
example, you could create an AU for all the users in the Sales department, or all users working in Slovakia, or simply
all users whose accounts have a specific attribute.

Like on-premises OUs, the real power of AUs is the ability to designate an administrator who can perform various
tasks against all users and only those users who are in the scope of a particular AU. This is very important for large
organizations in Office 365, as up until now we didn’t have any features that enabled admin role separation. The
limited-scope admin roles that exist in Office 365 control what their holders can do, but they don’t restrict which
users those role holders can manage.

Microsoft announced the availability of AUs for Azure Active Directory in December 2014. However, until February
2018, you couldn’t directly use AUs to assign or manage permissions in Office 365. Now you can, but you must use
PowerShell. The basic workflow you’ll follow looks like this:

1. Create an AU using the New-MsolAdministrativeUnit cmdlet. You must specify a display name and a comment.
2. Get the object ID of the AU you just added by using the Get-AzureADAdministrativeUnit cmdlet.
3. Use the Add-MsolAdministrativeUnitMember cmdlet to add one or more users to the AU. You’ll need the object ID
of the user (available from Get-MsolUser ) and the object ID of the AU.
4. Use the Add-MsolScopedRoleMember cmdlet to specify the required elements for the “ triangle of power ”: who
(a UPN) is allowed to do what (exercise either User Account Administrator or Helpdesk Administrator rights) and
where (the AU scope) they can do it.

The last step needs some explanation. Recall that all RBAC-based access in Office 365 depends on three elements:
who, what, and where. In this case, you use the Add-MsolScopedRoleMember cmdlet to grant an individual user
permission to take either User Account Administrator or Helpdesk Administrator privileges on the specified
administrative unit. The result is that the specified user can exercise the privileges of the selected administrator role,
but only on users who are in the targeted administrative unit. The Office 365 Admin center will filter out any users
who aren’t in the selected AUs, so the administrator won’t even see them. If a user who’s been granted access
through Add-MsolScopedRoleMember logs in through PowerShell, she can still see the users that would be hidden in
the Admin center, but she can’t change them. The scoping specified by Add-MsolScopedRoleMember still applies to all
write operations that user might attempt.
License management
Everyone who wants to use an Office 365 application besides the administration tools needs a license. Because they
control all aspects of Office 365, Microsoft can be more insistent on this point than they are with on-premises Client
Access Licenses (CALs). You cannot access an application like Exchange Online without a license and an account that
has its license removed will end up losing all its data. License management is therefore an important part of Office
365 administration, with the goal being to ensure that you have enough licenses to allow people to work while not
paying for unused licenses.

The Licenses option in the Billing section in the Office 365 admin portal provides an overview of the licenses used by
a tenant. As we can see from Figure 4-7 , some E3 and E5 licenses are unassigned. This might become an issue if the
situation persists because Microsoft charges for every license monthly even if no one uses the license.

There are several ways to buy Office 365 licenses. Depending on what Office 365 subscriptions you have, and
whether you have a licensing agreement with Microsoft, you may buy licenses directly through the web portal,
through a reseller or partner, or directly from Microsoft. For partner and Microsoft purchases, the license term and
cost are whatever you negotiate; for purchases directly through the portal, the licenses you buy cover a 12-month
term, but you may pay for them monthly or annually. In either case, you pay up front. Microsoft doesn’t give refunds
for unused licenses. As you add licenses, you will see them “roll in” at renewal time. For example, suppose that on
January 1 you create a new tenant and buy 100 E5 licenses. Then in March, you buy another 100 licenses. Come
January 1 next year, all 200 licenses will renew at the same time.

The usage reports in the Office 365 admin portal will provide you with a view of who is making use of the services that
they are assigned licenses for, allowing you to decide about reallocating licenses or reducing the number of licenses
that you're paying for. The exception is for free licenses, such as the basic Power BI license listed in Figure 4-7 .

Figure 4-7: Reviewing license availability through the Office 365 portal

License management is also available in the Setup section of the Office 365 Admin Center, under Products , as
shown in Figure 4-8 . Functionally the two areas of the portal are very similar, allowing you to assign licenses or buy
more if you have no unused licenses. The Products view includes a graphical representation of the software included
with a given license. The link labelled "Migrate user mailboxes" takes you to the Data migration page of the Setup
section, where you can begin onboarding from a variety of email and non-email data sources.
Figure 4-8: Managing licenses in the Setup section of the admin portal

Azure Active Directory Group-Based License Management

Managing the allocation of Office 365 licenses for a large user population has been a pain point for many customers.
For smaller tenants with simple requirements, the allocation of licensing can either be handled manually on an as-
needed basis using the Office 365 admin portal or built in to a simple provisioning script. For larger tenants,
automation is essential, as manual methods are far too time-consuming for any environment with a high rate of
change, such as dealing with new and departed users, or licensing sub-features and extra applications.

In the past, some customers have invested quite a lot of time into scripting solutions based on Active Directory group
membership. A user is added to a group whose members should receive an Enterprise E5 license, and the custom
script runs the necessary PowerShell commands to assign that license. This approach becomes challenging with more
complex licensing scenarios such as sub-SKU features and assigning multiple products to an individual user (e.g.
Enterprise E5 plus Project and Visio).

To solve these challenges Microsoft made group-based license management available through the Azure portal as a
public preview in February 2017. The feature requires an Azure AD Basic subscription which can be purchased
separately or is included with Enterprise Mobility and Security E3. When group-based licensing becomes generally
available, Microsoft has said that it will be available to Enterprise E3 and higher customers. However, as of May 2018,
the documentation says “when the feature becomes generally available, some aspects of the feature might require one
or more Azure Active Directory Premium licenses”. This is hardly confidence-inspiring.

The groups that you can assign licenses to can either be created in Azure AD or synchronized from on-premises Active
Directory. The license assignments can either be static (i.e. to the members of a group) or dynamic (e.g. based on user
attributes such as ExtensionAttribute1). For some organizations, a department-based model will be the preferred
approach, with licenses assigned to groups representing the different departments within the organization. For
others, a product-based approach will be more appropriate, with each type of license being assigned to a single group,
and the users being added to that group regardless of which department they are in. If you're using static groups
instead of dynamic groups then you will need to directly add the user accounts to the group, as nested groups are not
supported for group-based licensing.

Transitioning from direct license management to group-based license management is simple. Once you put the group-
based license assignments in place, users can have both a direct and group license assignment listed for their account
for the same type of license. However, they will only consume a single paid license during this time, allowing you to
remove the direct license assignments slowly to ensure there are no unexpected results. License assignments will fail
though if there are no available licenses to assign. The solution is to purchase more licenses, then force the group-
based license assignment to reprocess.

As Microsoft rolls out new features to your Office 365 tenant they will usually be enabled by default. This will apply to
group-based license assignments as well, with new sub-SKU features enabling by default. As new features arrive you
should review your group-based licenses to ensure that any new sub-SKU features are enabled or disabled to meet
your organization's requirements.

Configuring the group-based licensing assignments is performed in the Azure portal. After logging in to the Azure
portal choose Azure Active Director y from the list of services in the portal, and then select Licenses (Figure 4-9 ).
Figure 4-9: Group-based licensing in Azure Active Directory

Select a product license and click on the Assign button. From Users and Groups , choose the group that you want to
assign licenses to, and then click on Select . In the Assignment options you can select the sub-features for the
license that you’ve chosen to assign to the group. In the example shown in Figure 4-10 the StaffHub, Teams, Sway,
and Yammer features have been disabled for the license assignment.

Click OK when you’re happy with your selections, and then click Assign to create the license assignment. If there are
any errors at this stage, you’ll receive a notification in your Azure portal. License assignments are updated almost
instantaneously, but for on-premises Active Directory groups you should expect the normal directory synchronization
delay before group membership changes flow through to license assignment changes.

Figure 4-10: Configuring sub-SKU features for a license assignment

If you've already been using direct license assignments for your users, you will see each user in the Azure portal listed
with an Assignment Path of both Direct and Inherited (Figure 4-11 ).
Figure 4-11: Users with direct and group-based license assignments

To complete the transition from direct license assignment to group-based license assignment the direct assignment
must be removed. You can select one or multiple users, and then click the Remove button. You should approach this
with some caution, removing the direct license assignments for users in small batches so that you can be sure that
you do not inadvertently leave your users unlicensed for an application that they are still using.

User accounts can be members of multiple groups for license assignment. For example, if a user is a member of a
group that is assigned the Enterprise E3 license, and a member of a group that assigns the Enterprise Mobility and
Security E3 license, the cumulative effect will be that both licenses are assigned to that user. This allows you to
approach group-based licensing in a modular fashion, instead of needing to create separate groups for all possible
combinations of license assignments in your organization.

To reduce the risk of licenses being accidentally removed from users, if a user is removed from a group that is
managing their license assignments the associated Office 365 services will initially go into a suspended state, instead
of a deleted state. The user will not be able to access and use their Office 365 services, but their data will be safe
while their IT administrators correct the licensing mistake. If the license removal was intentional, the suspended
services will eventually age out to a disabled state and will begin their normal deletion and final purge processes.

Using PowerShell to manage licenses

The Office 365 admin portal and the Azure AD group-based license management both allow you to manage your
license assignments easily. Licensing can also be managed using the cmdlets in the Azure AD PowerShell module.

Reporting Licenses for a Tenant

One useful scenario is to look at the number of licenses that you have purchased and consumed to better manage your
license expenditure. The Get-AzureADSubscribedSku cmdlet reports the licenses purchased for your Office 365
tenant.

[PS] C:\> Get-AzureADSubscribedSku | Select Sku*, ConsumedUnits

SkuId SkuPartNumber ConsumedUnits

----- ------------- -------------

1f2f344a-700d-42c9-9427-5cea1d5d7ba6 STREAM 4

b05e124f-c7cc-45a0-a6aa-8cf78c946968 EMSPREMIUM 4

c7df2760-2c81-4ef7-b578-5b5392b571df ENTERPRISEPREMIUM 4

6fd2c87f-b296-42f0-b197-1e91e994b900 ENTERPRISEPACK 17

a403ebcc-fae0-4ca2-8c8c-7a907fd6c235 POWER_BI_STANDARD 7

26d45bd9-adf1-46cd-a9e1-51e9a5524128 ENTERPRISEPREMIUM_NOPSTNCONF 4

90d8b3f8-712e-4f7b-aa1e-62e7ae6cbe96 SMB_APPS 3

d3b4fe1f-9992-4930-8acb-ca6ec609365e MCOPSTN2 1
8c4ce438-32a7-4ac5-91a6-e22ae08d9c8b RIGHTSMANAGEMENT_ADHOC 3

One interesting thing about Get-AzureADSubscribedSku is that it returns all the Azure and Office 365 SKUs used in a
tenant. The example above shows that the tenant licenses Enterprise Mobility and Security (EMSPREMIUM), which
has nothing to do with Office 365 aside from being another component of the Microsoft 365 suite. Capturing the
information in a variable is useful for further inspection of licensing information.

[PS] C:\> $licenses = (Get-AzureADSubscribedSku)

[PS] C:\> $licenses | Select -Property SkuPartNumber, ConsumedUnits -ExpandProperty PrepaidUnits | Format-Table

SkuPartNumber ConsumedUnits Enabled Suspended Warning

------------- ------------- ------- --------- -------

STREAM 4 10000 0 0

EMSPREMIUM 4 5 0 0

ENTERPRISEPREMIUM 4 50 0 0

ENTERPRISEPACK 17 25 0 0

POWER_BI_STANDARD 7 1000000 0 0

ENTERPRISEPREMIUM_NOPSTNCONF 4 5 0 0

SMB_APPS 3 3 0 0

MCOPSTN2 1 50 0 0

RIGHTSMANAGEMENT_ADHOC 3 50000 0 0

You might notice in the output above that the SkuPartNumber does not precisely match the actual name of the
license. For example, ENTERPRISEPACK is the SkuPartNumber for the Office 365 E3 plan and
ENTERPRISEPREMIUM is for Office 365 E5, while EMS is the SkuPartNumber for the Enterprise Mobility and
Security E3 license, and EMSPREMIUM is for EMS E5. A complete list of part numbers and friendly names is not
available online, although with a little searching and common sense you can usually work out what they mean. If any
confusion exists, opening a support ticket with Microsoft will get you the answers you need.

Licenses for Individual Features

We can also examine the individual licenses for specific features and services in a plan, also referred to as sub-SKU
features. As with the SkuPartNumber values, the ServicePlanName values are not a match for the friendly names that
you see in the Office 365 or Azure admin portals, but names like SWAY , POWERAPPS_O365_P2 , and
EXCHANGE_S_ENTERPRISE are obvious. Others are not so obvious, such as MCOSTANDARD (Skype for Business
Online; the “MCO” stands for “Microsoft Communicator Online”), but again some searching online will usually clear
up any confusion. Taking the ENTERPRISEPREMIUM (Office 365 E5) plan as an example, it is number 3 in the list
returned by Get-AzureADSubscribedSku , so we can see the individually-licensed features as follows:

[PS] C:\> $licenses[2].ServicePlans | Format-Table ServicePlanName, ServicePlanId

ServicePlanName ServicePlanId

--------------- -------------

BPOS_S_TODO_3 3fb82609-8c27-4f7b-bd51-30634711ee67

FORMS_PLAN_E5 e212cbc7-0961-4c40-9825-01117710dcb1

STREAM_O365_E5 6c6042f5-6f01-4d67-b8c1-eb99d36eed3e

THREAT_INTELLIGENCE 8e0c0a52-6a6c-4d40-8370-dd62790dcd70

Deskless 8c7d2df8-86f0-4902-b2ed-a0458298f3b3

FLOW_O365_P3 07699545-9485-468e-95b6-2fca3738be01

POWERAPPS_O365_P3 9c0dab89-a30c-4117-86e7-97bda240acd2

TEAMS1 57ff2da0-773e-42df-b2af-ffb7a2317929

ADALLOM_S_O365 8c098270-9dd4-4350-9b30-ba4703f3b36b

EQUIVIO_ANALYTICS 4de31727-a228-4ec3-a5bf-8e45b5ca48cc
LOCKBOX_ENTERPRISE 9f431833-0334-42de-a7dc-70aa40db46db

EXCHANGE_ANALYTICS 34c0d7a0-a70f-4668-9238-47f9fc208882

SWAY a23b959c-7ce8-4e57-9140-b90eb88a9e97

ATP_ENTERPRISE f20fedf3-f3c3-43c3-8267-2bfdd51c0939

MCOEV 4828c8ec-dc2e-4779-b502-87ac9ce28ab7

MCOMEETADV 3e26ee1f-8a5f-4d52-aee2-b81ce45c8f40

BI_AZURE_P2 70d33638-9c74-4d01-bfd3-562de28bd4ba

INTUNE_O365 882e1d05-acd1-4ccb-8708-6ee03664b117

PROJECTWORKMANAGEMENT b737dad2-2f6c-4c65-90e3-ca563267e8b9

RMS_S_ENTERPRISE bea4c11e-220a-4e6d-8eb8-8ea15d019f90

YAMMER_ENTERPRISE 7547a3fe-08ee-4ccb-b430-5077c5041653

OFFICESUBSCRIPTION 43de0ff5-c92c-492b-9116-175376d08c38

MCOSTANDARD 0feaeb32-d00e-4d66-bd5a-43b5b83db82c

EXCHANGE_S_ENTERPRISE efb87545-963c-4e0d-99df-69c6916d9eb0

SHAREPOINTENTERPRISE 5dbe027f-2339-4123-9542-606e4d348a72

SHAREPOINTWAC e95bec33-7c88-4a70-8e19-b10bd9d0c014

We can confirm that the list of features shown above belongs to Office 365 E5 plan by examining the SkuPartNumber .

[PS] C:\> $licenses[2].SkuPartNumber

ENTERPRISEPREMIUM

The ServicePlanId is a unique GUID for an individually-licensed feature, like e212cbc7-0961-4c40-9825-

01117710dcb1 for Microsoft Forms (the second one in the list above). The same identifier applies in all tenants, which
means that we can use the value to assign or remove licenses for that feature. Some features use the same identifier
across multiple plans, which is the case with Teams (TEAMS1), which uses 57ff2da0-773e-42df-b2af-ffb7a2317929
everywhere. However, different variants of a feature do exist in different plans, like FORMS_PLAN_E5 is in Office 365
E5 while FORMS_PLAN_E3 is in Office 365 E3. This means that if you want to manage the assignment or removal of a
sub-feature across multiple plans, you must be prepared to process the GUIDs for all the different variants.

Licenses Assigned to an Individual User

Next, we examine the licenses assigned to an individual Office 365 account. There are two user properties to consider.
The first is the AssignedLicenses property, which we can fetch with Get-AzureADUser .

[PS] C:\> Get-AzureADUser -ObjectId Andy.Ruth@office365itpros.com | Select -ExpandProperty AssignedLicenses | Format-List

DisabledPlans : { b737dad2-2f6c-4c65-90e3-ca563267e8b9, 7547a3fe-08ee-4ccb-b430-5077c5041653 }

SkuId : 6fd2c87f-b296-42f0-b197-1e91e994b900

We now know the SkuId of the license assigned to the user. You can find the matching license by running Get-
AzureADSubscribedSku to match the SkuId from the license assigned to the user with the licenses known to the
tenant. The check reveals that the license is the ENTERPRISEPACK license (Enterprise E3).

[PS] C:\> Get-AzureADSubscribedSku | Where {$_.SkuId -eq "6fd2c87f-b296-42f0-b197-1e91e994b900"}

ObjectId SkuPartNumber

-------- -------------

2b9bca49-687e-4e5f-8a52-21350b719b06_6fd2c87f-b296-42f0-b197-1e91e994b900 ENTERPRISEPACK

An account might have multiple licenses assigned to it. Each of the licenses is identified by its SkuId and can be
resolved back to the name of the license as shown above.

Individually Licensed Features

The Get-AzureADUser output also includes the DisabledPlans property. This property holds the ServicePlanId values
of any sub-SKU features that are disabled for the user. There are two ways to match ServicePlanId values to the actual
names of the sub-SKU features. The first is to use the output of Get-AzureADSubscribedSku to view the ServicePlanId
values for the individual services, as shown earlier. The other approach is to look at the AssignedPlans property of the
user, which we will get to later.

[PS] C:\> Get-AzureADUser -ObjectId Andy.Ruth@office365itpros.com | Select -ExpandProperty AssignedLicenses | Format-List

DisabledPlans : {a23b959c-7ce8-4e57-9140-b90eb88a9e97, 7547a3fe-08ee-4ccb-b430-5077c5041653}

SkuId : 6fd2c87f-b296-42f0-b197-1e91e994b900

[PS] C:\> Get-AzureADUser -ObjectId Andy.Ruth@office365itpros.com | Select -ExpandProperty AssignedPlans | Format-Table

CapabilityStatus, Service, ServicePlanId

CapabilityStatus Service ServicePlanId

---------------- ------- -------------

Suspended Sway a23b959c-7ce8-4e57-9140-b90eb88a9e97

Enabled TeamspaceAPI 57ff2da0-773e-42df-b2af-ffb7a2317929

Suspended YammerEnterprise 7547a3fe-08ee-4ccb-b430-5077c5041653

Enabled exchange efb87545-963c-4e0d-99df-69c6916d9eb0

Enabled SharePoint 5dbe027f-2339-4123-9542-606e4d348a72

Enabled SharePoint e95bec33-7c88-4a70-8e19-b10bd9d0c014

Enabled MicrosoftCommunicationsOnline 0feaeb32-d00e-4d66-bd5a-43b5b83db82c

Enabled MicrosoftOffice 43de0ff5-c92c-492b-9116-175376d08c38

Enabled RMSOnline bea4c11e-220a-4e6d-8eb8-8ea15d019f90

Enabled ProjectWorkManagement b737dad2-2f6c-4c65-90e3-ca563267e8b9

Enabled PowerAppsService c68f8d98-5534-41c8-bf36-22fa496fa792

Enabled ProcessSimple 76846ad7-7776-4c40-a281-a386362dd1b9

In the output above, we see that the two suspended services, Sway and Yammer, have ServicePlanId values listed in
the DisabledPlans property for the user. You will also notice that the service names returned in the Get-AzureADUser
output do always not match the service plan names returned by the Get-AzureADSubscribedSku output. For example,
Get-AzureADUser shows a service name of "TeamspaceAPI" while Get-AzureADSubscribedSku shows the same service
as "TEAMS1." These differences are mildly irritating but do reinforce the idea that you should match two difference
pieces of data by the ServicePlanId , not by the friendly name, whenever you run PowerShell cmdlets or write scripts
to manage licenses.

In addition to the service plan names, we can also view the provisioning status for each service:

Success - The application is fully provisioned and ready to be used. For example, an Exchange Online mailbox
has been created.
PendingActivation - The application has not been activated by an administrator for this user. For example, the
Intune application needs to be activated before user accounts can be added to it.
PendingInput - The application needs some administrator input before it can be made available to users.
PendingProvisioning - Nothing has yet been provisioned (created) for the user. This usually means that the
user has not accessed the application to cause it to fully populate its user settings and become ready for use.
Deleted - The service has been disabled for this user because the license has been removed.
Warning - The license for the application is close to expiration or in a grace period.
Suspended - The service has been suspended for this user, but not yet deleted.

For administrators who are familiar with using the MS Online PowerShell module to manage licenses, there is one
minor difference to be aware of. The Get-AzureADUser and Get-MsolUser cmdlets return slightly different information
for the same user object. Get-AzureADUser returns sub-SKU features that are Enabled , Deleted or Suspended , while
Get-MsolUser returns the status of all sub-SKU features. Get-AzureADUser also reports the date and time that the
license was assigned to the user, which is useful when investigating issues that may have been triggered by a
licensing change for a user.

[PS] C:\> Get-AzureADUser -ObjectId James.Ryan@office365itpros.com | Select -ExpandProperty AssignedPlans



AssignedTimestamp CapabilityStatus Service ServicePlanId

----------------- ---------------- ------- -------------

1/05/2017 11:12:19 AM Enabled SharePoint 5dbe027f-2339-4123-9542-606e4d348a72

1/05/2017 11:12:19 AM Enabled SharePoint e95bec33-7c88-4a70-8e19-b10bd9d0c014

1/05/2017 11:12:19 AM Enabled MicrosoftOffice 43de0ff5-c92c-492b-9116-175376d08c38

1/05/2017 11:12:19 AM Enabled ProjectWorkManagement b737dad2-2f6c-4c65-90e3-ca563267e8b9

1/05/2017 11:12:19 AM Enabled TeamspaceAPI 57ff2da0-773e-42df-b2af-ffb7a2317929

1/05/2017 11:12:19 AM Enabled PowerAppsService c68f8d98-5534-41c8-bf36-22fa496fa792

1/05/2017 11:12:19 AM Enabled ProcessSimple 76846ad7-7776-4c40-a281-a386362dd1b9

[PS] C:\> (Get-MsolUser -UserPrincipalName James.Ryan@Office365itpros.com).Licenses[0].ServiceStatus

ServicePlan ProvisioningStatus

----------- ------------------

Deskless Disabled

FLOW_O365_P2 Success

POWERAPPS_O365_P2 Success

TEAMS1 Success

PROJECTWORKMANAGEMENT Success

SWAY Disabled

INTUNE_O365 Success

YAMMER_ENTERPRISE Disabled

RMS_S_ENTERPRISE Disabled

OFFICESUBSCRIPTION Success

MCOSTANDARD Disabled

SHAREPOINTWAC Success

SHAREPOINTENTERPRISE Success

EXCHANGE_S_ENTERPRISE Disabled

Who’s Licensed for a Feature

Knowing how to figure out if an account is licensed, we can write some code to check who is licensed for a specific
feature. This code looks for users licensed for Teams. We only check users with mailboxes because these accounts are
likely to use Teams and the filter excludes guest users, room and resource mailboxes, shared mailboxes, and so on.
Apart from reporting on who has a Teams license, the code outputs a variable that can be used for further processing.

[PS] C:\> $TeamsServicePlanId = "57ff2da0-773e-42df-b2af-ffb7a2317929"

$Mbx = (Get-Mailbox -RecipientType UserMailbox -ResultSize Unlimited | Select DisplayName, ExternalDirectoryObjectId)

ForEach ($M in $Mbx) {

$Plans = (Get-AzureADUser -ObjectId $M.ExternalDirectoryObjectId | Select -ExpandProperty AssignedPlans)

$TeamsEnabled = $False

ForEach ($P in $Plans) {

If ($P.ServicePlanId -eq $TeamsServicePlanId -and $P.CapabilityStatus -eq "Enabled") {

$TeamsEnabled = $True
 Write-Host "User" $M.DisplayName "has a Teams license" }

}}

Assigning and removing licenses with PowerShell

The Set-AzureADUserLicense cmdlet assigns and removes Office 365 licenses to user accounts. You cannot assign an
enterprise license to an account with this cmdlet if one is already present, but you can switch from one enterprise
license to another. The easiest example is when a new account needs to be assigned a license. Before you can assign a
license to an Office 365 account, you must specify the country that the account belongs to because some Office 365
licenses are only available in specific geographies. For instance, different E5 licenses are available in countries where
the PSTN conferencing feature is available.

Now run the series of PowerShell commands to add a license. The process requires you to know the SkuId of the
license, which you can obtain by running Get-AzureADSubscribedSku and noting the SkuId of the plan you want to
assign. You then build an assigned license (singular) object, add it to another object representing the assigned
licenses (plural), and then finally run the Set-AzureADUserLicense cmdlet.

[PS] C:\> $User = Get-AzureADUser -ObjectId Sanjay.Patel@Office365itpros.com

[PS] C:\> $License = New-Object -TypeName Microsoft.Open.AzureAD.Model.AssignedLicense

The SkuId we’re going to assign is for the ENTERPRISEPACK plan. The GUID for the plans don’t change acrpss
tenants, so the value you see here is the one used everywhere across Office 365.

[PS] C:\> $License.SkuId = "6fd2c87f-b296-42f0-b197-1e91e994b900"

[PS] C:\> $LicensesToAssign = New-Object -TypeName Microsoft.Open.AzureAD.Model.AssignedLicenses

[PS] C:\> $LicensesToAssign.AddLicenses = $License

The details of the license are now in the $LicensesToAssign variable and we can go ahead and assign it to the user.
Beforehand, it does no harm to run the Set-AzureADUser cmdlet to make certain that the target user’s country code is
set correctly. In this example, we set the country code to “US” for the United States. Other valid country codes include
“GB” (United Kingdom), “DE” (Germany), “FR” (France), “AU” (Australia), and “SK” (Slovakia). Accounts that do not
require licenses (see below) do not need to be assigned a usage location.

[PS] C:\> Set-AzureADUser -ObjectId $User.ObjectId -UsageLocation US

Now we run Set-AzureADUserLicense to assign the license in the $LicensesToAssign variable to the target user.

[PS] C:\> Set-AzureADUserLicense -ObjectId $User.ObjectId -AssignedLicenses $LicensesToAssign

Licensing Sub-Features
As discussed above, service plan licenses like Office E3 and E5 include licenses to allow user access to several
applications, like Teams, Yammer, and Microsoft Forms. You might decide that you do not want to use certain
applications within your tenant even though the applications are available in a service plan. You might also decide to
disable applications for specific users. We do this by building an assigned license object as before, but this time we
configure the object with the list of plans we want to disable for the user. When you assign a SKU to a user, she will
receive all its features by default unless you specify otherwise.

In the example below, only the Exchange and Office 365 ProPlus features are licensed for the user, and the remaining
features of the Office E3 license are disabled. To do this, we have to specify the other features within the SKU to be
disabled.

First, we identify the target user and set up a list of features to enable.ed. This is merely a convenience; we cannot
directly specify the set of features to be enabled.

[PS] C:\> $User = Get-AzureADUser -ObjectId "Kim.Akers@office365itpros.com"

[PS] C:\> $SkuFeaturesToEnable = @("EXCHANGE_S_ENTERPRISE","OFFICESUBSCRIPTION")

Now, set up a variable and populate it with a standard license for the ENTERPRISEPACK (Office 365 E3) plan. Again,
we use the GUID for the plan as returned by Get-AzureADSubscribedSku .

[PS] C:\> $StandardLicense = Get-AzureADSubscribedSku | Where {$_.SkuId -eq "6fd2c87f-b296-42f0-b197-1e91e994b900"}

The next step is to create a variable and populate it with all the features included in Office 365 E3 that we do not want
our user to access. We do this by reference to the array set up earlier holding the features to be licensed. This
variable will be used when we actually make the license assignment.

[PS] C:\> $SkuFeaturesToDisable = $StandardLicense.ServicePlans | ForEach-Object { $_ | Where {$_.ServicePlanName -notin

$SkuFeaturesToEnable }}

We now create a new license variable to hold the assigned license and update it with the SkuId for ENTERPRISEPACK
(for the license). The next command populates the list of features to be disabled in the same variable pointing to the
license.

[PS] C:\> $License = New-Object -TypeName Microsoft.Open.AzureAD.Model.AssignedLicense

[PS] C:\> $License.SkuId = $StandardLicense.SkuId

[PS] C:\> $License.DisabledPlans = $SkuFeaturesToDisable.ServicePlanId

At this point, if we examine the $License variable, we find something like the information shown below. We can see
that the SkuId is correct, and we can see that several disabled features are present. For example, the GUID c87f142c-
d1e9-4363-8630-aaea9c4d9ae5 is for BPOS_S_TODO_2 (Microsoft To-Do), while 2789c901-c14e-48ab-a76a-
be334d9d793a is for FORMS_PLAN_E3 (Microsoft Forms), To see the full list of disabled features, you expand the
DisabledPlans property.

[PS] C:> $License | Format-List

DisabledPlans : {c87f142c-d1e9-4363-8630-aaea9c4d9ae5,

2789c901-c14e-48ab-a76a-be334d9d793a,

9e700747-8b1d-45e5-ab8d-ef187ceec156,

8c7d2df8-86f0-4902-b2ed-a0458298f3b3...}

SkuId : 6fd2c87f-b296-42f0-b197-1e91e994b900

Our variable is loaded, so we can create a new variable to assign the licenses, populate it, and then run Set-
AzureADUserLicense to assign the license to the user account.

[PS] C:\> $LicensesToAssign = New-Object -TypeName Microsoft.Open.AzureAD.Model.AssignedLicenses

[PS] C:\> $LicensesToAssign.AddLicenses = $License

[PS] C:\> Set-AzureADUserLicense -ObjectId $User.ObjectId -AssignedLicenses $LicensesToAssign

You can use the same technique to enable a sub-SKU feature for a group of users. This is a common task because
Microsoft often releases new features or workloads faster than organizations can skill up to cope with the new
functionality; if you have an application (such as Microsoft Whiteboard or Planner) that you want to disable until you
are ready to deploy it, you might want to later enable it. To do so, put the appropriate sub-SKU feature in the
SkuFeaturesToEnable variable and you are ready to proceed.

Removing a Feature for Multiple Users

Life would be easy if an administrator only ever had to take care of a single user. This isn’t usually the case, so we
should look at an example of how to disable features for multiple users. The example code below disables Teams for
any user whose mailbox is marked by having the value “NoTeams” in the ExtensionCustomAttribute1 property. You
could use another criterion to find the users to process, such as everyone in a specific office or country, or those with
a certain job title. What matters is that we start off with a collection of mailboxes. After Get-AzureADSubscribedSku
fetches the list of current plans known to the tenant and a list of features to remove is set up (in this case, Teams and
Sway), the following processing then takes place for each user:

The current set of assigned licenses is fetched.

For each license held by the user, the code checks whether its feature set includes the features we want to
disable (note that we can check using either the feature name or the GUID for the feature). In other words, for
Teams, we could use 57ff2da0-773e-42df-b2af-ffb7a2317929 or TEAMS1.
If the feature is found, the code adds it to a variable holding the list of disabled features.
After all the licenses assigned to the user are checked, the desired features are removed from any licenses where
they are found.
The ExtensionCustomAttribute2 property in the user’s mailbox is updated with a message to tell what licenses
were removed and when.

[PS] C:\> $MBX = (Get-Mailbox -RecipientTypeDetails UserMailbox -Filter {ExtensionCustomAttribute1 -eq "NoTeams"}| Select
DisplayName, UserPrincipalName, Alias, ExternalDirectoryObjectID)

Write-Host "Processing licenses for" $MBX.Count "users"

$LicensesRemoved = 0

$SKUs = Get-AzureADSubscribedSku

$PlansToDisable = @("57ff2da0-773e-42df-b2af-ffb7a2317929","SWAY")

ForEach ($M in $Mbx) {

 $User = (Get-AzureADUser -ObjectId $M.ExternalDirectoryObjectId | ? {$_.AssignedLicenses})

$PlansRemoved = $Null

$FeaturesRemoved = 0

$UserLicenses = New-Object -TypeName Microsoft.Open.AzureAD.Model.AssignedLicenses

ForEach ($License in $User.AssignedLicenses) {

$SKU = $SKUs | ? {$_.SkuId -eq $License.SkuId}

ForEach ($PlanToDisable in $PlansToDisable) {

If ($PlanToDisable -notmatch "^[{(]?[0-9A-F]{8}[-]?([0-9A-F]{4}[-]?){3}[0-9A-F]{12}[)}]?$") { $PlanToDisable =

($SKU.ServicePlans | ? {$_.ServicePlanName -eq "$PlanToDisable"}).ServicePlanId }

If ($PlanToDisable -in $SKU.ServicePlans.ServicePlanId) {

$License.DisabledPlans = ($License.DisabledPlans + $PlanToDisable | Sort -Unique)

$SkuName = $Skus | ? {$_.Skuid -eq $License.Skuid} |

Select -ExpandProperty SkuPartNumber

$PlanNameDisabled = ($Skus.ServicePlans | ? {$_.ServicePlanId -eq $PlanToDisable} |

Select -ExpandProperty ServicePlanName)

$PlanNameDisabled = $PlanNameDisabled[0].ToString()

If ($FeaturesRemoved -eq 0) {

$PlansRemoved = $PlanNameDisabled }

Else {

$PlansRemoved = ($PlansRemoved, $PlanNameDisabled -Join ", ") }

$FeaturesRemoved++

#$PlansRemoved += $PlanNameDisabled

Write-Host "Removed license for" $PlanNameDisabled "from" $SkuName "for"

$M.DisplayName

$LicensesRemoved++ }

$UserLicenses.AddLicenses += $License

Set-AzureADUserLicense -ObjectId $User.ObjectId -AssignedLicenses $userLicenses

If ($FeaturesRemove -eq 1) { $Text = "Removed license for " } Else { $Text = "Removed licenses for " }

$LicenseUpdateMsg = $Text + $PlansRemoved + " for " + $M.DisplayName + " on " + (Get-Date)

Set-Mailbox -Identity $M.Alias -ExtensionCustomAttribute2 $LicenseUpdateMsg

Write-Host $Mbx.Count "accounts processed and" $LicensesRemoved "licenses removed"

The script seems long, but many of the lines exist to document and inform the administrator what happens. You could
strip it down to 13-14 lines of code to get to the bare essentials. And like any PowerShell code, there are a myriad of
changes and enhancements that you could make to improve and enhance the code.

A PowerShell script that implements basic license management with a UI is available in the TechNet Gallery .

Tracking down unlicensed accounts

You can also the Get-AzureADUser cmdlet to reveal the accounts in a tenant that do not have a license. Although
every user account needs an Office 365 license before they can access applications, you may have users that don’t
need access to Office 365. The Office 365 Admin Center includes a view of unlicensed users. In the dashboard, select
Users , then Active Users , and then select the Unlicensed users view from the drop-down menu. Unfortunately,
this view also includes accounts that don’t need to be licensed, such as:

Shared mailboxes.
Room mailboxes.
Equipment or resource mailboxes.
Guest accounts created by SharePoint Online in Azure Active Directory to enable sharing of documents with
external users.
Accounts whose sign-in status is set to Blocked.
Administration accounts (if they don’t need to use a mailbox).
Guest accounts created by Azure B2B collaboration by applications like Office 365 Groups and Teams.

We can run simple PowerShell commands against Azure Active Directory to return information about licensed and
unlicensed users. This example returns a list of user accounts that have licenses:

[PS] C:\> Get-AzureADUser -All $true | Where {$_.AssignedLicenses}

While this example returns unlicensed accounts, you may want to filter out guest user accounts because tenants often
have many of these accounts:

[PS] C:\> Get-AzureADUser -All $true | Where {!$_.AssignedLicenses -and $_.UserPrincipalName -NotLike "*#EXT#*"} | Select
DisplayName, UserPrincipalName

Managing Guest User Accounts

Any time you see an account that includes #EXT# in its User Principal Name, it is an account created by an
application using Azure B2B collaboration to support external access to tenant resources. Here is an example:

Paul Cunningham paul_oz.com.au#EXT#@O365itpros.onmicrosoft.com

SharePoint Online has moved away from using guest accounts to share documents and folders, and the most common
use within Office 365 today is to support guest members of Office 365 Groups and Teams. You can check whether an
account is a guest member of Office 365 Groups or Teams by using PowerShell to look through the membership
information for the groups within a tenant as shown below. If you get a match, it means that the account is a guest
member of a group or team. Note that if the tenant has many groups, this command is likely to be slow to run. If you
want to know the team or group that a guest user belongs to, you can find the commands to discover this information
in Chapter 14.

[PS] C:\> Get-UnifiedGroup | Get-UnifiedGroupLinks -LinkType Member | ? {$_.Name -Match "paul_oz.com.au#EXT"}

If no group memberships are found, the guest account might be obsolete and therefore could potentially be removed if
you want to clean up the directory. To search for and remove a guest account, you can run a one-line PowerShell
command like this:

[PS] C:\> Get-AzureADUser | Where {$_.UserPrincipalName -like "mike.ryan*#EXT#*"} | Remove-AzureADUser

This is not a command to execute on a whim because Remove-AzureADUser does not prompt for confirmation before
removing the accounts, nor does it support the -WhatIf parameter so that you can test the impact of the command
before you run it. Furthermore, removing accounts created by SharePoint Online to allow external users to share
documents also removes any permissions or access associated with the accounts. With this point in mind, you should
first check whether the accounts are needed and then test the Remove-AzureADUser command to make sure that the
right accounts are retrieved from Azure Active Directory before continuing to delete them with Remove-AzureADUser
. (This is one of the many cases where a reporting tool such as Quadrotech Radar can be very useful for determining
which objects or services are actually being used versus which are sitting idle and may be removed without harm.)

Managing Feature Releases

Microsoft uses a series of “rings” to control the release of new features within Office 365. In general, the first ring is
the Office 365 development team, the next is Microsoft, the third is composed of tenants who have nominated
themselves by signing up for “Targeted Release,” and the last is general availability (also known as “standard
release”). This approach varies by workload; individual applications, such as Teams or Planner, may use more rings in
their deployment model.

Standard release is the term Microsoft uses when the feature is made available to all tenants who buy Office 365
plans that include the functionality. Tenants who opt for Targeted Release see new features (or updates to existing
features) a few weeks ahead of general availability. However, for some new applications, the period covered by
Targeted Release can extend up to several months. The ability to control how new features become available to tenant
users is through Release Preferences in the tenant Organization Profile , available through the Settings section
of the Office 365 Admin Portal (Figure 4-12 ).
Figure 4-12: Enabling Targeted Release for a tenant with “selected users” chosen

Click Organization profile and go to the Release preferences section. Three options are available:

Standard release.
Targeted release for everyone.
Targeted release for selected users.

The last option allows the administrator to nominate specific individuals or groups (referred to as a staged rollout)
while the remainder of the users continue to access the functionality available in the standard release. For example,
you might decide that only the IT department should be exposed to a new application to allow some experience to be
gathered about its functionality and so drive a decision about how it might be used within the organization.

The selected option for service updates is stored in the tenant organization configuration and can be checked by
running the Get-MsolCompanyInformation cmdlet from the MSOnline PowerShell Module. The values returned are
StandardRelease , FirstRelease , and StagedRollout . You can view the current setting for your tenant, but you can't
change your choice for service updates using PowerShell.

[PS] C:\> Get-MsolCompanyInformation | Select ReleaseTrack

ReleaseTrack

------------

FirstRelease

If a you use a staged rollout, the user accounts chosen to receive Targeted Release updates are marked in Azure
Active Directory. The Get-AzureADUser cmdlet in the Azure AD PowerShell Module does not return the release track
configuration for a user account. However, you can use the Get-MsolUser cmdlet in the MS Online module to discover
the accounts enabled for Targeted Release.

[PS] C:\> Get-MsolUser | Where-Object {$_.ReleaseTrack -eq "StagedRolloutOne"}| Format-Table Displayname, UserPrincipalName,
ReleaseTrack

Unfortunately, the Set-MsolUser cmdlet does not allow you to update the ReleaseTrack property for user account. If
you want to add a user or remove a user to or from the list of those enabled for Targeted Release, you must do this
through the Office 365 Admin Center. Note too that switching an account from Targeted Release to Standard Release
or vice versa does not take immediate effect. It can take up to 24 hours before a switch is fully effective. When the
switch occurs, users gain access to features in the Targeted Release ring, meaning that you might need to coach them
about what to expect. Keep in mind that putting a user in Targeted Release just enables their ability to see or use a
feature; features that are implemented in the Office 365 Professional Plus software won’t be visible to the user until
she downloads the appropriate version of the software. Depending on your organizational policies, this may add an
extra delay before the new features become available.

Although Microsoft’s goal is to move features from Targeted Release into general availability reasonably quickly, the
exact period between the two phases can vary from feature to feature. Timings can be affected by user feedback, bug
reports, or discovery of flaws such as performance or scalability issues that must be addressed before a feature can be
made available to all. The shortest period is a week or so. The longest (to date) is six months. The rate of change also
varies within applications. Exchange Online tends to see more new features over a certain period than SharePoint
Online while the rate of change for new features can also be high. Yammer and Teams are exceptions to Targeted
Release because they use different mechanisms to give early access to new features to a select group of tenants.

General availability does not mean that a feature shows up everywhere overnight. Given the size of Office 365 it
should come as no surprise that it can take a few weeks before every tenant in every datacenter sees a new feature.
The exact speed of deployment depends on the results gained by making software available to tenants through
Targeted Release plus other factors such as local market support. Using a phased approach to introduce new software
inside massive cloud environments is a common tactic found across all providers. It is just impossible to have a “big
bang” where a new feature turns up for every user at a certain time. In addition to the overhead imposed by the need
to roll software out globally across hundreds of thousands of servers, keep in mind that Microsoft often staggers
feature release dates by geography. In general, features tend to appear first in North America, the UK, Western
Europe, and Australia, then in other areas. For example, nine months after the December 2017 introduction of Teams
Calling, the feature isn’t available worldwide yet.

Another factor in feature availability is the type of tenant you have. In general, enterprise tenants get features first,
and education and government tenants (including those in the Government Community Cloud, or GCC) come later, if
at all. The small-business versions of Office 365 fall somewhere in the middle. Keep in mind that Microsoft appears to
believe that they will make the most money by delivering all their features to all tenants everywhere, so this is their
aim; when they don’t make a feature available in a region or to a type of tenant, it’s either because there are
engineering, operational, or legal constraints that prevent it or because the feature requires enough investment that it
may not be profitable for them to deploy it.

When new features arrive in your tenant, they will almost always be enabled by default for all users who are assigned
the necessary licenses, regardless of your Organization Profile settings. That includes features such as Microsoft
Teams and Planner that were added as sub-SKU features; in other words, they were added to existing Office 365
licenses such as Business Premium, E1, E5 and so on, rather than being added as separately licensed services like
Intune or Dynamics. Applications that don’t require a separate license, such as Microsoft Whiteboard, are also usually
enabled by default.

Features that are released in Public Preview before General Availability are not enabled by default; however, they will
become enabled once the feature reaches General Availability. The approach of enabling features by default makes
adoption easier but is a concern for organizations that have strict change management procedures, or those who want
to control the rollout of new features so that they can provide appropriate training to their IT support staff and end
users. It is important that you maintain awareness of the changes that are rolling out to Office 365 customers by
monitoring your Message Center notifications, and keeping an eye on other sources of early information such as the
Office 365 roadmap .

Mastering Targeted Release : Targeted Release allows tenants to gain faster access to new functionality. However,
there is a downside to the value gained by seeing new features earlier than the norm. Microsoft tests new code
before making it available through Targeted Release, but it is well known that frayed edges (bugs) can appear in new
software when exposed to the stresses of production workloads. Other common challenges include new features
showing up in user interfaces without warning or any sign as to how they should be used with little or no available
documentation. Selective Targeted Release is available to allow some users in a tenant to access new features earlier,
but this option is sometimes not supported by an application.

To mitigate these downsides of Targeted Release, some companies run two tenants: a "production" tenant for most
users and another trial tenant that is configured as Targeted Release for a small subset of users responsible for
testing new software as it appears. This is analogous to the situation often found in the on-premises world where
customers keep a test environment to deploy new builds of Exchange as Microsoft makes them available.

It is important to realize that the appearance of a new feature to tenants who have enabled Targeted Release does
not mean that the feature will be made available across Office 365 soon afterwards. Microsoft can and does use time
to gain knowledge of how customers use a feature and can adapt and change the feature over an extended period
before deciding that the software is suitable for general consumption and ready to be rolled out across Office 365.
Take Office 365 Groups and Delve as examples. Microsoft launched both features in late 2014 and Groups achieved
general deployment across Office 365 in early 2015. However, Delve stayed in Targeted Release for six months and
its functionality was significantly enhanced during this time before it eventually achieved general roll-out status in
mid-March 2015. Localization can also delay final release. Targeted Release is a large-scale beta program for Office
365 and no assumption should be made when or if any feature will be available just because Microsoft makes the
functionality available to tenants configured for Targeted Release.

Backup for Office 365

Taking backups for Office 365 data isn’t strictly necessary and Microsoft only takes backups for SharePoint Online.
However, many companies believe that backups are a good thing because they want to have a method of restoring
user and configuration data should the need arise. Other reasons cited to justify backups include:

The need to be able to go back to a specific point in time in case a tenant is infected with ransomware or other
logical damage.
The ability to meet regulatory or audit requirements to have multiple copies of important corporate data outside
the control of a single provider.
The ability to recover from administrator errors such as when accounts, mailboxes, or important pieces of data
are removed by mistake. This category also includes actions by rogue administrators, usually people who are
leaving the company who decide to do something malicious on their way out.
Being able to recover from a hacking attack, or if files in the tenant are encrypted by ransomware.

Office 365 includes mechanisms to address some but not all these requirements. The question is whether third-party
backup products can fill the gap. When we discuss Office 365 from a backup perspective, the following aspects of
Office 365 should be considered as you determine what data should be backed up. Knowing what you need to backup
and why the backup is necessary will drive the choice of the backup technology to use. You should consider:

Backup of the base storage workloads: SharePoint Online, Exchange Online, OneDrive for Business, and the
configuration and user information held in Azure Active Directory.
Backup of applications that use Azure for all or part of their storage: Office 365 Video and Stream.
Backup of applications that use multiple Office 365 components such as Teams, Planner, and Groups.

In addition, you should consider:

How are backups processed? A backup product might be able to process the volume of data generated by a
small tenant and struggle to process the volume created by a large tenant. The backup application must move
data from Office 365 to the backup location, which might mean that the data must travel via the internet, so
network considerations and the ability of the backup vendor to process inbound data come into play. On the other
hand, if the backup location uses Azure, the data might stay within the Microsoft datacenter network and the
transfer is faster and easier.
What APIs are used for backups? Office 365 includes many APIs that can interact with mail, documents, tasks,
groups, and so on. However, not every API can stream large volumes of data to a backup destination, so some
testing should occur to ensure that a backup application can handle the quantity of data produced by a tenant,
especially at peak load.
What data are backed up? Many backup vendors use cloud versions of on-premises products to handle the base
storage workloads. Office 365 applications like Teams and Planner don’t have on-premises equivalents, so on-
premises backup products can’t deal with their data. Thus, you might select a product that can copy Exchange
mailboxes and SharePoint documents but ignores everything else.
How often is the data backed up? A daily backup might be enough to deal with small tenants, but constant
and ongoing backups (trickle mode) might be necessary to process the quantity of data produced by large
tenants.
Where is the backup data stored? Most backup vendors propose using their own cloud datacenter to hold
backup data. This approach is perfectly acceptable, if it meets the customer’s data at rest and data sovereignty
requirements. Unsurprisingly, few backup vendors can aspire to the same widespread distribution of datacenters
that Office 365 provides, but a growing number use of Azure or Amazon Web Services bulk storage as a backup
target.
How easy and rapidly can the data be restored? Having a backup copy of data is one thing; being able to
restore it is another. How quickly can data be brought online within Office 365 from the backup copy? Can the
data be merged with live data or will it overwrite what’s there (for example, does a complete document library
need to be restored).
Can data be restored in context? Restoring a single mailbox or single document library is straightforward. It is
much more complex to rebuild an entity like a group, plan, or team as it was at a point in time. This is because
those entities depend on multiple Office 365 components, each of which must be restored in such a way that the
links between the different components are preserved and accurate. Data that cannot be restored in context
might still be valuable, but it will be raw and need manipulation before it is fully useful again.
How much does the backup cost per user per month? What basis is used for this calculation (per site,
mailbox, licensed user, etc.).

In short, backing up Office 365 data is not simply a matter of stretching the application-centric technology and
techniques used to process on-premises data. Office 365 is a radically different environment that demands a different
approach to backup.

Backup for Exchange Online

Microsoft is deadly serious when it says that native data protection is the best approach for Exchange. This means
that four copies of each database exist, including a lagged copy, and that data is spread across at least two
datacenters to ensure resilience. Users must recover deleted items if they make a mistake and if they don’t do this
before the default 14-day retention period (extendible to 30 days), they won’t be able to retrieve the data.

Given that over two million databases are in use by Exchange Online, deployed over more than two thousand
Database Availability Groups, the use of native data protection is understandable when you consider how many pieces
of backup media might be used to copy all the databases running inside Exchange Online, not to mention the
enormous cost of managing and validating the backups and handling and storing the backup media.

Third party software companies offer online cloud-based backups that can extract data from Exchange Online and
send the data to their repositories from where the data can be retrieved if necessary. However, the cost of these
services might not be justified for the benefit gained, especially if data is protected by features such as litigation
holds. Don’t use backup utilities that extract mailbox data and store it in PSTs. This is a terrible idea from a
performance, logistical, data protection, and compliance standpoint.

Backup for SharePoint Online

Unlike Exchange Online, which relies on native data protection to protect mailbox databases, Microsoft takes backups
for SharePoint Online site collections (these backups cover OneDrive for Business too). Here is what happens:

Microsoft backs up the content in site collections every 12 hours. The backups are kept for 14 days.
Office 365 tenant administrators have no control over backups or restores. If a restore is necessary, it can only be
started by contacting Office 365 support. Microsoft will ask you to say the best time for the restore (i.e. a time
 when the required information was known to exist in the site collection). Determining that time can be quite a
challenge!
You cannot restore a single item, document, list, or library. A full restore of a site collection is the only option.
The restore is targeted at the URL for the site collection and will therefore overwrite whatever data currently
exists in the site collection. If you need to preserve data in a site collection that is to be restored, you must copy
or move all the information that exists in the collection before the restore is done and then readjust the content
afterwards as needed.

Before running to ask Office 365 support to restore a site collection, it’s important to understand that SharePoint
Online allows information to be recovered in two other ways. First, the recycle bin for a site holds deleted items, lists,
documents, and so on for up to 93 days after deletion (in this article , Microsoft says “approximately 90 days”). A user
can recover items that they have deleted from the recycle bin with just a few clicks, so if they make an error deleting
something, that error can be quickly fixed without administrator intervention. (Teaching your users that they can do
this is a separate issue that you’ll have to solve on your own!) If a user empties their personal recycle bin, or content
has been moved from the site recycle bin to the site collection recycle bin (also known as the second stage recycle
bin), a site collection administrator can recover items during the retention period.

Most users won’t know that items are removed from document libraries through the processing of SharePoint Online
document deletion policies that are configured to remove items after a fixed period, just like mailboxes are cleaned up
through Exchange retention policies. However, the retention period for policy-driven removal of files from libraries is
likely to be in the order of five or ten years and important or frequently-used documents are probably kept in libraries
that are excluded from automatic deletion.

Versioning is the other way that a user can recover data; when versioning is enabled, a previous version of an item
can be restored if required. Versioning is enabled by default for both SharePoint Online and OneDrive for Business to
keep every version of documents uploaded to libraries or items to lists. You can change versioning behavior by editing
the library settings and defining what kind of versioning is needed. For example, you disable versioning or limit the
number of versions of an item to keep. Libraries used to manage the creation of formal documents such as the
chapters of a book often keep both minor and major versions of files (minor versions are usually regarded as drafts
created between major versions). SharePoint Online automatically purges versions that are not needed.

The fact that multiple versions of files can be kept makes it possible to recover from accidents and some disasters. If a
user makes a hash of updating a document, they can start over by reverting to an earlier version. More importantly, if
a virus or ransomware attack hits a tenant and infects all the files in a library, it is possible to redress much of the
damage by restoring earlier copies of affected documents. OneDrive for Business includes the ability to restore files to
any point over a 30-day moving window; the same feature is coming to SharePoint Online libraries soon.

All administrators will be familiar with the plaintive cries from users who have deleted an important file and need it to
be restored. Hopefully, the combination of recycle bin and versioning is enough to respond to most user pleas.
However, in some cases, the 14-day restore window might be insufficient for a user to realize that something
important is missing and the need to restore a complete site collection can also be problematic. Along with the ability
to perform more granular restores, these are the reasons why some tenants look at third-party backup solutions for
SharePoint Online. The basic idea behind backup solutions for Office 365 is that you have remote agents that make
scheduled connections to applications like SharePoint Online to grab information and copy it out of Office 365 to some
other location controlled by the organization. As in the case of any ISV solution, you should test products thoroughly
to ensure that they meet your needs, including aspects such as security, privacy, and data at rest. Companies such as
AvePoint, Spanning, CloudAlly, and Skykick offer backup solutions in this space.

The decision whether to invest in a third-party backup solution for SharePoint Online largely depends on the faith a
tenant puts into the way that Microsoft manages SharePoint data and the perceived risks that exist. For example, in
2015, some tenants suffered data loss through the CryptoLocker virus when users uploaded infected files from their
PCs to SharePoint Online or OneDrive for Business sites. Short of paying the demands of the virus authors, the usual
solution is to restore back to a point in time before the infection occurred. This might be possible with Microsoft’s
processes and procedures (such as the Restore to selected date and time option available for OneDrive for Business) if
you catch the problem early but recovering from a problem such as virus infestation or malicious deletion is likely to
be easier with a third-party backup.

Backup for Office 365 apps

Traditional backup products address the need to copy information belonging to an individual workload. For example,
you take backups to protect data in on-premises Exchange databases or SharePoint sites. The different nature of
Office 365 creates problems for this approach because new functionality is built by combining features taken from
different workloads. Take Microsoft Planner as an example. Data is held in group mailboxes managed by Exchange
Online, group document libraries managed by SharePoint Online, and the Tasks service running in Azure.

The Microsoft Teams application is similar in the way that it brings data together from multiple sources to deliver its
functionality to users and introduces a further complication in that Teams can use Planner as one of its resources.
Office 365 Video is another example. This application uses a mixture of SharePoint site collections and Azure Media
Services to store and deliver videos to players. The point to remember is that if you use backups to copy elements of
one workload and miss the others, then you might not be able to fully reconstitute the data necessary for an
application if the need arises.

AvePoint introduced a new version of its DocAve backup software in early 2017 that can backup and restore the
documents and conversations belonging to an Outlook group (an Office 365 group that uses Exchange Online to hold
its conversations). DocAve can process complete libraries and mailboxes or individual documents, conversations, or
calendar items. Although this solution cannot process content belonging to Yammer conversations, Planner, or Teams,
it is the first to expand from a traditional single-application approach to accommodate Office 365 apps.

Backup Technology Changes

Backup technology changes over time to improve functionality and make it easier to backup and restore Office 365
data. It’s also true that the Office 365 feature set changes over time, which can make it harder for an organization to
backup all the data in use. For instance, the lack of a suitable API to extract Teams data for backup is a problem for
tenants who use Teams. Likewise, if you enable expandable archives for Exchange Online mailboxes, you might find
that your selected backup product is unable to process this type of archive mailbox.

It’s also the case that new Office 365 features might remove some or all the stated needs for backup. For instance, the
potential impact of users removing information when they should not can be mitigated by implementing retention
policies for mailboxes, SharePoint libraries, and teams. Deploying Privileged Access Management and using Azure
Active Directory Access Role Reviews can restrict the influence that a rogue administrator might have. To reduce the
likelihood that hackers can penetrate your tenant, you can make sure that all accounts are protected with multi-factor
authentication and disable basic authentication, while Office 365 Cloud App Security or third-party products that use
the audit events generated by all workloads can highlight anomalies that need investigation.

Because technology changes at a fast cadence, especially in the cloud, the need for external backups might not be the
same today as it was a couple of years ago. For this reason, it’s wise to review your backup strategy (or rather, the
need for backups) on an ongoing basis to make sure that your organization only uses what is necessary instead of
following the dogma that often comes from on-premises systems.

Monitoring
Although Office 365 removes much of the responsibility for ongoing support and maintenance for its applications
away from administrators, the need still exists for tenants to know how well the Office 365 services function and if any
technical issues exist that might affect the availability or quality of a service at any time. Office 365 includes
mechanisms to understand the current health of its applications while third-party applications are also available to
increase and improve the quality of monitoring. Service health and other important notifications are available in the
left pane of the Office 365 Admin Portal, in the Health section.

Service Health Dashboard

The Service health menu item under the Health section of the Office 365 Admin Portal takes you to the Service
Health Dashboard (SHD). Microsoft uses the SHD (Figure 4-13 ) to communicate the current health status for each of
the services consumed by your Office 365 tenant. Each service is displayed with an icon showing whether it is
currently healthy, has an active incident that is causing an impact to customers, or has a non-incident advisory.
Figure 4-13: Service Health status for Office 365 services

Clicking on an incident displays extra information, including ongoing details of the investigation for an incident, which
may span multiple messages posted by Microsoft over a period. After Microsoft resolves an incident, they often inform
customers what happened in more detail through the publication of a post-incident review (PIR). See Chapter 2 for a
description of what you can expect to see in a PIR.

Because of the sheer scale of Office 365, you can expect that an outage is occurring somewhere in the service at any
given point in time. The service health dashboard in the Office 365 admin center is a view of problems determined by
Microsoft as potentially affecting your tenant, rather than every single issue impacting any tenant across the service.
It is also possible that you tenant experiences an issue that is not shown in the service health dashboard. Not every
issue can be detected by the automated monitoring systems used within Office 365. In some cases, multiple customers
need to report issues and escalate the issues beyond level 1 support teams before Microsoft accepts the issues as
confirmed incidents. At that point, Microsoft might add details of the incident to the service health dashboard.

Furthermore, you might be experiencing an issue with your tenant that is due to a misconfiguration on your part, not
a service fault. For example, if you misconfigure your DNS records and your Exchange Online recipients stop
receiving email, that is not something that the service health dashboard is going to alert you to because it is not a
fault with Office 365 itself. Regardless of any limitations or issues with timeliness of information the service health
dashboard is a useful resource for administrators.

Microsoft continues to develop the capabilities and intelligence behind the Service Health Dashboard. For example,
when logging a new service request the portal tries to proactively inform you of known issues, as well as notifying you
when Microsoft detects actions by your users that may lead to a support request such as clients connecting with out of
date versions of Outlook. Microsoft also uses telemetry data and machine learning to detect issues, to reduce the
amount of time between an incident occurring and the first notification posted to the Service Health Dashboard.
There is also a feedback mechanism built-in to the SHD, so that customers can rate the accuracy and usefulness of the
information that Microsoft communicates to them.

Message Center

Figure 4-14: Messages in the Message Center

The Office 365 admin center dashboard also includes a view of the most recent messages from the Message center.
The Office 365 operations team uses the Message center to notify customers of changes such as updated features and
user interfaces, as well as changes that may interrupt service such as IP address range changes. Items shown in the
Message Center are supposed to be specific to a tenant rather than the more general view of what is changing within
Office 365 found in the roadmap. To make sure that you do not miss anything important, you should select Message
Center from time to time to reveal the full set of notices (Figure 4-14 ) posted by the Office 365 operations team. If
you find something interesting in the notices, you can select Share to send the notice to someone else via email,
adding a personal message if you want. The email includes the complete text of the update notice.
Major Updates
The top panel of the Message Center features major changes to a service. Microsoft posts these updates at least 30
days before they come into effect. To decide whether a change falls into this category, Microsoft considers how the
change might affect users and the tenant with questions such as:

Does the change affect the way people work daily ? For example, the introduction of the Focused Inbox
changed the way that people deal with new email. Changes to meetings, delegations, and sharing and access
control are also in the “daily productivity” bucket.
Does the change affect how the tenant customizes Office 365 ? For example, if Microsoft changes a theme or
web part, it might affect how pages are rendered and overwrite a change made by the tenant.
Does the change increase or decrease capacity ? A good example is when Microsoft changed the default
storage quota for mailboxes (for enterprise plans) to 100 GB. Another example is the change in SharePoint per-
licensed user storage quota.
Does the change affect how Office 365 appears to users ? Any change that might affect the ability of a help
desk to support users or change how users access functionality is a major update. A minor branding change such
as the replacement of “Office 365” with “Microsoft 365” for the Admin Center is not in this category but forcing
people to use content searches instead of workload-specific searches is. Changing a URL used by an Office 365
feature also falls into this category.
Does the change introduce a new service or application to Office 365 ? Obviously, Microsoft needs to tell
tenants when they launch a new application like To-Do or Teams.
Does the change require administrative action ? For instance, the introduction of a new TLS certificate for
Exchange Online in September 2018 could impact the mail flow within a hybrid organization, so administrators
need to check out their configuration to ensure that the flow is maintained.
Does the change involve the storage location for data ? Often, applications launch with storage in the U.S. As
the roll-out proceeds across the world, storage moves into other Office 365 datacenter regions. Customers need
to be told where their data is stored, such as when Teams moved its data services into the U.K. or India
datacenter regions.

Once you see notice of a major change coming in Office 365, it’s a good idea to search for more information such as
the blogs written up by people who have tested the change or already have the change in production in their tenant.
Independent information added to Microsoft formal documentation creates a more complete picture of what you can
expect to see happen in your tenant.

Message Center Preferences

Despite Microsoft’s efforts to improve the relevance of Message center items, some of the messages appearing in your
tenant’s Message center might not be directly relevant to you. For instance, if you do not use Office 365 Mobile
Device Management, you are probably uninterested in any of those messages. You can filter the Message center
messages by clicking the Edit Message center preferences link and select the services you want to know about.
Figure 4-15: Weekly digest of Office 365 Message Center announcements

The Message center preferences include the choice to have Office 365 send you a weekly email digest of new
messages (Figure 4-15 ). Even if it is not a substitute for keeping your eyes open and looking out for information about
developments inside Office 365, the weekly digest is a practical way to stay updated with Microsoft announcements.
By default, Office 365 sends the weekly digest to the global administrator for the tenant and addresses the note to
both the primary and backup email addresses for that account. A third address is also available for use as the tenant
wishes. You can use this address to send copies of the mail digest to a distribution group, an Office 365 Group, or a
channel in a Microsoft team for other people to learn about changes inside Office 365. If you do not wish to receive
the mail digest, uncheck the boxes for one or more of the addresses and save your preferences.

If you want to track message center posts more formally, you can explore a connection between the Service Health
Dashboard and Planner where new notifications result in the creation of new tasks in a plan. The idea is that you can
then assign responsibility for tracking the impact of changes Microsoft make inside Office 365 to different
administrators. As explained in this blog , some work is needed to create the connection.

In September 2018, Microsoft began rolling out messaging capability for specific administrative roles. Originally, only
global admins could see Message Center messages, but as the new capability is deployed, users who hold roles such
as Teams administrator or Exchange administrator will start receiving messages (which they can opt out of) for the
workloads they manage.

Automatic Translation
Office 365 update messages are composed in English. Administrators who use non-English languages can have the
Office 365 Admin Center translate messages to another language by selecting their preferred language through Office
365 settings (cogwheel). If some difficulty occurs in understanding the translated version of a message, you can
choose to view the message in English selecting English from the drop-down list of available languages. This is a
dynamic choice that reverts to the language selected in settings when you navigate away from the Message Center.

Monitoring Systems
As with many Microsoft products there is a System Center Management Pack for Office 365 . Organizations that are
running System Center Operations Manager (SCOM) can create alerts and monitoring dashboards for the health of
their tenant. For tenants who do not have an existing SCOM implementation, Microsoft also offers an Office 365
management solution for the cloud-based Operations Management Suite (OMS). This solution has been available as a
public preview since May of 2016. You can add the Office 365 management solution to an existing OMS workspace or
create a new workspace. You can add multiple Office 365 tenants to one OMS workspace, which is useful for support
providers who wish to keep an eye on multiple Office 365 customers. However, an Office 365 tenant can only be
associated with a single OMS workspace, which means that if a customer is already checking their own Office 365
tenant with OMS, another party cannot also add the same tenant to another OMS workspace.

The OMS solution enables you to review Office 365 user activity, perform searches, generate email alerts, and create
reporting dashboards. If you connect OMS to on-premises data sources, you can correlate the Office 365 activity data
with other monitoring data. For example, you can associate a specific client IP address where suspicious user activity
appears to originate with other data such as missing security patches.

The Microsoft Office 365 Service Communications API is available to customers and third-party vendors to
programmatically access service incident communications. You can develop your own scripts or software solutions
based on the information available through the API or invest in a third-party monitoring solution that can do it for you.
Example of third-party monitoring products include GSX Monitor, MessageOps Monitor, Analyzer for Office 365, and
365 Command by Kaseya.

Figure 4-16: An activity report in Operations Management Suite from an Office 365 tenant

Service requests
Service requests are problem reports filed with Microsoft, although you are almost certainly going to be working with
third-party support engineers under contract with Microsoft rather than blue-badged Microsoft employees. Telephone
support is also available for Office 365 tenants if urgent problems occur. Because a lot of configuration and other
information will need to be given to support engineers, it is logical that calls should be made only by users with
administrative access to the tenant.

When you submit a service request, it is routed to a Microsoft support center covering the Office 365 region for the
tenant. The support request is examined by a support engineer who will probably call the tenant administrator to
discuss the reported problem and to look for added information to help diagnose its cause. Because of the scale of
Office 365, Microsoft follows a carefully-structured approach to gathering and checking facts about service requests.
At times, it can be frustrating for a tenant administrator when a support agent asks them what appears to be much
the same question several times or to be asked to test components that are known to be failing, but it is all part of the
process. Eventually, the problem will either be solved or escalated to second-level support (this can take several days)
and, if necessary, to the product group that supports the workload where the problem exists. Remember, support
engineers cannot make changes to code or to how Office 365 works as this kind of intervention can only happen
through a process controlled by product group. Microsoft is rightly cautious about making changes because a small
change apparently made for the benefit of a single tenant might have a ripple effect elsewhere within the service that
impacts multiple tenants. In general, support engineers can only access tenant data when you allow them to do so.
The need to preserve customer confidentiality and privacy is paramount, even if this slows things down at times. You
should mentally prepare yourself to be asked to look up and provide data that you might think the service engineers
should already have access to.

Naturally, before you add a new service request, it is a good idea to do some troubleshooting of your own as it is
entirely possible that you’ll discover the answer much faster than through the Microsoft support process. Use your
favorite search engine to check for obvious solutions and, if nothing turns up, try asking a question in a community
forum such as the Microsoft Technical Community . Don’t be lazy and expect someone else to do the work to solve
your problem. Always take the time to investigate and share information in your requests to prove that you have done
some troubleshooting for the problem and the results of your investigation. People in support forums generally want
to help, but only if you are prepared to help yourself first. Even if the support forums do not help, the information you
gather in your investigation will give invaluable background to the Microsoft support engineers who work on the
service request.

In mid-2017, Microsoft made a critical change to the way that service requests are processed. As a general rule, each
tenant administrator can now only have one active service request open at a time. Microsoft positioned this change as
a positive one, since it encourages a linear, first-in/first-out approach to fixing problems, but it’s often frustrating
because in complex environments, you may very well run into multiple issues requiring service requests at the same
time, yet you can only have one active. To work around this, you can always use multiple tenant administrator
accounts.

Adding a new service request

To add a new service request, click Support and then New service request from the Office 365 portal. You can also
view existing service requests from the same area of the portal. Any open requests are marked as “Action Required,”
meaning that Microsoft needs a tenant administrator to do something (for instance, collect some data) to allow an
open service request to be progressed. You are then asked to give some details about the problem you need help with.
This first stage of the service request process is designed to surface existing knowledge that might help you solve
your problem without needing to speak directly with Microsoft support representatives. Microsoft uses a mixture of
machine learning, telemetry that they have for your tenant, and searches against their support databases to find
potential solutions based on the text you enter. Figure 4-17 shows the interface for creating a new SR: based on the
problem description, the portal has offered a few potential solutions, and the Let us call you and Email us links
allow you to move forward with creating the case if you need to.

Figure 4-17: Creating a new service request

If none of the suggestions solve your problem, you can request a call from Microsoft so that they can help you
troubleshoot the problem, or you can send them an email. The wait time for a response is usually quite short, but it
will depend on the number of cases that the support representatives are dealing with. IT professionals love to
complain when they have been let down by a poor support experience, so you will always find stories of very long
waits for a call back from Microsoft. If the delay for a call back is too great a risk for your organization then you
should consider a Premier Support contract to get faster support.

Note that neither of these options gives you a way to expand on your original problem description with more detail:
you must create the ticket, then add notes (via email or through the portal), and there’s no guarantee that the first
engineer who contacts you will see these notes before they contact you.

When Microsoft receives a service request, they assign it a service request identifier such as 10312033. You should
give this identifier to Microsoft whenever you communicate with the support team, as it allows them to track the
progress of a service request through Microsoft’s support and escalation systems. You should also receive a message
from the support engineer assigned to the service request. The subject of this message has some tracking information
to capture the interaction between support staff and tenant administrators in Microsoft’s support databases. To keep
a record of the case as it unfolds, reply to the message whenever you have something to add or want to request an
update. It is a good idea for you to keep your own record of interactions with Microsoft just in case this data is
necessary to prove that a problem hasn’t received sufficient or timely attention.

The details that you add to a service request can be extraordinarily useful to the support engineer assigned to handle
the case. You should give information such as:

Detailed steps to reproduce the problem. If the problem surfaces in different ways, include steps for each way
that you know to reproduce the issue.
Scoping information for the problem’s impact. Does it affect one user, all users, or something in between? Are all
the affected users in the same geographic region?
If the problem occurs only in one location, is there something special about how users connect to Office 365 from
that location?
Has anything changed recently? For example, have you made any configuration changes that might be related to
the problem?
Did this functionality ever work, or did it stop working at some point?
Screenshots showing any error messages that are visible to users.
PowerShell commands and the output from those commands that illustrate the problem.
Background information such as the version number of clients (including browsers) that are affected by the
issue. For instance, the Edge and Chrome browsers can behave differently to Internet Explorer. Does the problem
occur with all browsers or just a specific browser? Can it be reproduced on multiple workstations or just one that
is running a certain version of Windows 10?
If the problem affects a hybrid component, details of the hybrid connection and other associated components
such as how directory synchronization is performed and whether single sign-on is used.

Just like in any support system, the various kinds of problems that can occur within Office 365 need different periods
to resolve. Some issues might never be resolved because they need substantial engineering investment that Microsoft
considers unjustified or unnecessary. Microsoft should resolve straightforward problems in a day or so, but to be
brutally honest, as the service grows, any problem that can’t be addressed by a simple troubleshooting script is likely
to linger. It’s very difficult to attract and retain good support engineers, and Microsoft has struggled with doing so
over the last few years. Problems that need engineering intervention, the acquisition of more detailed diagnostic data,
or are simply complex in nature will need more time. Microsoft is usually good at keeping tenant administrators up to
date with the progress of service requests through email. Updates are posted to the service request and are visible
through the Office 365 portal. And if things go quiet, you can always email the assigned engineer to ask for an update.

Gathering tenant information

While you work through an Office 365 service request with Microsoft, you might be asked to give some information
about your tenant, usually to allow the support engineer to understand what version of the software the tenant is
running. Remember that a tenant can choose to use software released to the Targeted Release or Standard Release
rings or a mixture of both. Even within these rings, Office 365 deploys software at different intervals to reach every
tenant in all Office 365 regions. It is impossible to deploy new software to everyone at the same time, so it is
important to know what configuration is active within a tenant when you meet a problem.

The Office 365 Admin Center doesn’t provide a built-in way gather and report tenant configuration, so you must use
PowerShell for this purpose. Two cmdlets are especially important. The Get-OrganizationConfig cmdlet returns
information about Exchange Online and some generic tenant data while Get-SPOTenant returns information about
SharePoint Online. You will find examples of Get-OrganizationConfig in use for different purposes in other chapters,
but we will concentrate on its use to report tenant data here. The information reported by the cmdlet changes over
time and is difficult to review on screen. It is usually best to dump the output to a text file and review it with an editor,
which will also make it possible to extract information to share with Microsoft support. In the following example, the
first command lists several important settings that might be of interest when troubleshooting a support case while the
second redirects the output to a text file.

[PS] C:\> Get-OrganizationConfig

Name : office365itpros.onmicrosoft.com
ObjectVersion : 16500

ReleaseTrack : FirstRelease

SharePointUrl : https://office365itpros.sharepoint.com/

MapiHttpEnabled : False

IsLicensingEnforced : True

IsTenantAccessBlocked : False

IsTenantInGracePeriod : False

RBACConfigurationVersion : 0.1 (15.20.218.12)

AdminDisplayVersion : 0.20 (15.20.218.12)

ServicePlan : BPOS_S_E15_0

[PS] C:\> Get-OrganizationConfig > Config.txt

For instance, settings such as the RBACConfigurationVersion and AdminDisplayVersion will tell engineers what
version of software runs inside the tenant. The ReleaseTrack setting shows if the tenant uses Targeted Release for all
or some users, while the ServicePlan setting shows the basic Office 365 plan configured for the tenant.

SharePoint Online does not report as much configuration data for a tenant as Exchange Online does, but the devil
might be in the detail when engineers are debugging a problem. Here is what the Get-SPOTenant cmdlet reveals:

[PS] C:\> Get-SPOTenant

StorageQuota : 44032

StorageQuotaAllocated : 20174

ResourceQuota : 10900

ResourceQuotaAllocated : 1900

CompatibilityRange : 15,15

ExternalServicesEnabled : True

NoAccessRedirectUrl :

SharingCapability : ExternalUserSharingOnly

DisplayStartASiteOption : True

StartASiteFormUrl :

ShowEveryoneClaim : True

ShowAllUsersClaim : True

OfficeClientADALDisabled : False

ShowEveryoneExceptExternalUsersClaim : True

SearchResolveExactEmailOrUPN : False

RequireAcceptingAccountMatchInvitedAccount: False

ProvisionSharedWithEveryoneFolder : False

SignInAccelerationDomain :

Sometimes, you might be asked for the tenant identifier. This is a unique value for the tenant used in different places
by Office 365 and Azure Active Directory. One way to retrieve the identifier is to use the Get-AzureADTenantDetail
cmdlet:

[PS] C:\> Get-AzureADTenantDetail

ObjectId DisplayName VerifiedDomain

-------- ----------- --------------

b662313f-14fc-43a2-9a7a-d2e27f4f3478 Office 365 IT Pros Office365ITPros.com

The ObjectID reported by the cmdlet is the tenant identifier.

Alternatively, you can input your tenant domain name into this web site to retrieve the identifier.

Customer lockbox
When Microsoft support personnel are working on an issue for you, they may need access to your data to resolve the
problem. These situations are rare, as most of the support operations performed by Microsoft are automated. Where
possible any support tasks that humans perform are also isolated from customer data. However, there will always be
some cases, such as when Microsoft support is trying to help a customer with problems with mailbox contents, that
access to the customer's data will be necessary.

Customer Lockbox is an extension of Microsoft's Lockbox system for managing "just in time" access to the Office 365
service infrastructure by support staff. Multiple levels of authorization are needed before support staff can gain
access, and their access is limited in scope and duration to the minimum needed for the task. Lockbox also includes
comprehensive audit logging of any activities that support staff perform.

Under normal circumstances, the support personnel receive approval from a Microsoft manager to access customer
data. Customer Lockbox is disabled by default; when it is enabled, an added approval by the customer is necessary
before support personnel can access your data. The Customer Lockbox request is sent as an email notification and can
be managed in the Office 365 admin portal in the Support section, where you can either approve or deny the request.
Approved access requests have a time limit applied to them (12 hours by default), and Office 365 removes the support
personnel's access automatically upon resolution of the issue or when the time expires.

Actions performed by Microsoft support or by the automated systems used in Office 365 are logged to the
management activity logs for the tenant. The management logs are accessed by an API, allowing third party security
monitoring systems to extract data and include it in your organizations overall reporting.

Custom Help
Tenants can create a custom help card to be displayed alongside the standard help text revealed when a user clicks
the question mark ( ? ) icon in a browser while accessing an Office 365 application. The idea is that you can provide
users with tenant-specific information to allow them to seek help if they have a problem. To create a custom help desk
card, open the Office 365 Admin Center and go to Settings and then Organization profile . Click the edit icon
beside Provide customized help desk contact info . The input screen shown in Figure 4-18 is displayed and you
can enter:

A title.

Phone number details.

Email address.
Web page.

Make sure to move the slider from Off to On and to click Save when the information is complete.
Figure 4-18: Populating custom help desk information

The next time a user signs in to Office 365 using a web browser, they see the custom help desk card when they access
help (Figure 4-19 ). Office supports a single custom help card per tenant. You can’t, for instance, have location-
specific help information presented to users. For this reason, if you manage a tenant that has users distributed in
multiple locations or countries, you should make sure that the information provided in the custom help card is
valuable for all users. For instance, it does not make sense to direct a user in Japan to a help desk in the U.S. if that
help desk is unable to speak Japanese or is only available during U.S. working hours.
Figure 4-19: How users see custom help

Custom Tiles
When users sign into the Office 365 portal, or when they click the Office 365 waffle menu (the App Launcher) from a
browser application such as OWA, a list of tiles for the applications and services to which they have access appears.
Users can select apps to appear in the launcher from their My Apps page, which lists all the apps known to the tenant.
The apps that appear in My Apps come from several sources:

Standard apps licensed from Microsoft as part of the Office 365 plan assigned to the user account. For example,
OneDrive for Business and Sway.
Apps that are developed for or by the tenant. Administrators then configure the app to assign it to individual
users. These might be line-of-business applications that are registered in Azure Active Directory or applications
written by ISVs and stored in the Azure application marketplace , such as DocuSign, DropBox, or Citrix
GoToMeeting.
Apps created with the Office 365 API Tools for Visual Studio that allow users to sign in using their Office 365
credentials. These apps automatically show up in the My Apps page.
Web apps that support single sign-on for Office 365 the Office Store . These apps are installed by users through
the Store icon in the App Launcher. For example, the Starbucks for Outlook app.

Custom tiles are available to accounts that have an Exchange Online mailbox and are the simplest and easiest way to
customize the App Launcher. Essentially, a custom tile serves as a pointer to a web page. The URL defined as part of a
custom tile can bring a user to a SharePoint site, Office 365 Group document library, external web site, or any other
page accessible through a URL. The custom tiles that are defined for a tenant are listed in the My Apps page
displayed to users and can be pinned to the App Launcher from there.

To add a custom tile, login to the Admin Center and navigate to Settings , and then Organization profile . Click
Edit in the Add custom tiles for your organization section. You can add any number of custom tiles by specifying
the name of the tile, the URL that the tile should invoke when clicked, a description (purely for administrative
purposes), and a URL for an image for the tile icon. The image must be in the JPEG format and should be sized at 50 x
50 pixels. It is easiest if the image file is stored in a SharePoint library as this means that it is accessible to Office 365.
In addition, make sure that all users have at least read-only access to the tile image. If users don’t have access to the
image file, a blank space is shown for the tile in the My Apps library.

Figure 4-20 shows the definition for a custom tile being edited (to the right) and the result in the Office 365 App
launcher (to the left). The custom tile is called “Thoughts of an Idle mind” and the URL points to a document library.
The image file is also held in SharePoint. The App Launcher contains two custom tiles, but you can see the custom tile
that’s just been created listed at the bottom of the “Admin selected apps”. Custom tiles do not automatically appear in
the Office 365 App Launcher. Users click the App Launcher and select All apps to browse the set of available tiles and
other Office 365 apps. They can then select which custom tiles they want to pin to the App Launcher.

Figure 4-20: Adding a custom tile

Hiding launcher tiles : Many administrators want to hide specific applications from users by removing the
application tiles in the app launcher for all (or some) users in the tenant. Unfortunately, there’s no way to do this. The
list of tiles the user sees is dynamically built based on the licenses you’ve granted them. Users may add and remove
tiles themselves but there’s no way for you as a tenant administrator to do this in bulk. If you really don’t want users
to use an application, instead of hiding it in the launcher and hoping they don’t stumble across it, you’ll have to
remove the license (or deactivate the application) for those users.

Reporting Portal
The Office 365 admin center includes a reporting portal with an activity dashboard and other actionable reports
(Figure 4-21 ). The reports provided in the portal give organizations a view into the adoption level for the different
services they buy with their Office 365 subscriptions. For example, the email activity report might show a healthy
amount of usage, justifying paying for Exchange Online mailboxes, while the SharePoint or OneDrive for Business
reports might show a different level of usage that prompts the organization to either reconsider the licensing of that
feature, or embark on an adoption project to encourage more use. Similarly, the Activations report lets an
organization analyze the consumption of licenses bought for their end users. For example, if you assign Visio licenses
to users who do not use them, then you can reassign the licenses to other users who need them rather than buying
more licenses. See Chapter 21 for more information about Office 365 Reports, including Microsoft’s Office 365 Usage
Analytics Power BI content pack and some third-party reporting alternatives.
Figure 4-21: The Office 365 reporting portal

Microsoft Secure Score

Microsoft acknowledges that it can be difficult for an administrator to understand how to best secure a tenant. Many
places exist in administrative consoles where you can tweak settings that affect how things work. In addition, a
multitude of data exists within applications that administrators should check on an ongoing basis. It therefore makes
sense to measure a tenant against a set of predetermined standards and score the tenant based on the actions taken
to increase security. At the same time, Office 365 can flag outstanding actions to the administrator, who then decides
whether to implement the action and so increase their tenant score. This feature, formerly known as the Office 365
Secure Score, was recently rebranded to Microsoft Secure Store. Here’s how it works.

When you go to https://securescore.office.com , you’ll see a dashboard like the one shown in Figure 4-22 . This
dashboard summarizes your overall security score, calculated based on inputs from your Office 365, Azure Active
Directory, Enterprise Management + Security (EMS), and Windows Defender Advanced Threat Protection (ATP)
services, in whichever combination you have them. As you enable more security features, your score goes up. The
dashboard includes a list of recommended actions which, if taken, will increase your score, along with a slider that
allows you to trade off score value with user impact—some security features will increase your score but will irritate
users or make it harder for them to do their jobs.

Suppose you configure Azure Information Protection to allow tenant users to protect confidential content, that adds
five points to your score. Even better, if users store documents in OneDrive for Business, adding AIP is worth ten
points. Although you can argue that OneDrive for Business is a more secure location for documents than a local hard
drive or a network file share, assigning ten points to this measurement seems like more of an encouragement to do
better. Other controls are easier to understand and more fundamental in terms of security. For instance, administrator
accounts should use multi-factor authentication and there should be less than five global administrators within a
tenant.
Figure 4-22: Viewing the Secure Score for an Office 365 tenant

Secure Score combines points awarded for the measured aspects into a tenant score, which at the time of writing, can
be up to about 1200 points. You might think that the tenant score shown in Figure 4-22 is low (182), but in fact the
average score as measured across all Office 365 tenants is just 31 (October 2018), admittedly an increase from 18 in
November 2016 and 29 in June 2017. The “Compare your score” widget in the dashboard shows your organization’s
score compared to other Office 365 organizations with a similar number of licensed seats and other organizations in
the same industry you’re in.

Reading that list, along with the list of best practices recommended for Azure AD administrators , and then making
appropriate changes can quickly boost your score. At Ignite 2018, Microsoft said that organizations that follow the
Secure Score recommendations are 30 times less likely to be breached.

Secure Score notes some actions as “Not Scored,” meaning that that if you take the recommended action, it will not
influence the tenant score now – but it might in the future when Microsoft incorporates the action into the Secure
Score assessment. Another point to consider is that some controls used in the analysis (like using an Advanced Threat
Protection safe link policy) require E5 licenses that you might not have or wish to buy. Fortunately, you can use Secure
Score’s Ignore Action feature to ignore any control that does not make sense in your environment. If conditions
change and you acquire licenses or just want to include an ignored control in the calculation, you can reenable the
control by selecting it from the list under the Ignored Action tab in the All Actions section of the dashboard and
clicking Undo .

It's also the case that the Secure Score mechanism may fail to notice and score an action you’re already taking or may
recommend actions whose true impact can’t be scored. For example, the “Review permissions & block risky OAuth
applications connected to your corporate environment” score item promises 15 points if you use Cloud App Security to
“block access to a risky OAuth app,” but the precise definition of “risky” is missing, so presumably you can earn 15
points by blocking any OAuth app.

To generate or check the score for a tenant, you can sign into the Secure Score service using an account that has been
assigned one of the administrative roles in Office 365. You can also access Secure Score through the Security &
Compliance Center. The first time you assess a tenant, Office 365 asks you to grant access for the Secure Score
service to your tenant. Assessment is not a one-time operation as Secure Score runs a daily check to figure out an
updated score, which then appears in the tenant’s Secure Score dashboard.

The Secure Score dashboard includes a Score Analyzer tab to allow administrators to:

Track progress of their score over time.

Understand the actions that contribute to the current tenant score.
Understand how they can improve their score by completing various actions. For example, a tenant score
increases by 30 points if you enable multi-factor authentication for all users while you gain 15 points if the
outbound spam policy notifies an administrator when a tenant user is blocked for suspicious activity.

Interestingly, the score data is only kept for 3 months, so if you want to see longer-term trends you’ll have to track the
scores yourself.

The engineering teams responsible for Secure Score constantly reviews threats and operational processes to ensure
accuracy and relevance. Microsoft assesses feedback from the tenants as they analyze tenant scores to understand
any gaps and weaknesses that might exist and fix the issues. The score for a tenant is likely to change over time as
Microsoft adjusts its scoring scheme and measurements. Administrators should review their tenant’s Secure Score
regularly to ensure that they leave no gaping holes for attackers to exploit.

Managing Exchange
Now that we know how to manage Office 365, it seems like we should look in more depth at some of the basic
workloads that run inside a tenant. Exchange Online seems like a good place to start, so let’s consider how to manage
Exchange in the cloud.
Chapter 5: Exchange Online
Tony Redmond

Office 365 is not just one application or workload. Rather, it is a complex ecosystem composed of multiple moving parts.
The ecosystem did not develop overnight. When Microsoft launched Office 365 in June 2011, an impartial observer could
say that the service was a loose collection of “cloudified” on-premises applications. Disjointed and uncoordinated
administrative interfaces for the different applications were the norm; the migration techniques were imperfect and
inefficient; and the client experience was no different to what on-premises servers could deliver.

Roll on to the present and the situation is very different. Although you can see a close connection between the major
Office 365 workloads and their on-premises counterparts, now the various parts of Office 365 are increasingly interlinked
and unified by new administrative tools like the Security and Compliance Center. An example of a change at a detailed
level is the switchover in SharePoint and OneDrive for Business from sending messages (like sharing invitations) from the
utility no-reply@sharepointonline.com mailbox to using the address of the sender’s Exchange Online mailbox, if one
exists. More fundamentally, new applications like Planner and Teams that do not exist in the on-premises world are
critical to the long-term success of the service. In short, the new Office 365 is a world removed from what customers
originally signed up for in 2011.

Beneath all the change and new functionality, two basic workloads are the critical underpinning for Office 365. Exchange
Online for email, and SharePoint Online (including OneDrive for Business) for document management. These workloads
are enough reason for many companies to use Office 365. The way that Microsoft uses the basic workloads as a software
parts bin to build other applications is a bonus. This chapter reviews Exchange Online and discusses how it differs from
its on-premises counterpart.

This brief chapter gives an overview of Exchange Online and discusses the differences between the cloud version and its
on-premises counterparts. We also look at the role Exchange Online fills within Office 365 and its administration.
PowerShell has been a critical part of Exchange since its introduction in Exchange 2007, so we look at how PowerShell
works with Exchange Online, including multi-factor authentication support.

Overview of Exchange Online

For many companies, email is the first workload that they move into the cloud. Given the popularity of Exchange since its
introduction in 1996, it was therefore unsurprising that many of the early Office 365 users came through migrations from
on-premises Exchange servers with more coming from migrations from other enterprise messaging systems such as IBM
(Lotus) Notes and Novell GroupWise.

Exchange Online is the cloud-based version of Exchange. Or is it? Well, both products share common roots and common
functionality, but the version of Exchange that runs inside the cloud is different to its on-premises counterpart. As we will
see, the difference is somewhat inevitable given that Exchange Online must function inside a massive multi-tenant
infrastructure while Exchange on-premises is designed to be able to serve the needs of many different customers, each of
whom might adopt a different method to deploy and manage Exchange.

An on-premises administrator who paid a visit to Exchange Online would see some very familiar things, but a lot would be
very different. The fundamentals of running a messaging system stay the same – users authenticate against Azure Active
Directory with credentials that allow them to connect to mailboxes to receive and send email. Messages exist in
databases on Exchange mailbox servers and transported to their destination via transport servers. Connectors link
servers together and to the rest of the Internet via SMTP. Database replication hums along in the background to ensure
that copies of user mailboxes are in four databases spread across at least two datacenters. Apart from the sheer size of
the infrastructure spanning over 175,000 mailbox servers (the total number exceeds 200,000) to support both Exchange
Online and Outlook.com (Figure 3-1 ), the structure of mailboxes in databases running in Database Availability Groups
spread across mailbox servers is very recognizable.
Figure 5-1: The scale of Exchange Online (source: Microsoft – Ignite 2018 conference)

It would be impossible to do justice to Exchange Online operations in just a few paragraphs (or even a complete book).
Instead, here are some of the obvious differences that an on-premises administrator might note as they work with
Exchange Online:

Compared to an on-premises organization, relatively few people have administrative permission for Exchange
Online. The operation is designed to be as automated as possible, which means that only a small number of people
have the authority and ability to make changes that might affect how Exchange Online or even a single tenant
domain behaves.
Because of the sheer scale of the service, a low tolerance exists for mistakes. The reason is that any mistake will be
horribly and gigantically magnified because of the number of moving parts. Consider this example – if you make a
mistake in a PowerShell script that makes a configuration update on every Exchange server, you might affect every
user in a 10,000-person deployment. This is bad and should not happen because all updates should be thoroughly
tested before implementation, but it is so much worse if the same mistake was to ripple across all of Office 365 and
affect hundreds of millions of mailboxes.
Exchange Online does not use backups. Instead, native data protection protects the contents of mailbox databases.
See Chapter 5 for more information on this topic.
Exchange Online used to only deploy single-role Exchange 2013 servers with dedicated systems assigned to handling
client connections, mailboxes, or transport. The Exchange 2016/2019 architecture only supports multi-role servers,
so a change has occurred here. However, Exchange 2016/2019 servers still do just one job. Microsoft has enough
hardware resources to overcome any potential reduction in hardware efficiency and resilience. Try explaining why
you need so many servers for the next refresh of your on-premises deployment to your CIO or to the purchasing
department!
Exchange Online only uses physical servers. Despite Microsoft having a solid hypervisor in Hyper-V that is used
extensively in other cloud properties, Exchange Online uses physical servers because they are easier to manage.
Virtualization is very popular in the on-premises world, but it creates another layer of management and therefore
introduces more complexity, something that you want to avoid when automation and simplification are key to your
operating philosophy.
Exchange Online uses JBOD storage throughout and depends on the native high availability features built into the
software to isolate it from the effects of failures in low-cost disks. Automated recovery from storage failures has
become a hallmark of Exchange’s high availability feature set.
Mailbox databases host mailboxes from multiple tenants. Extreme care is taken to ensure that only the mailbox
owner can access their mailbox.
Exchange Online offers users more mailbox storage than is usual on-promises. Apart from the standard 100 GB
quota assigned to users with Office 365 E3 and E5 licenses, Exchange Online supports auto-expanding archives,
meaning that an Exchange Online logical archive can be composed of multiple 100 GB auxiliary mailboxes. The
largest production archive is now well over 1 TB. Auto-expanding archives do not exist on-premises.
Exchange Online updates its massive pool of servers on a rolling basis. Up to 10% of the servers are offline at any
point. When the time comes to upgrade a server, Exchange is put into maintenance mode, messaging queues are
drained, and any active databases hosted by the servers are transitioned to other servers. The servers are then
removed from the active pool. The latest version of the operating system, any required patches, and the latest build
of Exchange are then laid down on the system disk to create a fully up-to-date server, which is then reintroduced into
the operating pool. The passive copies of databases that stayed on drives attached to the server while the upgrade
was performed are then brought back up to date using the normal log shipping and replay mechanism. Eventually
the upgraded server becomes a candidate for database activation and the update cycle completes. The wipe-and-load
technique is used because it is a much simpler process than trying to keep servers updated with Windows and
Exchange patches, service packs, and other fixes. At any point in time, Exchange Online servers run a variety of
software versions because of their position in the refresh cycle. This can account for why some tenants see slightly
 different functionality than others.
Managed Availability makes a significant contribution to the smooth running of Exchange Online. It is designed to
detect problems by analyzing the results of probes. Exchange Online has an SLA to achieve so if Managed
Availability figures out that a problem exists on a server, it will transfer workload quickly to another server to keep
the service online. On-premises administrators who get to learn about Managed Availability for the first time are
sometimes taken aback when they realize just how aggressively database copies are activated based on what seems
to be an innocuous problem, but that’s what is needed to keep a service running smoothly at scale.
On-premises Exchange includes a workload management system that is designed to achieve responsiveness for user
transactions by giving this workload priority over system operations (like moving mailboxes) that function in the
background. Exchange Online also uses workload management but it is more restrictive in terms of the throttling
that it uses to limit the amount of resources that an individual user can consume. Once again, the logic is impeccable
as workload management protects one tenant using more than it should and stops processes that might consume a
lot of resources from affecting other users.
With so many servers in use, it is inevitable that some servers will not function as smoothly as they should. When
this happens, it is more cost-effective and simpler to “shoot the server” and remove it from the service than to try to
perform a potentially complex hardware diagnosis and fix.
The nature of a multi-tenant environment means that different security rules are in place to protect user data.
Microsoft uses a "Lockbox" process to ensure that Office 365 datacenter administrators get elevated privileges to
user data only for specific tasks and limited periods. Compare this to the situation in the on-premises world where
users often keep privileges for much longer than they need them. Such a scenario is unthinkable when dealing with
confidential data belonging to many different customers. The lockbox feature is now available to tenants to allow
them to better control Microsoft access to their data and is a good example of how innovation flows from internal
solutions to customer-facing functionality.
Privileged Access Management allows tenants to implement a just-in-time and just-enough access regime for
administrative access to user data. See Chapter 4 for more information.
Exchange Online supports multi-geo capability. This is similar, but different, to the way that an on-premises
organization can deploy servers into different countries to keep data local and respect data sovereignty rules.
Exchange Online supports both traditional distribution groups and Office 365 Groups. For now, you can consider
Office 365 Groups as a modern distribution group with added intelligence and a document library.

New situations need new solutions. Exchange Online is far larger and more complex (in some ways) than any on-premises
enterprise deployment, which is why the Office 365 team has come up with some innovative ways to manage and operate
the service. You might not agree with some of their decisions, but Microsoft’s track record since 2011 shows that they are
doing well so far.

Some things do not change between cloud and on-premises. Even though OWA can be regarded as more of a feature-rich
client because it is easier for Microsoft to implement new functionality there first, Outlook for Windows remains the
single most popular desktop client connected to Exchange Online – by a factor of five! What this tells us is that cloud
users are like on-premises users in the way that Outlook is used as the fulcrum of their office life. Another interesting fact
is that Apple iOS devices are far more popular than Android and are the predominant mobile device connected to
Exchange Online. The same is also true of most on-premises deployments and it’s likely to be because many Android
devices are low-cost phones used by consumers rather than for business purposes.

Consumer Email
As noted above, Exchange Online and Outlook.com share the same physical infrastructure within Office 365. In fact,
Outlook.com user receive service from mailboxes running on Exchange Online servers and connect to their mailboxes
using a modified version of OWA. The functionality exposed in the consumer version of OWA is controlled by RBAC to
ensure that consumer accounts do not have the same level of functionality that’s available to Exchange Online users.
However, functionality developed for one platform does show up in the other. For example, the Sweep feature first
appeared in Outlook.com before Microsoft decided to make it available within Office 365. On the other hand, the Encrypt
email feature first appeared in Office 365 before it showed up in Outlook.com. Sharing the same physical infrastructure
makes it very easy for Microsoft to switch features between the consumer and business platforms.

The Future for On-Premises is in The

Cloud
Two things are very clear. First, what you see running today as Exchange Online in Office 365 is the future of the on-
premises product. The code in Exchange Online is always more recent and more developed than the code available to on-
premises customers, even in the very latest cumulative update. Although much the same basic code is used, different
code runs on the two platforms. Microsoft does transfer some features created for the cloud back to the on-premises
product, but only when the technology makes sense for on-premises customers.

Office 365 is a highly structured, standardized, and tightly managed environment. Most customer deployments of
Exchange are not particularly structured, have little in common with other deployments apart from the fact that the same
product is used, and are not managed with the same level of attention to detail that occurs for Exchange Online where
client connectivity and latency is measured to the millisecond. Then again, no customer deployment has the joint
resources of the Windows, Exchange, and other Microsoft engineering groups (including Microsoft Research) to support
them either.

Another difference between on-premises deployments and what happens inside Exchange Online is the use of third-party
products, which are generally not supported. There is good reason for this – if every customer could introduce their
favorite third-party product into their Office 365 tenant, anarchy would prevail, and Microsoft would have no chance of
being able to manage the resultant mess. All that you would have is a service prone to disruption because of the
unpredictable effects of mixing and matching. The exception to the rule are products that exploit public APIs to access
Office 365 data. No one can complain if you use a product based on the public Office 365 APIs to retrieve information
from Exchange Online or other Office 365 components. After all, that is the intention of having public APIs.

The second thing to understand is that the highly-structured nature of Office 365 lends itself to close integration between
the different components. In turn, close integration affords engineers the ability to design applications that depend on
those components in the knowledge that the integration will be available. In other words, instead of designing code that
is limited to a single application like Exchange, engineers can take a system-wide view across Office 365 and design code
that runs on that basis.

You can argue that this kind of integration is possible in an on-premises deployment too and it is true that it is entirely
possible to have all the necessary components in place so that they connect in a robust and reliable manner. However,
experience shows that it is difficult for an on-premises IT department to coordinate and implement all the necessary
products in the same regimented and structured way that Microsoft does within Office 365. For instance, the integration
necessary to make Exchange 2013 and SharePoint 2013 work well together to create functional site mailboxes was
usually difficult. This factor possibly contributed to the lack of popularity of site mailboxes and their later replacement by
Office 365 Groups and Teams, which also depend on a high degree of integration across Exchange and SharePoint. The
net result is that while on-premises deployments might aspire to the kind of integration seen inside Office 365, that result
is seldom achieved.

Another factor that must be considered is the massive resources that Microsoft can deploy to support a new feature or
product. It’s not just a matter of hardware, but also the engineering and operations talent that contributes to solutions.
Few on-premises customers would contemplate the investment necessary to support the machine learning that underpins
the Office Graph simply because no other business apart from a major cloud provider can extract a business advantage
from the investment.

The fabric that now exists within Office 365 therefore provides engineers with the confidence that the resources they
need to design and implement a new feature will be available. It is this confidence that carries through into new
applications like Teams where no plans exist for Microsoft to deliver an on-premises version in the future. A simpler
example exists in the range of reporting options and capabilities that Office 365 gives to administrators compared to the
situation that exists for on-premises software, which must content itself with whatever “do it yourself reporting” can be
cobbled together.

The Role for Exchange Within Office 365

Exchange on-premises servers sit at the center of an email-based ecosystem, tended by administrators who specialize in
Windows, Exchange, and Active Directory, and surrounded by third-party products to enhance Exchange, such as backup
utilities. The focus is entirely on Exchange.

As you’d expect, things are different in the cloud. The way Exchange functions as the email server for Office 365 is
comparable, but the focus and emphasis on Exchange as the hub of an ecosystem changes. Within Office 365, Exchange
is a provider of email and mailbox services to users and other applications. The use of Exchange-based calendars for
Office 365 Groups, Teams, and Yammer is a good example of how other applications consume Exchange functionality, as
is the way Teams captures compliance records for its conversations in user and group mailboxes. Still another example is
the way that expandable archives, a feature unique to the cloud, act as the target for import of third-party data like
legacy archives via the Office 365 Import Service. As new Office 365 applications come along, you can expect Exchange
Online to gain new customers for its services.

Exchange 2019
The competitive imperative that drives cloud providers to introduce new features on an ongoing basis means that
Microsoft dedicates less engineering resource to the on-premises version of Exchange. The upshot is that the on-premises
server is now a less functional version of its cloud counterpart. The on-premises version of Exchange is a highly-featured
email server that is more than capable of delivering a rock-solid comprehensive email service for even the largest
enterprise. Exchange has stood the test of time remarkably well in its development from Exchange 4.0 in 1996 to the
point reached today, but it cannot go any further than to be a dedicated email server. A few extra features here and there
that only run in the cloud do not change that picture.

Microsoft will release Exchange 2019 towards the end of 2018 along with SharePoint 2019 and Skype for Business 2019.
To some, these products represent the last hurrah for the on-premises base and Microsoft is only shipping the new
servers to satisfy support commitments and the need for some customers to run on-premises software because of special
circumstances (for example, a mail server running on a naval ship). Exchange 2019 includes several features developed
for Exchange Online and proven by being run inside Office 365 at cloud scale. The most notable new feature is the
metacache, an implementation of tiered storage where approximately 10% of a mailbox database is cached in SSD
storage to speed access to “hot data.” Among the benefits are faster user logins and quicker searches because the SSD
services I/O requests much faster than the mailbox databases running on JBOD. If the metacache experiences a problem,
Exchange fails over to use the traditional access path direct to JBOD.

Exchange 2019 will never be as functional as Exchange Online, but that’s understandable because both exist to serve
different purposes. The mission for Exchange 2019 is to be best general-purpose email server available anywhere. By
comparison, the mission for Exchange Online is to deliver mailbox services within Office 365, including anything from
email to storing information about files posted to SharePoint in mailboxes.

It is worth noting that the Exchange on-premises and cloud servers shared a common code base until the development of
Exchange 2019. From this point on, Microsoft maintains separate code bases to allow unique features to be developed for
each platform without interfering with the other. Although technology will be transferred between the two, the on-
premises server will not be dependent on its cloud counterpart.

Cloud Mailbox Storage Quirks

We’ve already commented that Exchange Online mailboxes offer more storage to users than is common inside on-
premises deployments. The standard 100 GB quota is supplemented by an equally-large Recoverable Items quota and, if
you enable archives, a standard archive mailbox quota of 100 GB or auto-expanding archives to whatever size is
necessary. However, these allocations are simply for user data, whether stored explicitly by the user or implicitly on their
behalf because the mailbox comes under the scope of an in-place hold. Behind the scenes, Office 365 uses mailboxes to
store information that users never see or never realize exist.

To illustrate the point, let’s use the Get-MailboxFolderStatistics cmdlet to reveal what’s stored in the Non-IPMRoot part of
a user mailbox:

[PS] C:\> $Folders = Get-MailboxFolderStatistics -Identity TRedmond -FolderScope NonIPMRoot

[PS] C:\> Folders.Count

304

[PS] C:\> $Folders[0] | Select Identity, ItemsinFolderandSubFolders, FolderAndSubFolderSize

Identity ItemsInFolderAndSubfolders FolderAndSubfolderSize

-------- -------------------------- ----------------------

TRedmond\ 451900 20.24 GB (21,727,913,262 bytes)

304 folders holding 451,900 items and occupying 20.24 GB. By comparison, here’s what the visible folders in the mailbox
contain.

[PS] C:\> $Visible = Get-MailboxFolderStatistics -Identity TRedmond -FolderScope All

[PS] C:\> $Visible.Count

92

[PS] C:\>$Visible[0] Select Identity, ItemsinFolderandSubFolders, FolderAndSubFolderSize

Identity ItemsInFolderAndSubfolders FolderAndSubfolderSize

-------- -------------------------- ----------------------

TRedmond\Top of Information Store 38360 4.907 GB (5,268,429,234 bytes)

So, 92 folders against 304 and only 38,360 items in 4.907 GB against the whopping 451,900 stored in the invisible
folders. If we look at the data in more detail and review the largest folders, we find some interesting points that help
understand why this situation exists (edited output for top 15 folders):

[PS] C:\ > $Folders | Sort ItemsInFolder -desc | Format-List Name, ItemsInFolder

Name : Audits

ItemsInFolder : 280483

Name : AllItems

ItemsInFolder : 38360

Name : NoArchiveTagSearchFolder8534F96D-4183-41fb-8A05-9B7112AE2100

ItemsInFolder : 38159

Name : Deleted Items

ItemsInFolder : 10592


Name : Calendar Version Store

ItemsInFolder : 6696

Name : Sent Items

ItemsInFolder : 6431

Name : Inbox

ItemsInFolder : 6122

Name : Unified Inbox

ItemsInFolder : 5993

Name :
OwaFV15.1AllFocusedAAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNAAuAAAAAAB+7ILpFNx8TrktaK8VYWerAQBe9CuwLc2fTK7W46L1SAp9AAAA2lHHAAA=

ItemsInFolder : 5826

Name : Files

ItemsInFolder : 5354

Name : GraphFilesAndWorkingSetSearchFolder

ItemsInFolder : 5354

Name : Calendar Logging

ItemsInFolder : 4009

Name : Contact Search

ItemsInFolder : 2988

Name : Contact Search 1

ItemsInFolder : 2988

Name : AllContacts

ItemsInFolder : 2981

Here we see:

System folders that we expect to be hidden from users. The Audits folder holds mailbox audit records (which are
also transmitted to the Office 365 audit log). In this case, the mailbox is configured for owner auditing, so large
numbers of audit records accumulate. The Calendar Logging folder holds change details for calendar items, while
Team Chat (not listed) holds Teams compliance records.
Copies of “regular” visible folders like Deleted Items , Sent Items , and Inbox . There is no obvious reason why
Exchange would duplicate folders, but it happens.
Folders used for different features. The Files folder records details of attachments and files accessed by a user (and
is seemingly duplicated by the GraphFilesAndWorkingSetSearchFolder folder). The OwaFV15.1AllFocused
folder holds items in the Focused view for the Focused Inbox (another folder called OwaFV15.1AllOther holds
items in the Other view).

Quite why there are Contact Search and Contact Search 1 folders is unknown, along with AllContacts, People I Know, and
Relevant Contacts. All of this proves a point that Office 365 administrators don’t need to worry about Exchange mailbox
storage in the same way as they would on-premises. The mailboxes are massive and what they hold away from view is
often much larger than you could ever imagine. This underscores a key point about the cloud: you don’t worry about how
a service is delivered as long as the service is delivered in the way that you expect.

Exchange Administration Center

The Exchange Administration Center (EAC) is a web-based interface (Figure 5-2 ) for administering Exchange Online. The
layout and features available in the Office 365 version of the EAC are like the consoles used to manage Exchange 2013,
Exchange 2016, and Exchange 2019, meaning that the Office 365 version is familiar to administrators who know how to
manage on-premises servers. The visibility of each section of the Exchange admin center will depend on the
administrative roles assigned to the user who has logged in.

In Hybrid deployments, the on-premises EAC supports administration of both on-premises and online objects. When you
connect to the on-premises Exchange admin center using a web browser you’ll notice that the upper left corner has both
Enterprise (on-premises) and Office 365 options available. This arrangement allows you to use a single browser session
to manage both environments. Your browser must accept cookies from both the on-premises organization and Office 365
for authentication to work correctly, which usually involves making sure that the on-premises URL and the Office 365
EAC URL are both in the Trusted Sites zone of Internet Explorer.

Managing Exchange Online doesn’t need any software to be installed on your computer, it simply uses the ability of
PowerShell to execute commands on remote servers (known as "remoting"), which is the same method used with on-
premises Exchange. Indeed, this model is reinforced in Exchange 2019 as the preferred platform for the server is Window
2019 Server Core, meaning that remote management becomes a necessity.

Figure 5-2: The Exchange Online Administration Center

Microsoft is adding some of the functions available in the EAC to the Office 365 Admin Center. For instance, you can
update the mailbox settings for an account, including setting a litigation hold. Over time, you can expect this trend to
continue and with this point in mind, it is a good idea to perform mailbox management through the Office 365 Admin
Center whenever possible. The EAC will also lose some functionality as compliance functionality like retention policies,
data loss prevention policies, and auditing becomes service-wide instead of workload-specific. This has already happened
for eDiscovery, as Microsoft has deprecated the in-place eDiscovery and Hold functionality available in the EAC in favor
of eDiscovery case management in the Security and Compliance Center. You can continue to use the eDiscovery cases
created through EAC, but the ability to create new cases will be discontinued at some time in the future. Office 365
content searches and eDiscovery cases are more functional, faster, and cover more workloads, so it is best to use these
instead of Exchange-specific eDiscovery whenever possible. See Chapter 20 for more information about Office 365
eDiscovery.

Auditing is also now best done through the Security and Compliance Center, and the Office 365 audit log ingests mailbox
and administrative audit events from Exchange Online alongside events from all the other workloads. See Chapter 21 for
information about how to interrogate the Office 365 audit log.
Managing Exchange Online with
PowerShell
PowerShell is critical to the smooth operation of Exchange Online. On-premises administrators can transfer the skills that
they accumulate working with the Exchange Management Shell to Exchange Online. However, do not expect that every
piece of script that you’ve developed to help manage on-premises Exchange will move across because a reduced set of
cmdlets is available in the cloud. The reason is logical – Microsoft takes care of a lot of the mundane work of operating
servers and databases, so you do not need access to the cmdlets used for these purposes. The cmdlets that are available
reflect the actions that you can take against different objects. The same RBAC mechanism used in on-premises
deployments to control the set of Exchange cmdlets and parameters available to a user is also used by Exchange Online.
You can check the definitions of management roles to see what cmdlets are available to each role.

If you have PowerShell version 3.0 or higher installed on your computer, then you can connect to Exchange Online by
creating a remote PowerShell session, supplying the necessary credentials, and loading the cmdlets that you need into
the session. Any Windows workstation with .NET Framework 4.5.1 or higher installed is enough to use PowerShell to
connect to Exchange Online.

Connecting to Exchange Online

To make a standard remote PowerShell connection to Exchange Online, the following steps are performed:

1. Create a PowerShell credential object.

2. Create a new remote PowerShell session to the PowerShell endpoint for Exchange Online.
3. Import the remote PowerShell session into your current PowerShell session to get access to the Exchange Online
cmdlets. When you import the session, Exchange evaluates the RBAC roles assigned to your account and adjusts the
cmdlets and their parameters so that you only see cmdlets and parameters that you can execute.

Let’s look at how to perform those steps. In a PowerShell console run the following commands:

[PS] C:\> $Credential = Get-Credential

[PS] C:\> $Session = New-PSSession -ConfigurationName Microsoft.Exchange `

-ConnectionUri https://outlook.office365.com/powershell-liveid `

-Credential $Credential -Authentication Basic –AllowRedirection

[PS] C:\> Import-PSSession $Session

Needing to run several commands to connect to Exchange Online quickly becomes tedious. To make things easier you can
create a PowerShell function to connect to Exchange Online .

Mixing Online and On-Premises Cmdlets

After connecting to Exchange Online many of the same PowerShell cmdlets you use with on-premises Exchange are
available, such as Get-Mailbox , Get-DistributionGroup , and Get-TransportRule . Because these cmdlets have the same
names inside Exchange Online as they have for Exchange on-premises, it can be a problem if you ever need to run a
series of commands, or a script, that uses both online and on-premises cmdlets. In this situation, you can use a prefix with
the Import-PSSession command to automatically prefix the Exchange Online cmdlet names with anything you choose. For
example, to add a prefix of “EXO” to the cmdlets you can add a prefix to the Import-PSSession command as follows:

[PS] C:\> Import-PSSession $session –Prefix EXO

An example of the effect of this is that the Get-Mailbox cmdlet for Exchange Online becomes Get-EXOMailbox . The Get-
Mailbox cmdlet will still work for on-premises mailboxes.

Multi-Factor Authentication for Exchange Online PowerShell

Multi-Factor Authentication (MFA) protects accounts by needing two authentication methods (password and one other)
before a process can connect. Given that administrative accounts are usually privileged, it makes sense to protect these
accounts with MFA. In fact, your Office 365 Secure Score improves if all accounts with administrative access use MFA.

Exchange Online supports MFA for PowerShell connections. The connection process is different to standard connections
and you must first install the Exchange Online Remote PowerShell module on the workstation. To install the module,
connect to EAC with the Edge browser. The choice to download the module is in the Hybrid section (even if you are not
running a hybrid configuration) and select Configure . An installer launches to download the latest build of the module
to a temporary location on your computer. A desktop icon is also created, and the application will check for updates to the
module each time you launch it. You can then connect to Exchange Online with MFA using the Connect-EXOPSSession
cmdlet:

[PS] C:\> Connect-EXOPSSession -UserPrincipalName Tony.Redmond@office365itpros.com

The normal MFA authentication sequence occurs to establish the connection and you might be prompted for the added
authentication method (for example, send a code via SMS to a nominated phone) if a suitable token is unavailable on the
workstation.

Running commands in Exchange Online with MFA is the same as a standard session: the only thing that changes is how
you authenticate. After you connect to Exchange Online with MFA, you have access to the same set of cmdlets. However,
the Exchange Online Remote PowerShell Module doesn't currently support the use of a prefix, so if you need to mix on-
premises and online commands into one session or script, you'll need to use the PowerShell remoting technique with an
account that is not MFA enabled. Since the account cannot use MFA, you should reduce the administrative rights of the
account to the minimum needed for the tasks you'll be performing.

Throttling PowerShell : Microsoft throttles the use of PowerShell within Office 365 to ensure that commands do not
absorb an excessive amount of resources. Throttles are a form of “micro delays” that cause processing to pause for a
short period. Although throttles exist for a good reason, they can be problematic when the need arises to process large
object sets, such as if you wanted to run the Get-MailboxStatistics cmdlet for every mailbox. Two techniques help to
avoid throttling. First, use the Invoke-Command method when you need to process large numbers of objects with
PowerShell. Instead of running commands locally, Invoke-Command passes the commands to Exchange Online servers
for the servers to process. Second, use filters to limit the number of objects requested from the server and the number of
properties for each object. Server-side filtering (the cmdlet provides a filter to allow the server to return only the
required objects) is much more efficient than client-side filtering (the items are all returned and then filtered with the
Where-Object cmdlet) and should be used whenever possible. See this TechNet article for details of Exchange cmdlets
and how to use filters when requesting objects.

Differences with On-Premises Exchange PowerShell

Because Microsoft manages Exchange Online, the set of cmdlets loaded into a remote PowerShell session connected to
Exchange Online is much smaller than for an on-premises session. No cmdlets are present to deal with servers,
databases, database availability groups, transport settings, and so on. On the other hand, other cmdlets that only exist in
the cloud, such as those used to manage Office 365 Groups and to search the Office 365 audit log, are loaded into
Exchange Online sessions.

PowerShell scripts developed to manage on-premises servers must be checked and tested carefully to make sure that
they work with Exchange Online. Apart from missing cmdlets, you might also find that some parameters are different
(they only work on-premises or only exist in the cloud). You might also discover that you need to load different PowerShell
modules to carry out a task. For instance, Exchange on-premises servers use Active Directory for account management
while Exchange Online uses Azure Active Directory: the cmdlet names and parameters are different. In addition, because
Exchange Online runs inside a more integrated environment, it is more likely that you will use cmdlets from modules
such as SharePoint Online or Teams. Finally, the Security and Compliance Center has its own PowerShell module. Some
of the cmdlets in the Security and Compliance Center module load into an Exchange Online session. However, these are
duplicate cmdlets loaded for convenience so that you can view information about objects like Office 365 labels that apply
across multiple workloads. If you need to add, update, or remove compliance objects, you must do so using the cmdlets
loaded from the Security and Compliance Module.

Real World: In some environments, the remote PowerShell connection to Exchange Online will fail due to a firewall or
web proxy server on the network. If a proxy is needed for access to the Internet, then you can use Michael Van
Horenbeeck’s method of adding a proxy configuration to your PowerShell function for connecting to Exchange Online.

Limited Sessions
An account can have up to three active sessions to a remote PowerShell endpoint for Office 365. The Get-PSSession can
be used to list the current sessions in use at any time. For example:

[PS] C:\> Get-PSSession

Id Name ComputerName ComputerType State ConfigurationName Availability

-- ---- ------------ ------------ ----- ----------------- ------------

1 WinRM1 outlook.offi... RemoteMachine Opened Microsoft.Exchange Available

After this point, if you need to create a new session, you must either remove a session with the Remove-PSSession cmdlet
or wait 15 minutes for one of the sessions to time-out. For example:

[PS] C:\> Remove-PSSession -Id 1

After you remove a session, the cmdlets loaded into the session are unavailable.

Blocking Basic Authentication

Basic authentication means what it says - a basic mechanism to authenticate a connection to a service. Basic
authentication is simple to use and simple to abuse, which is why attackers try to exploit its simplicity in exploits like
password spraying attacks . Exchange Online supports many different connection protocols from ActiveSync to POP3 to
IMAP4 to MAPI. In some parts, this for is historical reasons and reflects the fact that Exchange has been around since
1996. On a positive note, supporting multiple protocols allows people to use their client of choice to connect to their
mailbox. Unfortunately, the profusion of connection protocols creates a difficulty too because each must be secured to
stop penetration by attackers. To address the issue, Exchange Online supports (in preview) protocol authentication
policies to control what methods can be used to connect to mailboxes. A set of cmdlets set creates and manages the
policies.

Creating a Protocol Authentication Policy

Running the New-AuthenticationPolicy cmdlet creates an authentication policy that disables basic authentication for all
the protocols supported by Exchange Online. For example:

[PS] C:\> New-AuthenticationPolicy -Name "No Basic Auth"

RunspaceId : fd030e40-053a-404c-90f9-3cf9f2c2dcef

AllowBasicAuthActiveSync : False

AllowBasicAuthAutodiscover : False

AllowBasicAuthImap : False

AllowBasicAuthLogExport : True

AllowBasicAuthMapi : False

AllowBasicAuthOfflineAddressBook : False

AllowBasicAuthOutlookService : False

AllowBasicAuthPop : False

AllowBasicAuthReportingWebServices : False

AllowBasicAuthRest : False

AllowBasicAuthRpc : False

AllowBasicAuthSmtp : False

AllowBasicAuthWebServices : False

AllowBasicAuthPowershell : False

AdminDisplayName :

ExchangeVersion : 0.20 (15.0.0.0)

The only protocol enabled here is Log Export, which is probably not going to be used by an attacker.

Changing Protocol Settings

If you want to change a setting to allow basic authentication for a protocol, run the Set-AuthenticationPolicy cmdlet. For
example:

[PS] C:\> Set-AuthenticationPolicy -Identity "No Basic Auth" -AllowBasicAuthPop:$True

You can have multiple authentication policies in a tenant, each of which allows basic authentication for different
protocols. Before you block basic authentication, you must enable modern authentication for your tenant and be sure that
users have clients that support modern authentication, like Outlook 2016 as older clients will not be able to connect if you
disable basic authentication for protocols through a policy. See this support article for more details.

Assigning Policies to Users

After you've created the authentication policies you need, you assign them to user accounts to tell Exchange Online
whether users can connect using basic authentication. The easiest approach is to deploy a single policy to all user
accounts and implement the policy immediately, which means that you also reset the baseline for user refresh tokens. We
can do this by finding all user mailboxes and use the Set-User cmdlet to assign the authentication policy and reset the
refresh token for the account to the current date and time. This will force Exchange to request clients using basic
authentication for connections to reauthenticate using modern authentication.

[PS] C:\> Get-Recipient -RecipientTypeDetails UserMailbox -ResultSize Unlimited | Set-User

-AuthenticationPolicy "No Basic Auth" -STSRefreshTokensValidFrom $([System.DateTime]::UtcNow)

Checking Policies Are Applied to Accounts

To check that the policy is assigned to accounts, run the Get-Recipient command to fetch the set of user mailboxes and
Get-User to report the policy assigned to the account. You should see that each account is assigned the desired
authentication policy and the account’s refresh token is reset to the time when you ran the Set-User command.

[PS] C:\> Get-Recipient -RecipientTypeDetails UserMailbox -ResultSize Unlimited | Get-User | Format-Table

DisplayName, AuthenticationPolicy, Sts*

DisplayName AuthenticationPolicy StsRefreshTokensValidFrom

----------- -------------------- -------------------------

Deirdre Smith No Basic Auth 21 Oct 2018 14:30:42

Tony Redmond No Basic Auth 21 Oct 2018 14:31:06

TempAdminAC No Basic Auth 21 Oct 2018 14:31:11

Because protocol authentication policies are in preview, they don’t come with any sophisticated methods for tracking
when they are applied or how effective they are. These are the kind of finishing touches found when a feature becomes
generally available. Apart from finding out whether people use obsolete clients to connect to mailboxes, the biggest issue
you might face is that disabling basic authentication for PowerShell forces accounts to use multi-factor authentication
when they connect to Exchange Online.

Limiting basic authentication for connections using a protocol policy only affects Exchange Online and has no influence
over any other Office 365 workload.

Defining a Default Protocol Authentication Policy

New user accounts are assigned the default protocol authentication policy for the tenant. Unless you define a default
protocol authentication policy in the organization configuration, the value assigned to new accounts is $Null, meaning
that no policy is assigned. To change this, run the Set-OrganizationConfig cmdlet and define a new default:

[PS] C:\> Set-OrganizationConfig -DefaultAuthenticationPolicy "No Basic Auth"

You can check the value with the Get-OrganizationConfig cmdlet:

[PS] C:\> Get-OrganizationConfig | fl DefaultAuthenticationPolicy

DefaultAuthenticationPolicy : No Basic Auth

Moving to Mailboxes
Now that we understand how Exchange Online differs from its on-premises counterpart, we can move to consider how to
manage Exchange-related objects. Mailboxes await our attention in Chapter 6.
Chapter 6: Managing mailboxes
Tony Redmond

An email system is not much use if it cannot deliver messages to recipients and those recipients cannot interact with
the messages that arrive into their mailboxes. Early email systems only supported user mailboxes. Today’s email
systems are a lot more sophisticated and support many different mail-enabled objects designed for different purposes.
Exchange Online supports the following recipient types:

User mailboxes.
Shared mailboxes.
Room and resource mailboxes.
Site mailboxes (now deprecated).
Discovery mailboxes.
Distribution groups.
Public folders and public folder mailboxes.
Group mailboxes used by Office 365 Groups and Teams.

Exchange Online uses but does not allow tenant administrators access to system mailboxes. These mailboxes include
arbitration mailboxes, such as that used to generate the Offline Address Book, the health mailboxes used by Managed
Availability probes, and mailboxes created for system test purposes. On the other hand, the Exchange Online
mailboxes you can manage have some unique characteristics not found on-premises, many of which we will meet here.
This chapter discusses cloud user mailboxes and the many settings that you can use to control these objects.

User Mailboxes
An on-premises Exchange administrator who begins working with Exchange Online mailboxes won’t notice much
difference between cloud mailboxes and their on-premises counterparts. Obviously, you can’t manage databases or
move mailboxes around because Microsoft takes care of these activities, but the essentials of managing mailbox
properties are similar. Many of the techniques used to work with mailboxes through the EAC or PowerShell are the
same on both platforms.

Some features available on-premises are not in Exchange Online. The cmdlet extension agent is an example. This
agent often runs in on-premises deployments to automate the population of mailbox properties during the creation of
a new mailbox. For instance, you might set the time zone and language for a mailbox so that its owner has a seamless
introduction to OWA the first time they access their mailbox.

Because Office 365 is a massive multi-tenant infrastructure, it is logical that Microsoft imposes some throttles and
controls over the resources that an individual user can consume, the size of mailboxes, and the volume and type of
messages that the system handles. These limits exist to protect the integrity and performance of Exchange Online and
are documented online . Microsoft reviews the limits regularly and updates them in line with experience and user
demand. You should acquaint yourself with these limits as they might influence details of your deployment.

Data Caching : Exchange Online is a very different environment to an Exchange on-premises deployment, so it
would be unreasonable to expect that everything will work the same. Caching is an example. Sometimes a change
that you make to an object takes a few seconds – or even a few minutes – to be effective throughout Exchange Online.
It might even take some time for a new object to appear because of the need for synchronization between Microsoft
Online Services, EXODS, and Azure Active Directory. This is quite normal and is a side-effect of the caching of data to
improve performance and responsiveness within the service. The change or new object will appear eventually. Just
have faith!

Creating a New Mailbox

The Office 365 EAC does not include the ability to create a new user mailbox. Three ways to create new mailboxes
within Office 365 are to:

Create a new Office 365 cloud-only account through the Office 365 Admin Center and assign an Exchange Online
license to that account. The mailbox is available a few moments after you create the account.
Create mailboxes on-premises and synchronize them with Office 365 using a tool like AADConnect (hybrid
mailboxes).
Create a new Office 365 account and mailbox through PowerShell.

Apart from instructing Exchange to create a mailbox for an account, the Exchange Online license assigned to an
account controls some mailbox settings, such as its storage quota and access to other products such as SharePoint
Online. More information is available online about the consequences of assigning or removing licenses to or from
Office 365 accounts. If you create Office 365 accounts with PowerShell, you must make sure that the new accounts
are fully provisioned and licensed. Accounts created without a license will not be able to access some Office 365
applications until they gain a license.
Once created, you can change the settings for cloud-only mailboxes through the Office 365 Admin Center, EAC, or
PowerShell. Management of hybrid mailboxes is always performed from the on-premises environment. One major
difference between on-premises and cloud mailboxes is that Exchange Online does not apply naming policies to new
objects. In practical terms, this means that you cannot control the format of display names. If your company organizes
address lists using the last name, first name convention, you must input the display name according to that policy or
run a fix-up script afterwards to ensure that all mailboxes follow the same naming convention.

Display Names and Avatars : There are places in the Office 365 UI where avatars appear for users. If a photo is
available in the user account, Office 365 uses it. If not, initials taken from the display name serve as a fallback. For
instance, the avatar for the user Paul Robichaux will display PR. However, if you use the last name, first name
convention for display names, the avatar will display RP. The apparently “wrong” choice of initials can be hard for
users to understand.

Editing a Cloud Mailbox

Editing a mailbox-enabled user account through the Office 365 Admin Center (Figure 6-1 ) allows access to the most
commonly used mailbox properties. The Edit Exchange properties link opens a new browser window to access the
full set of properties. This is equivalent to going to EAC, selecting the mailbox, and editing its properties. It’s worth
mentioning at this point that Microsoft focuses on the Office 365 Admin Center for user and account management.
The functionality available in the EAC are becoming less important, except when it comes to dealing with settings that
are unique to Exchange.

Figure 6-1: Mailbox settings for an Office 365 account

Most of the settings and properties available for a user mailbox (Figure 6-2 ) are familiar to experienced Exchange
administrators. They control:

General settings : The user’s personal details (name, display name, initials) plus the set of 15 custom attributes
made available by Exchange Online for tenant purposes.
Mailbox usage : Gives a snapshot of the current mailbox usage against quota plus the date when the user last
logged on to the mailbox.
Contact information : The user’s address and phone numbers. This information is in Azure Active Directory.
Organization : Information about the user’s job title, department, company name, and reporting relationships.
This information is stored in Azure Active Directory.
Email address : A mailbox can have one or more email addresses. Exchange Online will accept and route
messages to the mailbox for any of the addresses. This section displays all the current email addresses and
allows you to create new addresses. Exchange Online does not apply email address policies in the same way as in
on-premises organizations, so email addresses for cloud mailboxes are more free-form. See Chapter 3 in the
Companion Volume for more information about addressing.
Mailbox features : Allows you to enable or disable protocols such as ActiveSync (EAS), MAPI, IMAP, and POP
plus client access such as OWA, OWA for Devices, and Unified Messaging. You can enable an archive mailbox
here or place the mailbox on litigation hold. Information about delivery options (forward mail to another
recipient) and restrictions are at the bottom part of the section. Finally, this section also allows control of the
various policies that apply to the mailbox:

Sharing policy: The policy controlling how to share calendar information with other external domains.
User Role Assignment policy: The policy controlling the rights granted to the user to access different user-level
functionality.
 Messaging retention management (MRM) policy: The policy controlling automatic background maintenance of
mailbox contents.
Address book policy (ABP): An optional policy that controls the address lists visible to the user.

Member of : Shows the set of distribution groups that include the mailbox.
MailTip : You can create a MailTip to display to other users when they address messages to the mailbox if they
use a client that supports MailTips (Outlook or OWA). The ActiveSync, IMAP4, or POP3 protocols used by mobile
clients do not support MailTips. Setting up a MailTip for a mailbox is an optional setting. It is all too easy to fall
into the trap of creating a MailTip that becomes outdated, such as informing senders that someone is on vacation
for a certain period. Autosignatures are usually better at informing correspondents about transient situations
while it is best to reserve MailTips for purposes such as telling people that moderation applies to messages sent
to a mailbox or group before delivery.
Mailbox delegation: Allows you to assign delegated access to the mailbox to other users. Three groupings are
available: Send As (those who can send email as if they were the mailbox owner); Send on Behalf (those who
can send mail from the mailbox while marking the message as sent on behalf of the mailbox owner); and Full
Access (those who have full access to the mailbox contents). For more information on how mailbox delegation
works, see the discussion covering delegate access in the next chapter.

Figure 6-2: Editing the properties of a mailbox with EAC

A note on mailbox aliases : As its name implies, an alias is another name or way for Exchange to recognize a
mailbox. Microsoft’s help suggests that “The user's alias is the portion of the email address on the left side of the @
symbol. It must be unique in your organization .” However, although the advice is to be unique, Exchange Online
allows you to create mail-enabled objects with the same alias. This is a habit to be avoided because it can create
confusion when processing objects, especially with PowerShell. By default, Office 365 uses the first and last name for
a mailbox separated by a full stop to create the alias (for instance, Jim.Smith ). However, you can use any form of
alias if it has no spaces or special characters and is unique within the tenant. For example, the alias assigned to the
mailbox belonging to Jim Smith might be in the form:

Jim.Smith
JSmith
SmithJim
SmithJ
JimSmith
Or even, to revert to the earliest days of email, a totally obscure value such as “X11ABC12.” Whatever format you do
choose, make sure to apply the format consistently across all accounts in the tenant.

User Role Assignment Policies

By default, Exchange Online has a single user role assignment policy to govern access to several aspects of user
functionality. In later sections, we will see how to use the policy to control whether users can create distribution
groups and add personal retention tags to their mailbox retention policy. For now, all we need to do is introduce the
concept that such a policy exists and what it does.

Exchange allows users and administrators access to features using role-based access control (RBAC), a method of
ensuring that people who need to do something have the right to do it. The user role assignment policy divides into
sections, each of which controls some aspect of functionality.

MyContactInformation : Enables users to update their contact phone numbers and address.
MyProfileInformation : Enables users to update their name information, such as their display name.
MyDistributonGroups : Controls what actions users can take with email distribution groups.
MyDistributionGroupMembership : Allows users to control their membership in groups that they join.
MyReadWriteMailboxApps : Allows the user to give read-write access to their mailbox to apps that need this
permission.
MyRetentionPolicies : Allows the user to add new personal retention tags to their mailbox.
MyBaseOptions : Controls whether the user can view and change basic mailbox information.
MyTextMessaging : Controls whether the user can change their text messaging settings.
MyVoiceMail: Allows the user to control their view mail settings.
MyMailSubscriptions : Allows the user to control settings for mail subscriptions.
MyMarketPlaceApps : Allows the user to control settings for marketplace apps.
MyCustomApps : Controls settings over custom apps.

Not every tenant uses all these settings. Some are remnants of technology that was once more important than it is
now (like text messaging). You control the features available to users by enabling sections in a policy and then
assigning the policy to their mailboxes. You do this by amending the default user role assignment policy or by creating
a new user role assignment policy and assigning that to mailboxes. For example, if you wanted to be sure that certain
users cannot change their contact and profile information through OWA Options because you want to manage this
information centrally, perhaps using information held in a HR system, you can do the following:

1. Go to the EAC, select Permissions , User Roles, and create a new user role assignment. Give the new policy a
name like “Restricted Personal Info Update” and add a suitable description.
2. Do not check the MyContactInformation and MyProfileInformation sections in the policy.
3. Check the other sections that you want to apply.
4. Save the new policy.
5. Assign the policy to target mailboxes with the Set-Mailbox cmdlet. In this case, we assign the policy to a single
mailbox.

[PS] C:\> Set-Mailbox -Identity "Jeff Guillet" -RoleAssignmentPolicy "Restricted Personal Info Update"

6. Check that the policy is effective by signing in as the user (or have them sign-in). Go to OWA Options and select
Mail, General, and then My Account. You should not be able to update contact information or upload a new user
photo to their account.

Creating a Mailbox with PowerShell

PowerShell is often used to script the creation of new user accounts and mailboxes because it allows companies to tie
the creation of an account into other processes, such as the creation of a new HR record and access records, printing
of an employee badge, and so on. On-premises administrators are familiar with the concept of creating and updating
mailboxes with PowerShell because the New-Mailbox cmdlet has been used for this purpose for well over ten years.
The New-Mailbox cmdlet exists in Exchange Online but the environment in which it functions is very different, largely
because of the multi-tenant nature of Office 365 and the need to license users before they can access functionality.
Office 365 strictly enforces the need for mailboxes to be licensed and will remove unlicensed mailboxes if they don’t
get a license within 30 days of creation. Thus, you cannot take scripts used to create mailboxes on-premises and
expect to be able to use them with Exchange Online. Invariably, some adjustment is necessary, if only to assign a
license to a new mailbox immediately following creation.

It is not the intention to have an exhaustive discussion about how to use PowerShell to create every form of Exchange
Online mailbox or how to create a process for bulk mailbox creation. A quick scan of the Internet will provide many
examples of code that you can examine and repurpose to suit your needs, including some from Microsoft (for instance,
a script to create multiple mailboxes is available as is one to assign Office 365 licenses to multiple mailboxes ).
Instead, this walkthrough will help you understand the differences that exist between Exchange on-premises and
Exchange Online.

Remember that if you run a hybrid deployment, the on-premises environment is always the master and Office 365 is
the slave. Mailboxes and user accounts are created on-premises and then synchronized to Office 365. However, even if
you run a hybrid deployment and will never create cloud-based Office 365 mailboxes, it's good to understand what
happens when a new cloud mailbox is created from scratch. The basic steps in the process are as follows:
 1. Make sure that your PowerShell session has the necessary cmdlets loaded into the session. You need both the
Exchange Online and Azure Active Directory cmdlets to create mailboxes and to manipulate the underlying Azure
Active Directory objects.
2. Run the New-Mailbox cmdlet to create a new Office 365 user object and its mailbox.
3. Run the Set-User cmdlet to update the organizational and personal settings for the user object.
4. Optionally, run the Set-Mailbox cmdlet to update the new mailbox to apply a retention policy, or update other
settings. You can also enable an archive for the new mailbox with Enable-Mailbox .
5. Run the Set-AzureADUser cmdlet to set the correct country code (location) for the new mailbox. You assign a
country to a user account to ensure that Office 365 makes the services available in that country to the user. For
example:

[PS] C:> Set-AzureADUser -ObjectId Kim.Akers@Office365itpros.com -Country Germany

6. Assign an Office 365 license to the user. See Chapter 4 for more information.

The New-Mailbox cmdlet runs in a comparable manner to on-premises Exchange. In this example, you will see that we
assign a Microsoft Online Services identity to the mailbox. The identity becomes the user principal name for the user
and ideally should match the primary SMTP address for the mailbox. The email address must belong to a domain
owned by the tenant.

[PS] C:\> New-Mailbox -LastName Akers -FirstName Kim -DisplayName "Kim Akers" -Name "Kim Akers"

-Alias "Kim.Akers" -MicrosoftOnlineServicesID Kim.Ak ers@Office365ITPros.com

-PrimarySMTPAddress Kim.Akers@Office365ITPros.com –Password (ConvertTo-SecureString –String

"Testing123!" –AsPlainText –Force) –ResetPasswordOnNextLogon $True

Creating a new mailbox with the New-Mailbox cmdlet on an on-premises Exchange server usually completes in a
matter of seconds. The same is not true for Exchange Online. Although the cmdlet and syntax are the same, the
creation of the new object across EXODS and Azure Active Directory takes some time to synchronize across the
directories.

Generating passwords with PowerShell : The need to use secure passwords that meet Active Directory policy
exists when you create new mailboxes. You can find lots of ways to approach the task of generating a secure random
password in PowerShell on the web. An excellent example is authored by Simon Wåhlin , but many other examples
are available. Always be careful to test and check the code in any PowerShell scripts that you download from the web
before using them in production.

As when creating Exchange on-premises mailboxes, some of the attributes commonly populated for mailboxes are not
set with the New-Mailbox or Set-Mailbox cmdlets. These are attributes controlled by Azure Active Directory and are
common to all user objects, whether they have a mailbox, and are updated using the Set-User cmdlet. As it happens,
many of these attributes are useful when creating filters for dynamic distribution groups or address lists, so it is
important that some attention is given to making sure that they hold the correct values. For example, the code shown
below updates the user object for the mailbox that we just created with the organizational and personal information
that you might expect to find in the corporate directory. The example shown here updates several properties,
including the Manager property with a value that points to the name of the new user’s direct manager. The update
will fail if the supplied name cannot be found in Azure Active Directory.

[PS] C:\> Set-User –Identity "Kim Akers" –City "Dublin" –CountryOrRegion "Ireland" –Department

"Marketing" –Title "VP Marketing (New Products)" –Manager "TRedmond" –Office "Dublin HQ" –Phone

"+353 1 2070888" –Company "Office 365 Books"

Once the new mailbox exists, you can update its settings with Set-Mailbox and other cmdlets. For instance, here is
how to enable an archive mailbox with the Enable-Mailbox cmdlet. When you enable an archive, Exchange Online
automatically assigns the default messaging records management retention policy to the primary mailbox. See
Chapter 19 for more information about how Office 365 and Exchange mailbox retention policies work.

[PS] C:\> Enable-Mailbox –Identity "Kim Akers" -Archive

In this example, the Set-Mailbox cmdlet updates two of the fifteen custom attributes available for mailboxes and
creates a MailTip. Custom attributes are used for many purposes, including the control of address book policies.

[PS] C:\> Set-Mailbox –Identity "Kim Akers" –CustomAttribute10 "Engineering" –CustomAttribute1 "VP"

–MailTip "Kim Akers is alive and well"

Using the extension attributes

The set of fifteen custom attributes available in Exchange Online for mail-enabled objects are well-known. What might
not be so well known is that Exchange Online also makes five "extension" attributes available for custom use with
mail-enabled recipients. There are two big differences between the two sets of attributes. The first is that the fifteen
custom attributes are visible in EAC, so you can edit a mailbox and input details into those attributes while no UI is
available to allow access to the extension attributes. The same fifteen custom attributes are available for distribution
groups and mail contacts, but again no UI is available for the extension attributes. The second difference is that the
fifteen custom attributes can store simple strings while the extension attributes can store multiple values. For
instance, if you run this command:

[PS] C:\> Set-Mailbox –Identity "Kim Akers" –CustomAttribute5 "Custom", "Attribute", "5"

Exchange Online stores the string "Custom Attribute 5" in the attribute. But if you run this command:

[PS] C:\> Set-Mailbox –Identity "Kim Akers" –ExtensionCustomAttribute5 "Custom", "Attribute", "5"

Exchange Online returns an array of "5, Attribute, Custom." And because it is an array rather than a string, we can
use the information contained in the attribute differently. For example, to add an item to the array:

[PS] C:\> Set-Mailbox –Identity "Kim Akers" –ExtensionCustomAttribute5 @{Add="Great"}

Or to remove an item:

[PS] C:\> Set-Mailbox –Identity "Kim Akers" –ExtensionCustomAttribute5 @{Remove="Attribute"}

These attributes are very useful to store information for different purposes. One note of caution. If you are
synchronizing objects from an on-premises Active Directory and use these attributes, make sure to include the
attributes in the synchronization set.

Creating a Mailbox-Enabled Office 365 Account

Running the New-Mailbox cmdlet to create a new Exchange Online mailbox also creates a new user object in Azure
Active Directory. All the account and mailbox properties are populated. However, although a mailbox is created, the
user account is not yet licensed to use Exchange Online, and the user won't be able to log-on to the mailbox. The next
step is to update the user object with an Office 365 location code for the country and then assign a license. A license
must be assigned to a new account within 30 days to avoid it being blocked and entering Office 365’s automatic
deletion routine.

The Office 365 Admin Center shows the country to which a user is assigned (Figure 6-3 ). You can use the same
country names with PowerShell, but the country code can also be passed – FR for France, DE for Germany, IE for
Ireland, and so on. A tenant can accommodate users in multiple countries and you can update a user's location as
many times as you like, and the multi-geo capability available to large Office 365 tenants allow them to distribute
mailboxes across multiple Office 365 regions. However, you can't update a tenant's location as this affects factors
such as billing and the datacenters used to hold user data.

The country name is not as not essential as the UsageLocation property, because the location controls the services
that Office 365 can provide. You can pass any value you like into the Country property as the Set-AzureADUser cmdlet
does not check the input. However, instances do exist where some limited Office 365 functionality is unavailable in a
specific country, as in the case of a voice calling plan, so it is best to be correct. In addition, when Azure Active
Directory properties are populated that might be used as the basis to construct filters to find user accounts (for
instance, to determine the membership of a dynamic group), you should use a consistent approach for country names
such as the values assigned in the ISO 3166 standard .

[PS] C:\> Set-AzureADUser –ObjectId Kim.Akers@Office365ITPros.com –Country "Ireland" –UsageLocation IE

As explained previously, it can take some time before the new account is synchronized across Office 365. You need to
take this factor into account if you plan to script the creation of new mailboxes. After we update the user object with a
valid location code, we can assign it a license. We do this by running the Set-MsolUserLicense cmdlet. See Chapter 4
for information about how to manipulate Office 365 licenses through PowerShell, including how to perform this task
using the Azure Active Directory cmdlets.

After you assign a license, we should have a licensed Office 365 user with a functioning mailbox. You can confirm the
assignment of the license to the user by examining their account through the Office 365 Admin Center (Figure 6-3 ). It
is easy and straightforward to assign licenses to accounts through the Office 365 Admin Center, especially when
dealing with the finer points of turning on or off different services licensed by a plan such as Office 365 E3 or E5.
However, it is often more efficient to use PowerShell when you want to check the licenses that are already assigned to
accounts or to reallocate licenses to accounts, as might be the case if you choose to upgrade a plan for selected users.

If you want to script the creation of a new user account, setup a mailbox, populate user attributes, and assign an
Office 365 license, you should leave an interval between running New-Mailbox and the other commands to
accommodate synchronization. This example shows the complete flow that we've discussed to date and includes code
to force a delay by checking for the existence of the Office 365 user account before trying to assign a license.
Figure 6-3: Examining the license information for an Office 365 account

[PS] C:\> Write-Host "Adding new mailbox..."

New-Mailbox -LastName "Anderson" -FirstName "Nancy" -DisplayName "Nancy Anderson"

-Name "Nancy Anderson" Alias "NAnderson" -MicrosoftOnlineServicesID Nancy.Anderson@Office365ITPros.com

-PrimarySMTPAddress Nancy.Anderson@Office365ITPros.com –Password (ConvertTo-SecureString

–String "Testing123!" –AsPlainText –Force) –ResetPasswordOnNextLogon $True

Do {

$MbxAvailable = Get-MsolUser -UserPrincipalName Nancy.Anderson@Office365ITPros.com

-ErrorAction SilentlyContinue | fw IsValid

Write-Host "." -NoNewLine

Sleep -Seconds 5

} While (!$MbxAvailable)

Write-Host "Updating mailbox attributes..."

Set-User –Identity "NAnderson" –City "Dublin" –CountryOrRegion "Ireland" –Department "Marketing"

–Title "Marketing Manager" –Manager "Kim Akers" –Office "Dublin HQ" –Phone "+353 1 2070888" –Company "Office 365 Books"

Write-Host "Updating Office 365 account..."

Set-AzureADUser -ObjectUd "Nancy.Anderson@Office365ITPros.com" –UsageLocation "IE"

–Country "Ireland"

Set-MsolUserLicense –UserPrincipalName "Nancy.Anderson@Office365ITPros.com" –AddLicenses "Office365ITPros:EnterprisePack"

Write-Host "All Done!"

The Link Between EXODS and Azure Active Directory

Many Office 365 mail-enabled objects, including user mailboxes and Office 365 groups, exist in both the Exchange
Online directory service (EXODS) and Azure Active Directory. This arrangement allows Exchange to control some
extra properties for mail-enabled objects over and above the set available in Azure Active Directory. Background
processes synchronize EXODS and Azure Active Directory. These processes use an identifier stamped on EXODS
objects to link to Azure Active Directory. You can retrieve and use the object identifier with PowerShell. For example,
this snippet retrieves the value of the ExternalDirectoryObjectId property for a mailbox and uses it to fetch
information about the Azure Active Directory user account to which the mailbox belongs.

[PS] C:\> $ObjectId = (Get-Mailbox -Identity Kim.Akers).ExternalDirectoryObjectId

[PS] C:\> Get-AzureADUser -ObjectId $ObjectId

The same technique works for Office 365 Groups. In this example, we use the ExternalDirectoryObjectId to list the
members of a group.

[PS] C:\> $ObjectId = (Get-UnifiedGroup -Identity ExchangeGoms).ExternalDirectoryObjectId

[PS] C:\> Get-AzureAdGroupMember -ObjectId $ObjectId | Select DisplayName

Mailbox Plans
When you create a new Exchange Online mailbox, the new mailbox inherits many of its settings from a mailbox plan or
template. Several mailbox plans are available within a tenant. To see which mailbox plans are available in your tenant
and to figure out which plan is the default, you can use the Get-MailboxPlan cmdlet:

[PS] C:\> Get-MailboxPlan | Format-Table Name, IsDefault

Name IsDefault

---- ---------

ExchangeOnline-12c139bc-eafa-4a43-b4d2-e285f83e075d False

ExchangeOnlineDeskless-bc1e76cc-4c0b-491c-a518-3a0a43cbf78e False

ExchangeOnlineEnterprise-8fc1c029-5e32-485e-9810-179fb4701447 True

In this instance, three mailbox plans are present in the tenant and the enterprise plan is the default. The reason why
multiple plans exist is that a tenant can include accounts that have different Office 365 licenses. As you can see by
running the Get-MailboxPlan cmdlet to examine the settings supported for a plan, many, but not all, of the standard
settings that you might want to use to configure mailboxes exist in mailbox plans. However, Microsoft restricts the
ability of the Set-Mailbox cmdlet to update plan settings, so you can only update these settings after the creation of a
new mailbox.

To update a mailbox plan, use the Set-MailboxPlan cmdlet. In this example, we update the default plan so that the
deleted item retention period is adjusted from 14 to 30 days, the largest supported message size for send and receive
operations is set to 125 MB, and a default mailbox retention policy is assigned.

[PS] C:\> Set-MailboxPlan -Identity ExchangeOnlineEnterprise-8fc1c029-5e32-485e-9810-179fb4701447

-RetainDeletedItemsFor 30.00:00:00 -MaxSendSize 125MB -MaxReceiveSize 125MB -RetentionPolicy "General Mailbox Retention Policy"

Exchange Online uses the default mailbox plan when a new mailbox is created, but you can override this and select
another mailbox plan by specifying the name of the plan in the MailboxPlan parameter of the New-Mailbox cmdlet.

If you want to disable protocol settings for new mailboxes, you can do this with the Set-CASMailboxPlan cmdlet. For
example, here we disable the POP3 and IMAP4 protocols for all new mailboxes:
[PS] C:\> Get-MailboxPlan | Set-CASMailboxPlan -PopEnabled $False -IMAPEnabled $False

We can then check the result by examining the ProtocolSettings held in each mailbox plan:

[PS] C:\> Get-MailboxPlan | Format-Table Name, ProtocolSettings

You will see a 0 (zero) against POP3 and IMAP4 and 1 against the enabled protocols, like MAPI.

Determining Mailbox Location

As explained in Chapter 1, Microsoft runs Office 365 datacenters around the world. New datacenter regions come
online on an ongoing basis to deliver enough processing power to handle demand and to serve different markets.
When you create a new tenant, you associate the tenant with a specific country. That decision then drives the choice
of Office 365 datacenter region as the home of the tenant. Depending on the region, multiple datacenters become
available to host the physical location of the mailbox databases that hold tenant mailboxes and other tenant data.
Tenants who use the Office 365 multi-geo capability can distribute mailboxes across multiple datacenter regions and
move mailboxes between those regions to meet local data regulations.

You can check on the country associated with a tenant at any time by running the Get-MsolCompanyInformation
cmdlet. For example, this tenant is in the United States. The country letter code follows the ISO standard, so you see
values such as “IE” (Ireland), AU (Australia), BE (Belgium), and so on.

[PS] C:\> Get-MsolCompanyInformation | Format-Table DisplayName, CountryLetterCode

DisplayName CountryLetterCode

----------- -----------------

Office365ITPros US

Sometimes the decision as to where Office 365 stores tenant data is very specific, as in the case of the German
sovereign cloud where data must stay inside the country and the mailboxes are in a German datacenter, and
sometimes it is regional, as in the case of New Zealand where the Australia datacenter region delivers service. As you
would expect given the utility nature of Office 365, users are generally unaware of the physical location of their data.

In all cases, Office 365 pairs the primary datacenter with a secondary datacenter to ensure that service can continue
should a datacenter outage occur. Exchange Online stores mailboxes in databases within a DAG, which distributes
database copies across at least two datacenters in the same region. If a datacenter outage occurs, Exchange Online
activates the databases in the secondary datacenter, just as it would happen with a stretched datacenter
implementation for on-premises Exchange. You can see some information about the status of a mailbox with the Get-
Mailbox cmdlet. For example:

[PS] C:\> Get-Mailbox –Identity TRedmond | Format-List Database, ServerName

Database : EURPR04DG049-db159

ServerName : vi1pr04mb3039

It is difficult to tell exactly which datacenter hosts the database because the “EUR” prefix in the database name is
non-specific. We know that the database is in the EMEA region, but the mailbox could be active on a server in any
datacenter within the region. The database name is composed of the DAG name and the database within the DAG, so
we can say that this mailbox is in the DB159 database in the EURPR04DG049 DAG. Each database has four copies
within the DAG, so we still do not know which copy is active and what server hosts that copy.

The ServerName property is a hangover from on-premises Exchange and only tells us the server where Exchange
Online originally provisioned the mailbox (you cannot control where Exchange creates a new mailbox within a region).
The first two characters of the server name tells us the datacenter owning the original server. For instance, “db” is a
server in the Dublin datacenter while “vi” belongs to the Vienna datacenter. Exchange does not update the
ServerName property when it moves mailboxes across servers within a datacenter to rebalance load or to transfer
load to a different server within a DAG, nor when databases move between regions because of a tenant relocating
when a new Office 365 datacenter region is available or if they decide to use Office 365’s multi-geo capability.

Given these limitations, it can still be interesting to understand the way that Exchange distributes mailboxes (at least
at the time of last update). This code shows that even for a small tenant, Exchange places mailboxes in all the
datacenters within a region (in this case, EMEA).

[PS] C:\> $Mbx = Get-Mailbox -RecipientTypeDetails UserMailbox -ResultSize Unlimited | Select DisplayName, ServerName

[PS] C:\> Write-Host $Mbx.Count "user mailboxes found"

[PS] C:\> $Mbx | Group {$_.ServerName.SubString(0,2)} | Select @{Name="Datacenter";Expression={$_.Name}}, Count

32 user mailboxes found

Datacenter Count

---------- -----

am 11

vi 4

he 5

db 12

Account languages
In addition to the Office 365 datacenter region to which a tenant belongs, each user account has a country setting,
created when an administrator assigns an Office 365 license to their account. The license governs the set of services
that Microsoft can deliver to the user. However, a user can select their own language and region. This is more
important when using OWA because the language selected for a mailbox controls the language used in the user
interface. It has less effect when using Outlook because the user interface is set by the version of Outlook installed on
the workstations. A user can switch their language setting for OWA at any time, which can lead to interesting results
if someone selects a language that they don't understand.

Unlike on-premises Exchange, OWA’s language setting does not control the language used by EAC. This is likely
because Office 365 has many other management interfaces that would not necessarily use the language selected in
OWA. However, if you would really like to use the EAC in a different language, you can force this by specifying the
desired language identifier in the URL used to invoke EAC. For example, to run the EAC in German, enter the
following URL:

https://outlook.office365.com/ecp?mkt=de-de

A list of language identifiers supported by the Microsoft Office applications is available online .

Administrator Mailboxes
Strictly speaking, accounts used to manage an Office 365 tenant don’t need to have an Exchange Online mailbox
unless they need to manage Office apps or perform eDiscovery searches. Nevertheless, the usual situation is to assign
mailboxes to administrator accounts so that email can be sent to them. Best practice for Exchange has always been to
perform administrative actions with specific administrator accounts created for that purpose instead of assigning
elevated permissions to “normal” user accounts. This approach has the benefit of restricting permissions to a limited
set of accounts rather than allowing for a proliferation to user accounts, sometimes for doubtful reasons. It also
ensures that you do not have to perform regular reviews of highly permissioned accounts to remove permissions from
accounts that have no need for them.

Ideally, you should protect administrator or any highly permissioned accounts by using multi-factor authentication.
This topic is explained in depth in Chapter 3.

Sending Messages with PowerShell

When you strip everything away, email and mailboxes are all about sending messages. Normally, you use a client to
compose and send messages, but it is possible to do the job with PowerShell. Being able to create and send messages
in code is often useful when the need exists to dispatch notification messages, as in the case where a script is used to
create new Office 365 accounts and their associated mailboxes, as explained earlier. The basic code to create and
send a message using the Send-MailMessage cmdlet is very simple:

[PS] C:\> Send-MailMessage –From Administrator@Office365ITPros.com –To SomeOne@Elsewhere.com

–Subject “Job is complete” –Body “Everything you asked me to do is done.” –UseSsl

–SmtpServer outlook.office365.com –Credentials (Get-Credential)

This code is enough to send a one-line message to a single address. It could be enhanced so that multiple addressees
are copied (separate the addresses with commas), by using a set of previously-obtained credentials instead calling the
Get-Credential cmdlet to prompt for a username and password, and by passing a HTML-formatted message body. BCC
and CC recipients are also supported, and it is also possible to add attachments to the outbound message. Here’s a
slightly more complicated example:

[PS] C:\> Send-MailMessage –From Administrator@Office365ITPros.com –To SomeOne@Elsewhere.com, Another@there.com –Subject “Job is
complete” –Body “<b>This is an automatically generated message</b><p> The job you asked me to do is <i><u>done</i></u>.” –UseSsl –
SmtpServer outlook.office365.com –Credentials $O365Cred –BodyAsHTML –Attachments “c:\temp\JobDone.txt”

Very importantly, the messages created using this method must be sent using the credentials of a mailbox-enabled
account. However, because PowerShell creates and sends the messages, no trace of the messages will be found in the
Sent Items folder of the user mailbox. This is probably a good thing for system-generated messages used to send
notifications or updates to users, but if you want to keep a copy for message-keeping purposes, you need to add the
mailbox that you want to use to the recipient list, probably as a BCC.

Personal Mailbox Settings

Occasionally, users need help from an administrator to update mailbox settings. Two methods can be used by an
administrator:

1. Click the user avatar in the top right-hand corner of EAC and select Another User from the menu (Figure 6-4 ).
EAC displays a picker dialog to select the mailbox that you want to manage. The selected mailbox is opened in a
browser window in a modified form of OWA Options (Figure 6-5 ). The user making the changes and the mailbox
to which the changes are applied are clearly shown. Apart from needing to be a member of at least the Recipient
Management role group, no special access rights need to be granted to the administrator to allow them to
change settings for a user in this way. It is also possible to call up OWA Options directly by inputting a link to
take you to the user mailbox. For example:

https://outlook.office365.com/ecp/Frank.Clonan@Office365ITPros.com

2. Through PowerShell. Exchange Online supports a set of cmdlets to allow administrators to manipulate different
personal mailbox settings, or options usually selected by users. You can use these cmdlets to configure a mailbox
on behalf of a user or to check that certain settings are in place. An account that performs maintenance on user
accounts through PowerShell should be assigned the Recipient Management role. If the need exists to view the
data in the mailbox, the account will need Full Access permission.

Using a browser is easiest when you only have one or two mailboxes to update. PowerShell is the preferred method
when changes need to be applied to multiple mailboxes, or when you want to update mailbox settings in a script that
creates a new mailbox.

Figure 6-4: Selecting a user’s mailbox to update their settings

Figure 6-5: Editing settings for a user mailbox using the modified form of OWA Options

To gain permission to change settings for a mailbox, run the Add-MailboxPermission cmdlet. This example grants Tony
Redmond full rights over the mailbox owned by Kim Akers:

[PS] C:\> Add-MailboxPermission –Identity 'Kim Akers' –User 'Tony Redmond' -AccessRights FullAccess

–AutoMapping $False

Note the use of the AutoMapping parameter. If omitted, Autodiscover will open the mailbox because it recognizes that
the user has full access to the mailbox. As in the situation where you are configuring a mailbox for another user, this is
probably not what you want to happen. Setting AutoMapping to $False instructs Autodiscover to ignore the mailbox
when it builds the list of resources available to Outlook. The user can edit their mailbox settings to open any extra
mailboxes that they have access rights for at any time. For more information, see the section on AutoMapping later in
this chapter. To remove FullAccess rights, run the Remove-MailboxPermission cmdlet:

[PS] C:\> Remove-MailboxPermission –Identity 'Kim Akers' –User 'Tony Redmond'

–AccessRights FullAccess

You can also add folder-level permissions to allow a user to access a folder in someone else’s mailbox. For instance,
you might want someone to check new messages that arrive when you are on vacation. In this example, we give Marc
Vigneau permission to access the Inbox folder for Kim Akers

[PS] C:\> Add-MailboxFolderPermission -Identity "Kim Akers:\inbox" -User "Marc Vigneau"

-AccessRights Reviewer

Automapping does not apply for folder-level permissions. The user who receives the permission must add the other
user’s folder as a shared folder. In addition, it can take several hours for Exchange to propagate the new permission
and make it effective.

Granting Delegate Access to a Calendar

Administrators can run the Add-MailboxFolderPermission cmdlet and Set-MailboxFolderPermission cmdlets to give
delegate access for a user’s calendar to another user. For example, this command gives delegate access to Michael
Harty for the calendar of Kim Akers. The Editor access right is needed, and the sharing permission flags tell us that
the delegate can see private items in the target calendar. In this case, SendNotificationToUser is specified to create
and send a normal sharing notification to the new delegate to tell them that they can now access someone else’s
calendar. In most cases, you will not want to generate a sharing notification as the recipient could refuse the
invitation.

[PS] C:\> Add-MailboxFolderPermission -Identity Kim.Akers@office365itpros.com:\Calendar -User Michael.Harty@office365itpros.com -

AccessRights Editor -SharingPermissionFlags Delegate, CanViewPrivateItems -SendNotificationToUser $True

FolderName User AccessRights SharingPermissionFlags

---------- ---- ------------ ----------------------

Calendar Michael Harty {Editor} Delegate, CanViewPrivateItems

To remove delegate access from a mailbox, run the Set-MailboxFolderPermission cmdlet and set the
SharingPermissionFlags parameter to None.

Regional Settings
When you create a new Exchange Online mailbox, none of its regional properties are set. These are the properties
that define the language used to interact with the mailbox, the time zone that the mailbox owner is in, and their
preferred date format. The first time someone logs into a mailbox with OWA, the client prompts them for this
information, sets the properties, and creates a set of default folders in the mailbox based on the language setting. For
example, mailboxes that use the English language have an Inbox folder, while mailboxes configured for French have
Boîte de réception . If you connect to a brand-new mailbox with Outlook desktop or a mobile client, the mailbox takes
the language setting of the client and creates the default folders based on that value, but the other regional settings
might not be set. To see the default regional configuration for a mailbox, we run this command:

[PS] C:\> Get-MailboxRegionalConfiguration –Identity 'Rob Young'

If necessary, you can run the Set-MailboxRegionalConfiguration cmdlet to tweak the regional settings. In this
example, the mailbox language, time zone, and date format match the settings for a Dutch user working in the
Netherlands. Notice the use of the LocalizeDefaultFolderName parameter, set to $True to force Exchange Online to
create the default folder names in the target mailbox using specified language:

[PS] C:\> Set-MailboxRegionalConfiguration –Identity 'Rob Young' –Language nl-NL

–TimeZone 'W. Europe Standard Time' –DateFormat ‘d-M-yyyy’–TimeFormat 'HH:mm'

–LocalizeDefaultFolderName:$True

Exchange Online is picky about the time and date formats that you specify as the formats must be valid for the
selected language. Sometimes it can be difficult to decide what is and what is not acceptable for these values, so the
most practical step is to use OWA to change the regional settings for a mailbox and then examine the values stored in
the mailbox’s regional configuration. You can then reuse those values with other mailboxes. To check what language a
mailbox uses and what effect it has on mailbox folders, you could use PowerShell code like this:

[PS] C:\> $Mbx = (Get-Mailbox -RecipientTypeDetails UserMailbox | Select DisplayName, Alias, Languages)

ForEach ($M in $Mbx) {

$Inbox = (Get-MailboxFolderStatistics -Identity $M.Alias -FolderScope Inbox)

Write-Host $M.DisplayName " has an Inbox folder called " $Inbox.Name " and their mailbox language is " $M.Languages.Name }

Auto Replies and Out of Office

The Get-MailboxAutoReplyConfiguration and Set-MailboxAutoReplyConfiguration cmdlets query and set the auto
reply settings for a mailbox. Users receive auto-reply messages when they send email to mailboxes that have an auto-
reply message configured and enabled. Exchange Online also uses the auto-reply message as a MailTip for display to
internal users when they add a user who has auto-reply enabled to a message if the client protocol supports MailTips.
To find out the mailboxes that have auto-reply set and the time the auto-reply lapses, you can use the following
command:

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox | Get-MailboxAutoReplyConfiguration |

Where {$_.AutoReplyState –eq "Scheduled" –or $_.AutoReplyState –eq "Enabled"} |

Format-List MailboxOwnerId, StartTime, InternalMessage

The first check looks for auto replies scheduled for a specific period, the second finds mailboxes where the auto-reply
is enabled without dates. The InternalMessage property reveals the HTML-formatted text of the auto-reply message
for internal correspondents (use ExternalMessage to see the text send to external correspondents).

You can use the Set-MailboxAutoReplyConfiguration cmdlet to create an auto-reply for a user who has gone on
vacation and forgotten to let anyone know. In this instance, you enable auto-reply, set a time limit, and create separate
messages for internal and external audiences. You also tell Exchange Online that auto-replies go only to external
correspondents who are known contacts for the recipient (select “All” instead of “Known” if you want the auto-reply to
go to any external correspondent who sends a message to the mailbox).

If you create an autoreply for a certain period, make sure that you set the AutoReplyState parameter to be
“Scheduled” rather than “Enabled” as failure to do this will enable the auto-reply for the mailbox at once rather than
in the future. The start and end dates should be in the locale of the user running the command. In the example shown
below, the dates are in U.S. format. 05-01 means May 1 rather than 5 January as would be the case elsewhere.
[PS] C:\> Set-MailboxAutoReplyConfiguration -Identity "Kim Akers" -StartTime "09-23-2018 19:30" -AutoReplyState "Scheduled" -EndTime
"09-30-2018 07:00" -InternalMessage "Kim Akers is attending the Microsoft Ignite event in Orlando and will respond to your message
after she returns on October 1" –ExternalMessage "Kim Akers is on vacation" –ExternalAudience 'Known'

The example auto-replies shown so far are in plain text. You can input HTML tags if you want to add a little more
emphasis to the text. For example:

[PS] C:\> Set-MailboxAutoReplyConfiguration -Identity "Kim Akers" –InternalMessage "<b>Kim is away until <i>10-Oct-2018<i> but she
will be <u>delighted </u>to respond to your note thereafter</b>”

To turn off auto-reply for a user, set the AutoReplyState property to “Disabled”:

[PS] C:\> Set-MailboxAutoReplyConfiguration –Identity "Kim Akers" –AutoReplyState Disabled

Users might continue to see an auto-reply MailTip even after it expires or is disabled. This is because Exchange
Online caches auto-reply information to prevent it from having to look up mailbox auto-reply data every time a user
adds a recipient to a message. The cached data is refreshed every hour.

OWA includes some settings to process incoming calendar requests during periods when an out of office notice is
active. These settings are not yet visible to Outlook, but because they are acted upon by the server, their effect is felt
when set by OWA. The options are:

Block my calendar for this period: Other users will see this user’s calendar as blocked out if they try to schedule
time during the period when auto-reply is active. The CreateOOF property managed by Set-
MailboxAutoReplyConfiguration controls this setting.
Automatically decline new invitations for events that occur during this period: If new event invitations arrive,
they can be automatically declined. The AutoDeclineFutureRequestsWhenOOF property controls this setting.
Decline and cancel my meetings during this period: This option scans the user’s calendar for meetings that are in
place for the period when the auto-reply will apply. Meetings that the user was invited to attend will be declined
while meetings that they set up will be cancelled. The DeclineAllEventsForScheduledOOF property controls this
setting.

Calendar Configuration
The Get-MailboxCalendarConfiguration and Set-MailboxCalendarConfiguration cmdlets manage calendar settings. For
example, this command configures the calendar to use Greenwich Mean Time (GMT) as the time zone with a starting
time for the workday of 8:30 A.M.:

[PS] C:\> Set-MailboxCalendarConfiguration -Identity "Kim Akers" –WorkingHoursTimeZone

"GMT Standard Time" -WorkingHoursStartTime 08:30:00

The cmdlet also controls the appearance of a user’s calendar when viewed through OWA (Outlook keeps its own
settings and uses the time and time zone configured for the PC). For example, this command makes Monday the first
working day of the week, starts a new year on the first day of the year, changes the default time increment from 30
minutes to 15 minutes, sets the weather unit to be Celsius, and defines the user’s location to be County Dublin,
Ireland. Figuring out the right longitude and latitude for a user’s location might seem hard, but online tools help (like
this example ) or you can experiment by inputting different locations into the Weather section of the OWA Calendar
options and noting what values are set.

[PS] C:\> Set-MailboxCalendarConfiguration –Identity "Kim Akers" –WeekStartDay Monday

–FirstWeekOfYear FirstDay –TimeIncrement FifteenMinutes –WeatherUnit Celsius

–WeatherLocations "{LocationId:9480;Name Dublin, County Dublin;Latitude:53.348;Longitude:-6.248}"

The Set-CalendarProcessing cmdlet controls how the Calendar Assistant processes calendar requests that arrive in
mailboxes. For example, this command sets the properties that control how forwarded meeting notifications and
external meeting requests are processed.

[PS] C:\> Set-CalendarProcessing –Identity "Kim Akers" –RemoveForwardedMeetingNotifications $True

–ProcessExternalMeetingMessages $True

The Set-CalendarProcessing cmdlet can do much more than manipulating the limited set of properties revealed
through OWA, especially for resource mailboxes. See TechNet for more information.

Calendar Events from Email

Exchange can create automatic calendar events from information in messages sent by airlines, hotels, booking
agencies, and other sources. When a message is delivered to a mailbox, a background agent scans the content to
figure out whether it relates to an event. If it does, Exchange creates a calendar event based on the information in the
email, such as the time and date for a flight together with the airline booking reference, the name of the departure
and destination airports, and includes the original message as an attachment. If you don’t want Exchange to create
these events for you, go to the Calendar section of OWA Options, then Events from email , and disable the entire
feature or select the events that you want to be created.
You can disable event creation with PowerShell as follows:

[PS] C:\> Set-MailboxCalendarConfiguration -Identity TRedmond -EventsFromEmailEnabled $False

If event creation is enabled for a mailbox, you can further control creation of events for specific categories. For
example, the command below enables event creation for shows, flights, hotel bookings, and rental car booking but
stops Exchange creating calendar events for dining reservations and package deliveries.

[PS] C:\> Set-MailboxCalendarConfiguration -Identity TRedmond -DiningEventsFromEmailEnabled $False

-EntertainmentEventsFromEmailEnabled $True -FlightEventsFromEmailEnabled $True

-HotelEventsFromEmailEnabled $True -PackageDeliveryEventsFromEmailEnabled $False

-RentalCarEventsFromEmailEnabled $True

Before Exchange processes an email to extract event information, Microsoft must recognize the sending organization
(like an airline). The full list of recognized senders is available online .

Automated Processing for Room Mailboxes

From November 2018, any new room mailbox is assigned a default value of AutoAccept for its AutoProcessing
calendar property (before the default was AutoUpdate) . The idea behind the change is to speed acceptance of
meeting requests for room mailboxes. If this change doesn’t fit with your corporate policies, make sure that you
update new room mailboxes after creation. For example:

[PS] C:\> Set-CalendarProcessing -Identity NewRoom -AutomateProcessing AutoUpdate

General Mailbox Configuration

The Get-MailboxMessageConfiguration and Set-MailboxMessageConfiguration cmdlets retrieve and set general
properties of a mailbox. These settings control how OWA (some are also available for OWA for Devices) behaves and
although some are also respected by Outlook, you’ll have to use a Group Policy Object to exert any real control over
Outlook settings. A basic example of how the cmdlet can be used is to define that Exchange Online should append an
autosignature to every outgoing message, insert text for the autosignature, and set the default format for new
messages to “Plain text” (rather than the default HTML):

[PS] C:\> Set-MailboxMessageConfiguration -Identity "Kim Akers" -AutoAddSignature $True

–SignatureText "From the desk of Kim Akers" –DefaultFormat PlainText

The equivalent command to create a HTML-format signature might be as shown below. In this instance, some simple
HTML code is used to create a header-style autosignature. In this case, the default font name is changed from
"Calibri", which is used if the specified font is unsupported by the browser. The font size is also increased from the
default (3 = 12 point) to 4 (16 point). The last parameter suppresses autosignatures for replies.

[PS] C:\> Set-MailboxMessageConfiguration -Identity "Kim Akers" -AutoAddSignature $True

–SignatureHTML "<h3>From the Desk of Kim Akers</h>" –DefaultFormat HTML –DefaultFontName "Verdana"

–DefaultFontSize 4 -AutoAddSignatureOnReply $False

Other common settings that you might consider updating for mailboxes include:

EmptyDeletedItemsOnLogoff : Controls whether the contents of the Deleted Items folder are emptied when
the user logs out of OWA.
AlwaysShowBCC : Display the BCC control when composing new messages.
AlwaysShowFrom : Display the From : control when composing new messages.
HideDeletedItems : Controls whether deleted items appear in conversion views. By default, clients show
deleted items so this property is set to $False. Both Outlook and OWA respect this property.
EmailComposeMode : Set to “Inline ” (default) to compose new messages in a pane within the same windows or
“SeparateForm ” to always launch a separate window for new messages.
NewItemNotification : Set to “All” (default) to instruct OWA to signal the arrival of new messages (divided into
email, fax, and voicemail) in every way that it can. Other values include “Sound” (a noise announces the arrival)
and “EmailToast ” (a pop-up “toast” notification signals the arrival).
PreviewMarkAsReadBehavior : Controls how the unread status of messages is dealt with in terms of marking
them as read. The default is “OnSelectionChange ,” meaning that a new message is marked as read if the user
selects another message. If set to "Delayed ", the item is marked as read if the user spends more than the time
specified in the PreviewMarkAsReadDelayTime property (by default, 5 seconds). You can also set this value to
Never , meaning that the unread status of a message is never altered no matter what the user does in the
preview pane.
IsReplyAllTheDefaultResponse : Controls whether reply-all is the default response for messages. If $True , a
response to a message includes all recipients (this is the default value). Set the property to $False to force OWA
to create responses only addressed to the sender of the original message.
SignatureTextOnMobile : Specifies the auto-signature text used by Outlook mobile clients.
The normal course of events is that Microsoft updates PowerShell to allow scripting control over OWA options as new
features appear in Exchange Online. The Undo Send feature, which allows a user to recall an outbound message for
up to 30 seconds after dispatch is a notable exception. No PowerShell cmdlet is available to enable or disable this
option in a mailbox or to set a value for the delay (OWA allows a choice of 5, 10, 15, or 30 seconds).

Autosignatures : Apart from allowing users to create their own autosignatures using their client of choice or using
the Set-MailboxMessageConfiguration cmdlet to create autosignatures with PowerShell, two other methods exist to
control autosignatures. First, you can use an Exchange transport rule to insert an autosignature (see Chapter 17 for
an example). This method has the advantages that it works for all clients and can use data extracted from Azure
Active Directory to populate the autosignature. Second, you can use a commercial ISV product to manage
autosignatures. Examples of ISVs that provide this service for Office 365 include Code Two Signatures , Exclaimer ,
and Outlook Signatures . ISV products cost, but they are very capable and provide the ability to manage
autosignatures much more easily than is possible with either PowerShell or transport rules.

Controlling Email Sent by Delegates

Delegate settings refer to the ability to give another user the right to access another person’s mailbox. These rights
include the ability to send messages on behalf of the mailbox owner or as the mailbox owner, or the SendOnBehalfOf
and SendAs permissions. By default, when messages are sent by a delegate using these permissions, the outbound
messages are stored in the Sent Items folder of the delegate’s mailbox. This is because the
MessageCopyForSentAsEnabled and MessageCopyForSendOnBehalfEnabled settings are set to $False.

Mailbox owners often want to have copies of messages sent for them by a delegate. To force Exchange to create
copies of these messages for the mailbox owner, set the properties to $True. For example:

[PS] C:\> Set-Mailbox -Identity TRedmond -MessageCopyForSendOnBehalfEnabled $True

-MessageCopyForSentAsEnabled $True

Similar control can be exerted over messages sent by delegates for a shared mailbox. For more information, see the
discussion in Chapter 7.

Junk Mail Settings

The Get-MailboxJunkEmailConfiguration and Set-MailboxJunkEmailConfiguration cmdlets query and set the
properties that control a user’s junk mail settings. The most interesting of these properties are the trusted senders
and blocked sender lists. Both are multivalued properties. This cmdlet generates an error if you try to run it against a
mailbox whose owner has never logged into because Exchange Online does not create the junk mail configuration for
a mailbox until after its first use. Some organizations solve this issue by forcing Exchange Online to fully provision a
new mailbox by sending it a welcome message after creating the mailbox. The welcome message usually includes
some useful information for the user such as the contact details for the help desk, pointers to other sources of help,
and so on. The following example adds an entry to the blocked senders and domains list (because we do not want to
receive messages from this domain), specifies that Exchange should always treat the user’s contacts as safe senders,
and replaces the set of trusted domains.

[PS] C:\> Set-MailboxJunkEMailConfiguration –Identity "Kim Akers"

–BlockedSendersAndDomains @{Add="Badgirls.com"} –ContactsTrusted $True

–TrustedSendersAndDomains "Microsoft.com", "Office365.com", "Outlook.com"

Maximum Message Size

Office 365 allows users to send and receive messages up to a maximum of 150 MB. The actual size of messages
supported by a mailbox are set by the MaxSendSize and MaxReceiveSize properties, which can be changed using the
Set-Mailbox cmdlet or by editing mailbox properties through the EAC (the values are available through the mailbox
features page). For example, this command sets the send and receive size to 100 MB for the Kim Akers mailbox:

[PS] C:\> Set-Mailbox –Identity 'Kim Akers' –MaxSendSize 100MB –MaxReceiveSize 100MB

Although it is great to be able to send large messages, it is quite another matter to make sure that the recipient will
be able to receive them as you probably have zero influence over the connectors and configuration of the email
systems involved in the transfer of the message after it leaves Office 365. This is especially important in a hybrid
situation as it is probable that the on-premises servers support message sizes significantly smaller than the values
supported in Office 365. It is also important to understand that the message size used for this purpose is the size of
the message after it is coded into BASE64/MIME format to allow it to be accepted by other mail systems. This process
can add up to a third to the size of a message, so a 60 MB message as seen by the user might become an 80 MB
message when presented to the transport system for transmission. In turn, this might end up exceeding the permitted
threshold and cause some bewilderment to the user.

Enabling Third Party Cloud Attachments

Working inside Exchange Online, it’s natural to use “cloudy attachments” stored in SharePoint Online or OneDrive for
Business document libraries. Your company might use other cloud-based document storage repositories like Dropbox
and you might want to allow users to add attachments to messages from these repositories. By default, Exchange
Online doesn’t allow attachments from third-party repositories. To change the behavior, you must update the
ThirdPartyFileProvidersEnabled property controlled by the OWA mailbox policy assigned to mailboxes. By default, this
property is $False . To support downloads, it must be set to $True. Here’s how to update all the OWA mailbox policies
in the tenant:

[PS] C:\> Get-OWAMailboxPolicy | Set-OWAMailboxPolicy -ThirdPartyFileProvidersEnabled $True

The ThirdPartyFileProvidersEnabled property only applies to OWA clients and has no effect on Outlook desktop or
mobile. After making the change to the policy, wait an hour or so to allow the cached policy to be refreshed. Users can
then go to the Storage accounts section (under Mail – Attachment options) of OWA options to connect third-party
storage accounts to their Office 365 accounts.

Autodiscover
Microsoft originally introduced Autodiscover to stop users from having to create a “profile” with the details of the
server and database hosting their mailbox before that they could connect MAPI clients like Outlook to Exchange.
Since its introduction, Autodiscover has proven to be extremely valuable and has expanded to give information to
clients about other resources (such the location of public folders, the OAB, and Exchange Web Services). Equipped
with this information, clients know about these resources and how to access them when needed. Clients that use
Autodiscover (desktop Outlook, Outlook for Mac, and many Exchange ActiveSync clients) can connect to Exchange
Online to retrieve all the information necessary to configure settings in the user profile. The technical details about
where their mailbox is within Office 365 are invisible to the user. Simplifying the first connection to a mailbox is an
important contribution in making it easier to onboard new users.

Figure 6-6: The XML results for a mailbox returned by Autodiscover

All a user needs to know is their Microsoft Online Services identifier (usually the same as their SMTP email address)
and password. With this information, Autodiscover can connect to Office 365, retrieve information about the services
provided by Exchange Online, and return the information to the client as an XML-formatted manifest. You can see the
contents of the manifest by using Outlook’s “Test Email Auto-configuration” function. For example, the information in
Figure 6-6 shows the set of URLs needed by Outlook to access the different services available from Exchange Online.
You do not need to include the Guessmart and Secure Guessmart Authentication methods as they only work with
IMAP4 and POP3 servers. The Autodiscover process is a little more complicated in hybrid deployments because the
first connection goes to the on-premises organization and then to Office 365. See Chapter 4 for more information on
this topic.

MAPI over HTTP and RPC over HTTP

Exchange Online enables MAPI over HTTP for all tenants as a replacement for the older RPC over HTTP protocol.
Although RPC over HTTP is now a very old protocol, forcing tenants to switchover to MAPI over HTTP means that
tenants had to upgrade Outlook clients to a version that support the new protocol. This is not an issue if you use the
click to run version of Outlook as part of Office 365 ProPlus because these clients support MAPI over HTTP. It is only a
problem if you use Outlook 2013 or unpatched versions of Outlook 2016, or Outlook 2010 or 2007 (with reduced
functionality). See the system requirements for Office 365 for more information and this page for the latest updates
for Outlook .

Microsoft originally intended to block RPC over HTTP connections from October 31, 2017. Following customer
feedback, Microsoft softened their original approach. RPC over HTTP connectivity continues (for now), but it is no
longer a supported protocol. This means that Microsoft will no longer offer support for any issue connected with RPC
over HTTP and will not release any code fix or update to resolve problems, except when those problems compromise
security. Nothing dramatic will happen when Microsoft ceases support. RPC over HTTP connections will continue to
work, and Outlook will be able to send and receive email. However, if you use Outlook 2007, your clients are
unsupported because Outlook 2007 cannot use MAPI over HTTP.

Microsoft recommends that tenants only run clients that support MAPI over HTTP. This makes sense given that no
future development will happen for RPC over HTTP, its unsupported state, and that Microsoft has no interest in the
protocol.

Connecting to Your Mailbox

You can connect to your Office 365 mailbox anywhere that a reasonable local internet connection is available.
Microsoft uses Anycast to ensure that the fastest connection is made by a client. This is particularly important for
Outlook because of the need to conduct auto discovery before the client can connect to the mailbox and other
resources such as the OAB, public folders, and so on.

When Outlook connects to Office 365, it performs a DNS lookup for Outlook.Office365.com. The DNS result returned
is dependent on the client’s physical location based on their IP address. The idea is to route the first connection to a
Client Access Server in the closest Office 365 datacenter and then use Microsoft’s high-speed internal network to
route the connection to the mailbox server that hosts the active copy of the database holding the user’s mailbox. This
approach avoids the need for extended connections across the public Internet that are more liable to interruption and
slowdown. See this page for more information about how Microsoft routes inbound connections to Office 365.

Connections to the Office 365 Portal use the same approach. However, because it tends to deal with larger amounts of
data transfer, SharePoint Online connects to the “home” datacenter for your tenant. This, plus the way that Outlook
uses background synchronization (in cached Exchange mode) to reduce the impact of network glitches, is one reason
why people might report degraded performance against SharePoint Online when they are quite happy with Exchange
Online. This blog offers some tips for optimizing network connections for Office 365 .

Inside Mailboxes
Exchange Online mailboxes hold a mixture of default, system, and user-created folders. The default folders are the set
of well-known folders like the Inbox that exist in every mailbox. Some people only ever use a small set of default
folders – Inbox, Sent Items, and Deleted Items – while others are dedicated filers and store items away in carefully-
selected folders. Colloquially, the two types of user are “pilers” and “filers.” As long as its limits are respected,
Exchange doesn’t care how data are organized in a mailbox. Limits for internal mailbox structures include:

Maximum number of items per mailbox folder: 1 million

Maximum number of items in the Recoverable Items folder: 3 million

Maximum number of subfolders per mailbox folder: 10,000 (including the root folder)
Maximum folder hierarchy depth: 300

Most won’t encounter the internal limits, but many have exceeded their storage quota.

Quotas
The size of the assigned storage quota is a major difference between on-premises mailboxes and their cloud
counterparts. Although it is still common for on-premises mailboxes to have relatively small quotas of between 2 and
10 GB, Office 365 assigns a basic 100 GB quota to mailboxes with enterprise E3 and E5 plans, 50 GB to E1 and
education plans, and 2 GB to frontline worker plans. Competitive pressure is one reason why Microsoft offers very
large mailbox quotas within Office 365. A free Gmail account comes with a quota of 15 GB while a G Suite paid
account comes with 30 GB (Gmail, Google+, and Google Drive share this quota). Table 6-1 lists the quotas assigned to
different forms of mailboxes as of January 2018.

Office 365 Frontline Enterprise E1 and equivalent Other enterprise plans (Office
worker (F1) government and academic plans 365 E3 and E5)

Primary 2 GB 50 GB 100 GB
mailbox size

Archive N/A 100 GB Unlimited

mailbox size

Shared N/A 50 GB 50 GB
mailbox size
Resource 10 GB 50 GB 100 GB
mailbox size

Group 50 GB 50 GB 100 GB
mailboxes

Public folder N/A 50 GB 100 GB

mailboxes

Table 6-1: Office 365 mailbox sizes

If you license a shared mailbox, you can increase its quota to 100 GB and gain the added benefit of being able to
create an archive for the mailbox. In the past, Exchange assigned some unlicensed shared mailboxes a 100 GB quota.
These mailboxes can keep the erroneous quota as long as their status does not change (for example, they are not
converted to a user mailbox and back again). New shared mailboxes receive a 50 GB quota.

You do not have to assign the full quota to mailboxes and can restrict users to lower amounts. As you can see in the
PowerShell example above, three properties control how Exchange applies a mailbox quota.

The IssueWarningQuota property tells Exchange the point at which nagging messages should be sent to the
mailbox owner to tell them that they are approaching the quota limit.
The ProhibitSendQuota property marks the limit at which Exchange will no longer accept new outbound
messages from the mailbox.
The ProhibitSendReceiveQuota property tells Exchange when to cut off both outbound and inbound service to the
mailbox.

For example, this code sets a warning limit of 23 GB, stops the user from sending messages at 25 GB, and stops the
mailbox receiving messages at 30 GB.

PS] C:\> Set-Mailbox -Identity TRedmond -ProhibitSendQuota 25GB -ProhibitSendReceiveQuota 30GB

-IssueWarningQuota 23GB

You can scan for user mailboxes that have a certain quota and increase their quota with a command like:

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox -ResultSize Unlimited |

? {$_.ProhibitSendQuota -lt 70GB } | Set-Mailbox -ProhibitSendQuota 99GB -ProhibitSendReceiveQuota 100GB -IssueWarningQuota 98GB

In addition to their basic quota, mailboxes also have a recoverable items quota of 30 GB, which is automatically
increased to 100 GB if the mailbox is put on hold. The added quota accommodates the need to keep held items in
mailboxes for extended periods. If necessary, it is possible to increase the recoverable items quota past 100 GB, but
only by filing a support request with Microsoft.

Together, the total available for user storage is between 250-300 GB made up from the mailbox, primary archive, and
recoverable items. As explained later, Exchange Online can expand archive mailboxes in a practically unlimited
method by chaining together 50 GB “mailbox chunks,” meaning that it is possible to keep very large quantities of data
online. In addition, Exchange Online holds system data in hidden folders within mailboxes that is inaccessible to
users. The amount of system data can exceed user data, meaning that the overall storage occupied by a mailbox can
be much larger than anyone expects.

Moving very large on-premises mailboxes to Office 365 : Enterprise plans assign large mailbox quotas to
accounts, which is more than enough for most users. However, there are examples where on-premises mailboxes are
well past 50 GB with some mailboxes being over 100 GB in size. You cannot move these mailboxes to Office 365
without first shrinking them to under 45 GB because the Mailbox Replication Service will not process a mailbox that
is over this size. Asking a user to prune such a massive mailbox is impossible, so the best idea is to create an archive
mailbox (if one does not already exist) and use a retention policy to move as much information as possible from the
primary mailbox into the archive. If the mailbox has any items larger than 150 MB (the maximum supported by
Exchange Online), you’ll need to remove or shrink them before movement is possible. Once you’ve got the mailbox
under 45 GB, it’s all set to move to the cloud.

Offline Storage
In a practical sense, once a quota is past 20 GB, adding more storage is nearly meaningless for the average user
because it will take them so long to fill their mailbox, even if they move large amounts of old mail from their on-
premises mailbox. Modern online clients will not notice much difference when connecting to large mailboxes because
these clients can deal with folders that can hold tens of thousands of items. Offline clients can also control the amount
of information that they download from a mailbox.
Figure 6-7: Configuring the Outlook 2016 slider to control synchronization

Both Outlook 2013 and Outlook 2016 include the OST “slider” (Figure 6-7 ) to allow the user to choose a
synchronization period ranging from the last month to everything for Outlook 2013 or, as shown, from just 3 days
for Outlook 2016. Outlook uses the synchronization period to decide what to download from the user-visible
folders in the mailbox on the server to their offline folder replica file (OST). The smaller synchronization period
introduced in Outlook 2016 is to support virtual desktop clients by reducing the amount of synchronized data. In
the case of laptop and other non-shared PCs, restricting the synchronization period to a year or less is an
effective way to ensure that the OST stays a reasonable size (in the 4 GB to 10 GB range, depending on the
number of items synchronized to the OST). The performance of larger OST files can suffer even on PCs equipped
with fast SSDs.
If configured for offline access, OWA only downloads the last three days of content (or 150 items, whichever is
larger) for up to 20 folders (if the user accesses more than 20 folders in that time, OWA downloads Inbox and
Drafts plus the 18 other most recently accessed folders). No attachments are downloaded. OWA also downloads
items for the prior month, the current month, and the future year for your calendar plus all your contacts.
Registry settings are available that can be set by group policy or manually to control the “slider window” and
whether Outlook 2016 synchronizes the contents of public folder favorites and shared mailboxes.
Most mobile clients offer controls over the folders synchronized to the device as well as the synchronization
period.
Outlook needs to run in cached Exchange mode to access Office 365 Groups (see Chapter 11 for details).

Tenants considering a virtual desktop infrastructure (VDI) deployment need to pay particular attention to the amount
of data cached by Outlook and perhaps consider OWA as a replacement client in these environments. If Outlook must
be used, it’s a good idea to consider VDI-specific solutions like the FSLogix Office 365 Container (acquired by
Microsoft in November 2018) to achieve good performance for users.

Mailbox Cleanup
As noted above, it can be difficult for a user to even begin to know how to start cleaning up a large mailbox. To help,
OWA includes the Mailbox Cleanup feature to scan for and remove older items. To begin a cleanup, go to the Mail
section of OWA options and select Clean up mailbox . A link to the same choice is available in the My account
section under General options.
Figure 6-8: Mailbox cleanup

Cleanup begins by listing folders in the mailbox, headed by Deleted Items and Junk Email, which are natural
candidates for cleanup. The other mailbox folders follow. Select a folder and click the wastebasket icon to reveal the
cleanup options. You can remove all the items in the folder or items older than 3 months, 6 months, or 12 months.
Click Delete email to go ahead.

OWA moves the selected items into the Recoverable Items folder, where they stay until the deleted items retention
period assigned to the mailbox elapses. During this period, the user can recover items, but once the retention period
elapses, the items are irrecoverable.

Folder Associated Information

Administrators sometimes comment that the number of items reported in a folder by a client differs from the server-
side data that they see when running a cmdlet like Get-MailboxFolderStatistics . The difference is the hidden items
(folder associated information or items, known also as FAIs) that Exchange stores in folders for different purposes
such as to hold configuration settings (like the town chosen for a weather display in Outlook’s calendar), RSS feeds,
and retention policies. The FAIs are usually small and do not take up much space in the context of an overall mailbox,
but they are very important to Exchange and its clients. The Inbox folder is the location for most FAIs. A test using
revealed that Outlook reported 8,734 items in the Inbox, but when I ran the following command:

[PS] C:\> Get-MailboxFolderStatistics –Identity TRedmond –FolderScope Inbox | Select ItemsInFolder

The cmdlet reported the number of items to be 9,585. The FAIs make up the difference, which you can see by using
the MFCMAPI utility to examine the associated items table for the folder.

Hidden Mailbox Items

A mailbox divides into items accessible to users and items hidden in folders that clients never reveal. Apart from FAIs,
Exchange stores information in hidden folders for different purposes. The Recoverable Items structure is a good
example. A browse through the folders reported by the Get-MailboxFolderStatistics cmdlet exposes a couple of other
hidden folders including PersonMetaData, RecipientCache, and Files. The last is most interesting because it holds the
metadata of individual files that you interact with in Office 365, such as:

Attachments to email messages that you receive.

Documents in your OneDrive for Business site.
Documents from other SharePoint sites that people have shared with you.

These items exist in your mailbox to support fast search and access for the Office Graph. The last time I checked, my
mailbox had 422.6 MB of data in 4,033 items in the Files folder. Microsoft does not usually document how Exchange
or Office 365 use the items in hidden folders. After all, the items are for internal purposes and you cannot access them
through clients, but the increasing use of user mailboxes to hold information like this does increase in their ever-
swelling size.
System Folders
System folders exist to hold data for application purposes. On-premises mailboxes have far fewer system folders than
cloud mailboxes do because Office 365 applications use Exchange Online mailboxes to store a broader range of
information. For example, Teams stores its compliance records in the Team Chat folder in group mailboxes (for
channel conversations) and user mailboxes (for personal chats). System folders are usually hidden and do not appear
in clients.

The Files System Folder

The Files folder is a good example of a cloud-specific use of a system folder. Files holds information about files that
the user might be interested in, including attachments that received in email, “cloudy attachments” added to
outbound messages or received inbound, hyperlinks in email, and files from SharePoint Online and OneDrive for
Business that the user recently worked with. Files is hidden, so the only way to see the information is using a program
like MFCMAPI. However, you can get some idea of how much data Files holds by running the Get-
MailboxFolderStatistics cmdlet. The storage used by Files is not charged against a user’s mailbox quota.

[PS] C:\> Get-MailboxFolderStatistics -id TRedmond | ? {$_.Name -eq "Files"} | Select Name, ItemsInFolder, FolderSize

Name ItemsInFolder FolderSize

---- ------------- ----------

Files 5208 634.5 MB (665,287,869 bytes)

An example of how clients use this data is when you select Files for an Office 365 group in OWA. The information
presented comes from the Files folder in the group mailbox. Another example is the “Your recent documents” list
presented in Delve, which you can view by documents and attachments, just attachments, or just documents. The data
held in Files is also used when you view someone’s People card to show the email and documents shared between you
and that person. In all cases, the data comes from multiple Office 365 sources and it makes sense to extract and hold
copies of that data in Files for fast and efficient access.

Office 365 applications generate views about recent user activity with the Insights API , part of the Microsoft Graph..
The API makes calculated insights into what users do available to applications. Background assistants update the Files
folder based on the insights to make local copies of information available for faster access. Other data in Files comes
from local interaction with Office applications. The same assistants clean out old data as it is removed from the
underlying stores either through deletion or because the user loses access.

The Files system folder only exists in cloud mailboxes. You cannot have a user-created folder with the same name as a
system or default folder. This creates the possibility that you might move some on-premises mailboxes to Exchange
Online that already have a Files folder. When this happens, the background agents will hide the user-created Files
folder. The data is still there; it’s just hidden from view.

To fix the problem, you can run the MFCMAPI utility to reset the PR_ATTR_HIDDEN property to False to force
Exchange to reveal the folder again. This is only a temporary condition as the background agent will eventually detect
that the Files folder is viewable and reset the property to hide the folder again. However, the mailbox owner will have
the opportunity to create a replacement folder and move the items there.

The Big Funnel Mailbox Index

The “Big Funnel” is the replacement content indexer to the Search Foundation used by on-premises Exchange servers.
The major difference between the two technologies is that the Search Foundation indexes items on a database level
while Big Funnel does so for an individual mailbox and stores its indexes as hidden items within the mailbox. Holding
indexes within the mailbox makes searches faster and more precise; it also means that should a database failure
occur, the activation of another database copy cannot be stopped by the need to update an index as the index is
always updated in the copy of the mailbox within the database. To see information about the indexes within a mailbox,
use the following command (output edited for space):

[PS] C:\> Get-MailboxStatistics -Identity TRedmond | Format-List *funn*

BigFunnelIsEnabled : True

BigFunnelUpgradeInProgress : False

BigFunnelMessageCount : 321633

BigFunnelIndexedSize : 6.021 GB (6,464,565,475 bytes)

BigFunnelPartiallyIndexedSize : 124.5 MB (130,597,574 bytes)

BigFunnelNotIndexedSize : 3.816 KB (3,908 bytes)

BigFunnelCorruptedSize : 0 B (0 bytes)

BigFunnelShouldNotBeIndexedSize : 417.7 MB (437,972,533 bytes)

BigFunnelIndexedCount : 321487

BigFunnelPartiallyIndexedCount : 141

BigFunnelNotIndexedCount : 1

BigFunnelCorruptedCount : 0

Figuring Out the Last Login for a Mailbox

One of the pieces of information reported for a mailbox by the Get-MailboxStatistics cmdlet is the last logon time. It is
easy to assume that this is a timestamp noting the last time that someone logged onto the mailbox, and indeed, this is
exactly what the property was when the cmdlet first appeared in Exchange 2007, which is why you find so many
PowerShell scripts that examine the LastLogonTime property to figure out when someone last logged onto their
mailbox. For instance, this code finds all user mailboxes and outputs the LastLogonTime and
LastLoggedOnUserAccount properties. The latter is always blank because Microsoft deprecated it in Exchange 2013,
but you might still find some example scripts based on its use.

[PS] C: \> Get-Mailbox -RecipientTypeDetails UserMailbox | Get-MailboxStatistics | Format-Table DisplayName, LastLogonTime,

LastLoggedonUserACcount -AutoSize

DisplayName LastLogonTime LastLoggedOnUserAccount

----------- ------------- -----------------------

Deirdre Sith 25-Feb-18 3:43:36 PM

Tony Redmond 25-Feb-18 3:40:46 PM

TempAdmin 25-Feb-18 1:37:48 PM

Jeff Guillet 25-Feb-18 1:04:53 PM

The issue is that the timestamp reports the last time any process connected to the mailbox. When the cmdlet first
appeared, it was reasonable to expect that only users connected to mailboxes, so the timestamp was accurate in most
cases. Today, the number of mailbox assistants running inside Exchange Online means that you can expect that one or
more processes, like the Mailbox Folder Assistant, will connect to a mailbox daily. Any report or query based on the
timestamp returned by Get-MailboxStatistics is therefore inaccurate.

If you want to know when users last logged on, running queries against the Office 365 Audit Log (see Chapter 21) give
the best results. For instance, this code searches the audit log for the last 90 days to look for two types of records:

UserLoggedIn comes from Azure Active Directory and tells us when the user account last signed into an Office
365 service.
MailboxLogin comes from Exchange Online mailbox auditing and tells us the last time the user signed into the
mailbox. Two problems make these records less dependable than user logins. First, mailbox auditing must be
configured for mailboxes and the MailboxLogin event must be included in the Owner configuration. Second, the
process to ingest Exchange Online audit data into the Office 365 audit log is much slower than Azure Active
Directory events, which means that you might not see the events for some time after they occur.

Combining user login and mailbox login events in the search gives us more confidence that we can find when users
last logged in, especially if mailbox auditing captures mailbox login events. However, even if we restrict the search to
Azure Active Directory events, because user mailboxes form the basis of the objects processed, we can assume that
most logins involve mailbox access. This approach might not always deliver 100% accurate results, but it does give us
more reliable data than Get-MailboxStatistics does.

[PS] C:\> $Mbx = (Get-Mailbox -RecipientTypeDetails UserMailbox -ResultSize Unlimited | Select PrimarySmtpAddress, DisplayName,
UserPrincipalName)

$StartCheckDate = (Get-Date).AddDays(-90)

ForEach ($M in $Mbx) {

$AuditRecs = (Search-UnifiedAuditLog -StartDate $StartCheckDate -EndDate (Get-Date) -UserIds $M.UserPrincipalName -Operations

UserLoggedIn, MailboxLogin -SessionCommand ReturnNextPreviewPage )

If ($AuditRecs.Count -gt 0) {

Write-Host "Last Login date for" $M.DisplayName "is" $AuditRecs[0].CreationDate }

Else {
 Write-Host "No logins found for" $M.DisplayName "since" $StartCheckDate }

Audit log records are generated by user activity. They do not record access to non-user mailboxes, such as shared
mailboxes or group mailboxes. To figure out the last activity in a shared mailbox, you could use the Get-
MailboxFolderStatistics cmdlet to return the last modified date for the Inbox, like this:

[PS] C:\> Get-Mailbox -RecipientTypeDetails SharedMailbox | Get-MailboxFolderStatistics

-IncludeOldestAndNewestItems -FolderScope Inbox | Format-Table Identity, LastModifiedTime -autoSize

Identity LastModifiedTime

-------- ----------------

Customer Services\Inbox 25-Jan-18 3:18:04 PM

Office 365 Book Feedback\Inbox 23-Feb-18 4:18:24 AM

Shared Mailbox\Inbox 10-Jan-18 6:27:18 PM

Redmond Shared Events\Inbox 26-Jan-18 3:15:17 AM

Company Information\Inbox 26-Jan-18 10:09:30 PM

The same technique works for group mailboxes if you pipe the output from the Get-UnifiedGroup cmdlet to Get-
MailboxFolderStatistics .

Mailbox Backups
Exchange Online does not use traditional backups to protect mailboxes and databases against data loss. Instead,
Exchange uses the full range of its “Native Data Protection” features to protect data.

Each database has at least four copies spread across two datacenters. One of these is a lagged copy.
As explained in Chapter 19, the default retention policy applied to Exchange Online mailboxes does not force the
removal of items from the Deleted Items folder. You can change this behavior by changing the retention policy
assigned to user mailboxes. If not, items stay in the Deleted Items folder until the user empties the folder or the
items move to an archive mailbox.
Exchange enables Single Item Retention (SIR) for every mailbox so that items moved into Recoverable Items stay
in the database for the full retention period set on the mailbox. The default for the deleted items retention period
used to be 14 days but is now 30 days for newer mailboxes (anything created since 2017), which is also the
longest period you can set for this property. If you want to update older mailboxes, you can do so with this
command:

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox | ? {$_.RetainDeletedItemsFor -Lt 30}| Set-Mailbox –RetainDeletedItemsFor
30.00:00:00

If the deleted items retention period for a mailbox does not update when you run the
command, check its UseDatabaseRetentionDefaults property. This must be $False before
you can update RetainDeletedItemsFor . Some older mailboxes have this property set to
$True (it’s a legacy of the on-premises heritage of Exchange Online).

Unless you place mailboxes on hold before removing their accounts from Office 365 (in which case they become
inactive mailboxes), Exchange keeps deleted mailboxes for 30 days, after which the Managed Folder Assistant
removes the mailboxes from the database and they become irrecoverable. You cannot change the retention
period, but you can force the immediate removal of a deleted Office 365 account (and its mailbox) by
permanently removing the account from the Azure Active Directory portal or by running the Remove-MsolUser
cmdlet twice. The first operation moves the account into the Azure Active Directory recycle bin (from where you
can recover the account). The second expunges it from the recycle bin because we specify the
RemoveFromRecycleBin parameter. For example:

[PS] C:\> Remove-MsolUser –UserPrincipalName Ed.Banti@Office365ITPros.com –Force

[PS] C:\> Remove-MsolUser –UserPrincipalName Ed.Banti@Office365ITPros.com

–RemoveFromRecycleBin -Force

Archive mailboxes are available to keep information over an extended period. The default retention period moves
items from the primary mailbox to the archive after they are two years old.
You can place mailboxes on-hold to ensure that Exchange keeps all items in the mailbox until the hold expires,
including items that the Managed Folder Assistant would normally remove when their retention period lapses.
See Chapter 20 for information about how to apply in-place holds to mailboxes through eDiscovery cases.
No going back : If you permanently delete an Azure Active Directory account from the portal or by running the
Remove-MsolUser cmdlet with the RemoveFromRecycleBin parameter, you tell Azure Active Directory that you want
a user object to be permanently removed in such a way that no recovery is possible. Azure Active Directory will
synchronize the removal to workload-specific directories, such as EXODS, and any workload-specific objects will also
be hard-deleted. The user’s mailbox, for instance, will be removed from the database where it is stored. Microsoft
cannot recover user objects after they have been hard-deleted and they cannot recover any workload-related data
either. Remember, Office 365 does not take backups of its directory or mailboxes. It is therefore clear that specifying
the RemoveFromRecycleBin parameter should only happen when you are certain that you want to remove a user
object and are quite happy that all the data associated with that object is also removed.

The lack of mailbox backups often prompts a discussion about third-party backup products that copy data from
Exchange Online and other Office 365 applications. This topic is discussed in Chapter 4.

Automatic Mailbox Maintenance

Like on-premises Exchange, the Managed Folder Assistant (MFA) runs on a workcycle basis to apply retention policies
to mailboxes and clean out deleted items that have exceeded their retention period. The SLA for the MFA calls for it to
process mailboxes is at least once weekly, but it is often the case that MFA processes mailboxes more often. You can’t
affect when mailbox management happens because it is intended that this should happen automatically, but you can
affect how MFA processes mailboxes by changing the mailbox properties that govern retention policy. To see what
retention policy is assigned to a mailbox, edit its properties with EAC and go to the Mailbox features section. Scroll
down in the same settings screen to see whether an archive is enabled for the mailbox and how much quota has been
used in the archive. Alternatively, you can run the Get-Mailbox cmdlet to retrieve the same information:

[PS] C:\> Get-Mailbox –Identity TRedmond | Format-Table DisplayName, RetentionPolicy, ArchiveName

Display NameRetentionPolicy ArchiveName

----------- ------------------- -----------

Tony Redmond Management retention policy {Grubby old stuff}

Exchange Online assigns the default Messaging Retention Management (MRM) retention policy to all mailboxes when
they are created, including mailboxes that are migrated from an on-premises server. In contrast, an on-premises
administrator must make an explicit choice to assign a retention policy to a mailbox. The rationale in having a default
retention policy in place for all mailboxes is that it allows MFA to exert some level of control over mailbox contents.

Figure 6-9 shows some of the retention tags that are included in the Default MRM Policy. We can see that the “Default
2 year move to archive” tag is selected. This is a default tag, meaning that the action it dictates will be taken for all
items not governed by a more explicit retention tag. The action is to move items to the archive once they are 730 days
(2 years old). In effect, MFA checks the retention period on items each time it processes a mailbox and will move any
older than 2 years to the archive mailbox. Logically, this action can only happen if a mailbox has been archive-enabled.
If not, MFA ignores the directive contained in the default archive tag. Unlike other retention policies that you might
be aware of, the default MRM policy used by Exchange Online does not include a default delete tag, so if items are not
archived, they will continue to accumulate in the primary mailbox unless the user deletes them.
Figure 6-9: Retention tags in the Default MRM Policy for Exchange Online

Retaining Mailbox Items

Although Single Item Recovery (SIR) ensures that Exchange keeps mailbox items for the full retention period, it is
possible that users will discover that they need some of the items in the deleted items folder after they empty the
folder. If this happens during the deleted items retention policy (between 14 and 30 days for regular items and 120
days for calendar items), the user can get an item back by using the "Recover Deleted Items" feature in either Outlook
or OWA. However, once the retention period expires, the Managed Folder Assistant removes the items from the
database the next time that it processes the mailbox. At this point, the items become irrecoverable.

Exchange Online relies on its native data protection functionality to ensure that a good copy of a mailbox is always
available. Office 365 does not use the traditional approach of taking backups, so backup media is unavailable for data
recovery. Once Exchange removes an item from a database, it is permanently gone and no amount of appeals to
Microsoft Support will result in its recovery. For this reason, some administrators place some mailboxes on a
permanent in-place hold or a litigation hold. The logic is as follows:

A permanent in-place hold keeps all deleted items in a hidden sub-folder of the Recoverable Items folder until the
hold lapses. The in-place hold feature is not part of every Office 365 plan and a mailbox needs to have at least an
E3 or Exchange Online Plan 2 license before an administrator can put the mailbox on hold.
Problems are more severe if information is irrecoverable for mailboxes belonging to executives or other critical
personnel.
Users will not remember to protect every piece of sensitive information – the system must do this for them.
The Recoverable Item folder (and its sub-folders) has a standard quota of 100 GB, which does not count against
the normal mailbox quota.
Deleted items under hold are invisible to the user and clients. An administrator can retrieve these items by
executing an eDiscovery search. Once found, an export to PST makes the items available for return to the user
(or, in the case of an investigation, to investigators).

It is hard to estimate how much a hold will consume of the Recoverable Items quota. However, even a busy mailbox
will remove less than 10 GB of unwanted messages annually and a hyperactive mailbox will consume perhaps 20 GB.
The default quota is therefore capable of holding five years of deleted items – and probably more for less active
mailboxes.

Archive Mailboxes
The motivation to introduce archive mailboxes (also known as “in-place archive” or “online archive”) for Exchange
arose for many reasons. Over the years, Microsoft obviously did not like the fact that customers deployed third-party
products like Symantec Enterprise Vault to offload information from user mailboxes to a separate repository. Often
there was good reason to move data, especially when Exchange mailbox quotas were small, and databases ran on
expensive SAN storage. Moving data to secondary storage managed by other products allowed Exchange mailboxes to
continue without the need for increased quotas, albeit with “stubs" left behind in mailboxes as pointers to the actual
items. If you plan to move previously archived items from external repositories back into Office 365, it is important to
ensure that the stubs are “rehydrated” (become fully complete Exchange items) as part of the process.

From a Microsoft perspective, moving information out of Exchange databases implied a certain loss of account control
as the data could be as easily migrated to a different server as moved back to Exchange. Another issue that has
become increasingly important is that PST data are invisible for the purposes of compliance and eDiscovery. An
example of how important it is for companies to protect their commercial interests and business reputation is seen in
the Sony hack of December 2014 when hackers stole and shared valuable contractual information held in PSTs. The
messages revealed to the public had details of negotiations, strategies, opinions about business partners, and so on. It
would have been so much better had this data been securely kept in online databases rather than being vulnerable to
hackers.

Small mailbox quotas encouraged the proliferation of PSTs and created a plague of insecure and potentially
corruptible files holding valuable user information. Because email messages are commonly shared between multiple
users, PST data usually includes a high percentage of duplicate items. The existence of so much duplicated
information slows down the transfer of data from PSTs to online mailboxes. In addition, users can attempt to protect
PSTs with passwords that range from very easy to very difficult to crack. If you gather PSTs from users for migration,
you must remove passwords from the PSTs to make their data accessible to migration processing. Some third-party
PST migration engines can deduplicate content extracted from PST prior to ingestion into Office 365 and crack open
any password-protected files to extract the information contained within. These are valuable features that you should
consider and evaluate during any PST migration (eradication) project.

Because PST migration is often tiresome and expensive, a better solution is to encourage users to keep their data
online. The solution to allow online storage to replace PSTs came about in two parts. To make it feasible to provide
very large mailbox quotas to users, Microsoft engineered the mailbox database engine to support JBOD. This effort
included the introduction of the Native Data Protection features used within Exchange Online today. Archive
mailboxes, first introduced in Exchange 2010, are the second part. At that time, the plan envisaged that larger
mailboxes meant that no one would ever want to use a PST because their mailbox had so much available space. The
accompanying archive mailboxes are the repository for long-term information and avoided the need for Exchange
customers to buy a third-party archiving product. Mailboxes have become larger and archive mailboxes are in
common use, but users have not yet discarded PSTs. It takes a long time and much effort for Outlook users to break
working habits based on PSTs, including using these files as shared repositories when much better options exist like
Office 365 Groups or SharePoint team sites.

Keeping everything in the primary mailbox : Office 365 enterprise plans (E3 -E5) grant 100 GB primary mailbox
quotas to users, which then creates a question whether archive mailboxes are necessary. After all, 100 GB should be
enough to hold as much data as anyone would want to keep. Although it is true that large mailboxes enable people to
avoid using archive mailboxes, a workable case exists to separate information into that needed to be on-hand and
that the data that users must keep for reference purposes. In this scenario, the items you need on-hand remain in the
primary mailbox and those needed for reference (or compliance) go into the archive. Good retention policies help
users achieve the split by automatically moving items into the archive after a set period (two years is the default
within Office 365).

Archive mailboxes are now available to any Exchange on-premises or cloud mailbox except those using online kiosk
plans. Some Office 365 plans limit archive mailboxes to 100 GB but plans E3 and above allow “unlimited” storage.
What "unlimited" means in practical terms is undefined because data needs to be populated into the archive over
time, but the term can be taken to mean that Microsoft has enough storage to handle anything you care to put in an
archive mailbox. In technology terms, unlimited storage took another step forward with the introduction of “auto-
expanding archives” to Exchange Online (and Exchange 2016). This feature is explained in detail later in this section.
The gradually-expanding scope of the Office 365 Import Service also makes it easier to bring information into
archives, including from third-party data sources.

Microsoft also offers a separate Office 365 service called Exchange Online Archiving to give the ability for on-
premises customers to use a cloud-based archiving service. In this scenario, mailboxes on Exchange 2010 SP3 (and
later versions) servers can connect to unlimited Exchange Online archive mailboxes without having to perform a full
hybrid deployment. Exchange Online Archiving is also available as an add-on plan for Exchange Online kiosk
mailboxes.

An archive mailbox can be defined as an online-only extension of the primary mailbox. In an on-premises deployment
an administrator controls the placement of the archive mailbox and can decide if it is in the same database as the
primary or in a different database (most often the case). Exchange Online almost always places archive mailboxes in a
different database to the primary mailbox and administrators have no control over where mailboxes are positioned.
The link between the two is through the ArchiveGuid , a property of the user mailbox that points to the location of the
archive. Other archive-related properties that are set when a mailbox is enabled include:

ArchiveDatabase : The Exchange Online database holding the archive mailbox.

ArchiveName : A user-friendly name for the archive mailbox that shows up in a client’s resource list. You can
change the name to whatever value you like. For example, “Joe’s Online Archive.”
ArchiveQuota : The current storage quota assigned to the archive. The current default for Exchange Online is
100 GB. The quota can be increased by filing a support ticket with Microsoft support.
ArchiveWarningQuota : The threshold for warning messages to be sent to the user to tell them that space is
running out in the archive. The current default is 90 GB.
ArchiveDomain : This value is null when an archive mailbox belongs to an Office 365 tenant. In a hybrid
deployment, it is set to the name of the on-premises domain.
ArchiveStatus : Set to “Active” when the archive mailbox is available to a user. It is set to “None” when an
archive mailbox is not present.
ArchiveState : Set to “Local” when the mailbox and archive are on the same platform as is the case when both
are hosted by Exchange Online.
 ArchiveRelease : The Exchange version number. E15 means the Exchange Online equivalent of the on-premises
Exchange 2013 or Exchange 2016 (both these versions share the same major build number).

You enable a mailbox to have an archive by selecting it in the EAC and then click Enable in the “In-place archive”
section of the details pane (Figure 6-10 ). Alternatively, edit the mailbox properties, go to the mailbox features section,
scroll down to archiving, and click Enable . The equivalent PowerShell command is:

[PS] C:\> Enable-Mailbox –Identity 'Joe Smith' -Archive

Rather bizarrely (but only because I haven’t yet come up with a good reason for doing this), you can enable an archive
for a shared or room mailbox (if you do this, remember you need to assign at least an Exchange Online Plan 1license
to the mailbox). However, you can’t enable an archive for a group or site mailbox.

You can check for mailboxes that have archives by running the Get-Mailbox cmdlet with the Archive switch. The first
command below returns a list of archive-enabled mailboxes. The second returns a simple count of archive-enabled
mailboxes.

[PS] C:\> Get-Mailbox –ResultSize Unlimited –Archive –Filter {RecipientTypeDetails –eq "UserMailbox"} | Format-Table DisplayName,
ArchiveName

DisplayName ArchiveName

----------- -----------

Deirdre Redmond {Deirdre's Online Archive}

Tony Redmond {Grubby old staff}

Jeff Guillet {Jeff's Archive}

Kim Akers {In-Place Archive - Kim Akers}

[PS] C:\> (Get-Mailbox -Archive -ResultSize Unlimited -Filter {RecipientTypeDetails -eq "UserMailbox"} | ? {$_.ArchiveName -ne
$null}).Count

15

EAC gives a visual clue for administrators by highlighting the mailboxes that have archives by displaying “User
(Archive)” in the mailbox type field (Figure 6-10 ). The easiest way to see archive-enabled mailboxes together is to
click the Mailbox Type header to force the EAC to sort mailboxes by this field.

Disabling an archive removes the ability of the mailbox owner to access the content held in the archive. In effect, this
action means that you break the connection between the primary and archive mailboxes. To disable an archive with
EAC, select the mailbox and click Disable under In-Place Archive in the action pane. It is important to realize that
Exchange Online will only allow an archive to be disabled if no in-place or litigation holds exist for a mailbox. This
restriction exists to ensure that no-one can remove data from Exchange Online that might be required for legal
discovery. The PowerShell command to disable an archive is:

[PS] C:\> Disable-Mailbox –Identity 'Joe Smith' -Archive

Although disabling an archive prevents access to the archive, it does not remove the content from the database that
holds the archive. Instead, a 30-day retention period starts. During this time, you can recover the archive and
reconnect it to the primary mailbox by re-enabling the archive. Any content in the archive mailbox will be removed
from the database once the 30-day deleted mailbox retention period expires. If you look at mailbox properties after an
archive is disabled, you'll see that the ArchiveGuid , a unique value used by Exchange to find the archive mailbox
within a database, is a set of zeros. This is how clients know that they shouldn't try to open an archive for this
mailbox. However, the original value of the ArchiveGuid is preserved in the DisabledArchiveGuid property, which
means that it can be used to re-establish the link to the archive when the Enable-Mailbox cmdlet is run. After the
archive is re-enabled, the two GUID values should be identical.

[PS] C:\> Get-Mailbox –Identity 'Joe Smith' | Format-List *ArchiveGuid

ArchiveGuid : 00000000-0000-0000-0000-000000000000

DisabledArchiveGuid: d7c65fee-c983-4eac-8fa3-6381a8673212

[PS] C:\> Enable-Mailbox –Identity 'Joe Smith' -Archive | Get-Mailbox –Identity 'Joe Smith' | Format-List *ArchiveGuid

ArchiveGuid : d7c65fee-c983-4eac-8fa3-6381a8673212

DisabledArchiveGuid: d7c65fee-c983-4eac-8fa3-6381a8673212

Figure 6-10: Finding archive-enabled mailboxes in EAC

Several of the PowerShell cmdlets commonly used to work with mailboxes include an Archive switch to point them to
the archive rather than the primary mailbox. The Get-MailboxStatistics and Get-MailboxFolderStatistics cmdlets are
good examples:

[PS] C:\> Get-MailboxStatistics –Identity 'Kim Akers' –Archive

[PS] C:\> Get-MailboxFolderStatistics –Identity 'Kim Akers' –Archive | Format-Table Name, ItemsInFolder

An archive section is also included in the Security and Compliance Center. Much the same functionality to setup and
manage archive mailboxes is available to administrators as through the EAC, with the difference being that the focus
is exclusively on enabling or disabling archives for user mailboxes. In a hybrid environment, some important points
exist about how archives are enabled for on-premises mailbox in the Security and Compliance Center.

Naming archives : The default name for an archive mailbox is “In-Place Archive – User ” (for example, “In-Place
Archive – Tony Redmond”). However, the Set-Mailbox cmdlet can be used to assign whatever name you like to an
archive. The name can be at least 80 characters (the largest value I have tested). However, OWA is the only archive-
capable client that will display the name that you assign as Outlook ignores the value and uses a normalized name of
“Online archive – username@domain. ” There is no reason why the two client development teams decided to display
different default names for archive mailboxes and no reason why Outlook did not follow OWA in allowing user-
specific names to be used. But they do not. It is just another Exchange quirk.

How Information Moves into an Archive Mailbox

Before anyone can move content into an archive, their mailbox must be archive-enabled. After the archive is created it
becomes visible to clients. OWA will recognize the archive the next time the user connects to Exchange Online and
Outlook will learn that the archive exists the next time it refreshes the list of available resources through the
Autodiscover process. When the archive appears, the user can move information to it by creating folders and dragging
and dropping items from their primary mailbox.

Behind the scenes, every Exchange Online mailbox is assigned a retention policy unless the administrator removes it.
The default retention policy includes a set of retention tags to control movement of items into the archive, the most
powerful of which is a default archive tag to specify that all items should be moved into the archive after they are two
years old unless an item is stamped with a tag that overrides the default. The net effect is that the archive is
populated by the actions of the Managed Folder Assistant (MFA) on an ongoing basis as items will move from the
primary mailbox into the archive as soon as they reach the two-year limit. Items can be automatically removed from
the archive by including a default delete tag in the policy to age items out after a further period. The MFA only moves
items into the archive if the user’s primary mailbox holds more than 10 MB of data.

Of course, most users are blissfully unaware of boring details such retention policies. They might not even notice that
items are being moved on an ongoing basis. That is, until they look for something and can’t find it because the item is
not where they thought it should be – in their mailbox. Of course, the item is still available, and it is, in a roundabout
way, still in their mailbox. It is just invisible in some respects because the user doesn’t know where it is or understand
how it got there.

Archive mailboxes are intended to be personal and cannot be used as the target for archiving information. Microsoft
explicitly prohibits the use of transport rules, journal rules, auto-forwarding, or other methods to move information
into a mailbox from multiple sources for archiving purposes. However, it is permissible to import data from multiple
PSTs that might have originated from multiple users into an archive mailbox. The difference between the two is that
using an archive mailbox as an archive destination creates the potential that the archive mailbox will expand on an
ongoing basis to cope with the archive traffic. On the other hand, if you perform a once-off import to an archive
mailbox, its content will probably not change all that much in the future as the archive is essentially historical rather
than live. In other words, it’s a question of dealing with “hot” data that is constantly growing or “cold” data that is
imported once and hardly ever accessed thereafter.

Effective Use of Archive Mailboxes

Because primary user mailboxes can hold up to 100 GB, many users can quite happily get along by just using their
primary mailbox and never need to go need an archive. Consider the following:

It will take most users several years to accumulate 10 GB of mailbox data. It is true that some users accumulate
20 GB+ of new mailbox content annually, but they are not the rule. On the other hand, people have had quite a
while to accumulate information in their mailboxes at this point and many of the mailboxes that move to
Exchange Online are already up at the 20 GB mark (or higher).
In the past many caveats were expressed about the desirability of holding more than 2 GB in the primary
mailbox. Apart from the cost of storage, most problems related to offline access and the need to synchronize such
a large amount of data to an OST file whose internal structure was never designed to cope with large volumes.
These issues have largely been dealt with today due to faster networks and smarter Outlook synchronization (like
the Outlook 2013 slider shown in Figure 6-7 that controls how much data is synchronized to the OST). The larger
amounts of data synchronized to clients do slow down OST access speeds, a factor that can be handled by
equipping laptops with SSDs that effectively disguise the inefficiency of the OST.
Archive mailboxes can only be accessed online. Outlook does not synchronize any archive folder into the OST.
While the bulk of data moved into the archive might never be accessed again, the potential that someone might
need an archived item when a network connection is not available does exist.
Searches can find items stored in archives but only if the user specifies that Outlook should search “All
Mailboxes.” Searching the archive in OWA needs the user to be positioned within an archive folder. Neither of
these operations are especially intuitive to someone who might be unaware that they have an archive.
ActiveSync clients cannot access an archive mailbox because the protocol does not support this type of resource.
The same is true for the Outlook for iOS and Android clients. The defunct OWA for Devices clients can access
archives, but these are now a minute percentage of the overall base of mobile clients.
Users seldom want to decide what items should be always available (and in their primary mailbox) and those that
can be safely moved into the archive. Expecting users to decide on a data storage strategy is an act of folly.

With these points in mind, it should come as no surprise to find that some tenants avoid the use of archive mailboxes
and tell users to exploit the storage now available in primary mailboxes. Indeed, some companies have reversed
course and decided to only use primary mailboxes. In some cases, they have had to move information back from
archives to primary mailboxes. Apart from dragging and dropping items from folder to folder using Outlook or OWA or
exporting archive items to a PST and importing them back into the primary mailbox, there is no in-built method to do
this. In most cases, the solution is to create a script to do the work through a combination of PowerShell and
Exchange Web Services (a good example is available here ).

On the other hand, some tenants consider archive mailboxes to be the perfect answer to the problem of unmanaged
and proliferating PSTs and make strenuous efforts to import data from user PSTs into archive mailboxes so that
everything is online and available for compliance. The availability of Microsoft’s Import Service for Office 365, which
allows tenants to load PSTs into Azure for later import into user mailboxes (primary or archive) has spurred many
companies to consider how they should deal with PSTs in the future. Moving information from PSTs into archive
mailboxes exposes the data for compliance purposes, but this is an exercise that involves much more than the
ingestion. Before you import anything, you need to decide how to collect the PSTs, clean them up (remove
corruptions), crack passwords set on the PSTs, and possibly deduplicate the content so that clean information is
imported. Remember the adage that rubbish in equals rubbish out.

Another effective use of archive mailboxes is to hold data migrated from older POP3 or IMAP4 systems. It is also fair
to say that companies who have moved data back from third-party archiving solutions to Exchange Online find archive
mailboxes to be a natural evolution. In summary, the decision to use or ignore archive mailboxes comes down to the
circumstances and business conditions that exist within a tenant.

PST Import Tools : Many tools are available to help you move data from user PSTs into primary or archive
mailboxes. Like any software utility, you should carefully test the software in your environment to discover which
product best meets your needs. Once you have collected and prepared (made sure no corruption exists) the PSTs, you
can use the Office 365 Import service to either upload the PSTs over the network or send them on hard drives to a
Microsoft datacenter for processing. In either case, the PST data will be ingested into Azure and can be imported
from there into user mailboxes.

Auto-Expanding Archives
The mailbox quotas assigned in enterprise Office 365 plans is more than enough to handle the storage requirements
for most users, which then raises the question of why users also need archives. An archive mailbox has a distinctly
different usage profile to its primary mailbox. Where a primary mailbox is usually very busy due to the processing of
incoming and outgoing messages, most archive mailboxes stay undisturbed for extended periods. In addition, Outlook
does not synchronize archive mailboxes to the OST. This is by design because archived material is “just in case”
information (often kept for compliance purposes), so it does not make sense to incur the overhead of network and
processing needed by clients to synchronize archive mailboxes.

The low demand for user interaction makes it practical to allow archive mailboxes to have much higher storage limits
than primary mailboxes. In turn, a greater storage capacity allows archive mailboxes to act as long-term repositories
to replace PSTs ingested through the Office 365 Import service or legacy email archives migrated from legacy
archiving systems such as Enterprise Vault. Upping the storage quota for an archive mailbox to 500 GB (or larger) is
certainly one way to create a high-capacity archive. However, the question then becomes as to what is the right
quota? Is 500 GB enough or should the limit be 1 TB? What happens when an archive needs to store more than this
amount? From a technical perspective, you need to keep an archive mailbox to a reasonable size so that a mailbox
database can store it and that database fits on a disk. Microsoft’s solution to these and other technical and
management issues is the “auto-expanding archive,” where an archive mailbox dynamically expands to accommodate
the demand to store more data as needed.

User or shared mailboxes with Office 365 E3 and E5 licenses or with the Exchange Online Plan 2 license or the
Exchange Online Archiving add-on can use auto-expanding archives. The largest auto-expanding archive mailbox in
production comfortably surpasses the terabyte mark and hundreds of thousands of auto-expanding archives are in
use. To figure out whether a mailbox is eligible, you can check its persisted capabilities, which expose the licenses
assigned to a mailbox. In this case, the presence of BPOS_S_Enterprise means that the mailbox can used auto-
expanding archives. The other values are BPOS_S_ArchiveAddOn (Exchange Online Archiving) and BPOS_S_Archive
(Exchange Online Plan 2).

[PS] C:\> Get-Mailbox -Identity TRedmond | Select -ExpandProperty PersistedCapabilities

BPOS_S_ThreatIntelligenceAddOn

BPOS_S_EquivioAnalytics

BPOS_S_CustomerLockbox

BPOS_S_Analytics

BPOS_S_Enterprise

In some respects, auto-expanding archives borrow an idea from the way that public folders use mailboxes to store and
manage data. Public folders store both their hierarchy and content in public folder mailboxes. The first public folder
mailbox holds the only writable copy of the hierarchy and all updates for folders go to that copy. Office 365 includes
an auto-split feature that automatically creates new public folder mailboxes when a folder approaches 50 GB. The
Mailbox Replication Service (MRS) transfers folders to the new mailbox to balance the load.

Enabling Auto-Expanding Archives

To enable auto-expanding archives for a tenant, run the Set-OrganizationConfig cmdlet to update the tenant
configuration:

[PS] C:\> Set-OrganizationConfig -AutoExpandingArchive

Once set, the new tenant configuration ensures that all existing archives become auto-expanding and any new
archives are auto-expanding. The process of enabling existing archives can take some time to complete due to the
need for the Mailbox Folder Assistant to process each mailbox.

If you do not want to enable auto-expanding archives for all mailboxes, you can control this on an individual basis by
running the Enable-Mailbox cmdlet. Before you can make the switch, an archive must already exist for the mailbox.
For instance, to enable an auto-expanding archive for Kim Akers:

[PS] C:\> Enable-Mailbox -Identity "Kim Akers" –AutoExpandingArchive

Some thought and planning are necessary in the deployment of auto-expanding archives. Enablement is a one-way
process. No method exists to collect the pieces of an auto-expanding archive together to make them into a single
“standard” archive mailbox. In addition, you cannot move a mailbox with an auto-expanding archive to an on-premises
server until Microsoft releases a version of Exchange that supports the transfer of these mailboxes. In the interim, if
you want to move items out of an auto-expanding archive to transfer data to another mail system or an on-premises
server, you will have to export the needed data to PSTs. Running an eDiscovery content search to find and export the
data is an effective way to achieve this goal. You can make a mailbox with an auto-expanding archive inactive by
putting it on-hold and then removing the user account. Although this allows for the long-term preservation of
mailboxes for compliance purposes, you should still review your procedures covering how to remove user accounts
from the tenant to ensure that everything works as you expect.

Quotas have different meanings when you use auto-expanding archives. Exchange still imposes the normal 100 GB
quota on the primary archive to manage the overall size of that mailbox, but the combination of primary and auxiliary
archives obviously goes well past the normal quota. In addition, the recoverable items quota (usually 100 GB) is set to
110 GB for mailboxes that are subject to a legal or in-place hold.

One limitation is that if you need to search auto-expanding archives with OWA or Outlook, you can only search within
a specific folder. eDiscovery content searches can find information stored in any part of an auto-expanding archive,
which also support legal and in-place holds as normal. You also need to run a supported client (Outlook 2016 for
Windows or Mac, or OWA) to access an auto-expanding archive.

Because off-boarding of mailboxes with auto-expanding archives needs manual intervention, you should not enable
these archives until you are happy that the need for the feature outweighs the possible downside. In many cases, it is
preferable to enable auto-expanding archives for selected accounts instead of a complete tenant. Those accounts are
usually those that have a genuine business need to keep massive quantities of information for certain periods.

How an Archive Mailbox Expands

When you enable a mailbox for archiving, it starts with a single 100 GB archive mailbox. After the occupied space
within the archive mailbox approaches the transition threshold (90 GB), a mailbox assistant automatically provisions a
new archive mailbox. Exchange calculates the occupied size from the total size of folders in the archive mailbox that
have their Movable flag set to $True or the FolderType set to DeletedItems or RecoverableItems .

To get some insight into the current state of the primary archive, we can write some PowerShell to scan the folders in
the primary archive for a user and reports the current occupied space. Later in this section we discuss how to retrieve
the GUID for the primary archive used here as the input to the Get-MailboxFolderStatistics cmdlet.

[PS] C:\> $CheckUser = Read-Host "Enter User to check"

$MLO = (Get-MailboxLocation -User $CheckUser | Select MailboxLocationType, MailboxGuid | ? {$_.MailboxLocationType -eq

"MainArchive"})

$Folders = (Get-MailboxFolderStatistics -Identity $MLO.MailboxGuid.Guid | Select FolderPath, Movable, FolderType, Name,

ItemsinFolder, FolderSize)

$NumFolders = 0

$TotalSize = 0

$Folders | Add-Member -MemberType ScriptProperty -Name FolderSizeInBytes -Value {$this.FolderSize -replace "(.*\()|,| [a-z]*\)", ""}

ForEach ($F in $Folders) {

If ($F.FolderType -eq "DeletedItems" -or $F.FolderType -eq "RecoverableItems" -or $F.Movable -eq $True) {

$TotalSize = $TotalSize + $F.FolderSizeInBytes

$NumFolders++ }

$TotalSizeGB = [math]::round($TotalSize/1GB, 3)

$ThresholdPercent = ($TotalSizeGB/90).ToString("P")

Write-Host $NumFolders "movable folders found. Occupied space" $TotalSize "bytes or" $TotalSizeGB "GB" " At" $ThresholdPercent "of 90
GB transition threshold"

Enter User to check: TRedmond

92 movable folders found. Occupied space 5834011419 bytes or 5.433 GB At 6.04% of 90 GB transition threshold

After provisioning, the new archive mailbox is a “chunk” or “shard,” or more correctly “auxiliary archive,” and joins
the overall archive. Exchange links the GUID for the new auxiliary archive with the GUIDs of the existing auxiliaries
and primary archive to form a chain or set of mailboxes that the Information Store considers a single logical entity.
The expansion of an archive to form an archive chain occurs without user intervention and without affecting
supported clients, which continue to query the Information Store for data and receive data back without knowing
which part of the archive holds the data.

The Mailbox Folder Assistant coordinates the movement of information out of the primary archive to an auxiliary
archive to reduce the size of the primary archive under the transition threshold. The aim of this exercise is to move
enough data to the auxiliary archive to get the primary archive under 50% of its current size. So, if the primary
archive grows to 95 GB, the Managed Folder Assistant examines the folders in the primary archive and selects enough
to move approximately 47.5 GB to the auxiliary archive. The Deleted Items and Recoverable Items folders are not
movable, but the Managed Folder Assistant can create sub-folders under these folders in the target archive and move
content there.

The Mailbox Replication Service moves the data from the primary to auxiliary archive and takes care of ongoing
synchronization to ensure that any changes made to data while the move progresses are in the moved data. The
copying of content occurs in the background. To ensure that no data loss occurs during the rebalancing of the archive,
the primary archive keeps the copied data for 30 days. When this period elapses, the Managed Folder Assistant
flushes the copied data from the primary archive to release the space.

Archive Links
When Exchange 2010 originally introduced archive mailboxes, the archives linked to their primary mailboxes by
storing the GUID pointing to the archive as a mailbox property. The GUID is enough for Exchange to find the archive
in a database. The auto-expanding archive replaces the single GUID that connects the mailbox to the archive with a
linked list of GUIDs. Each of the GUIDs points to a separate auxiliary archive of up to 50 GB, which Exchange Online
combines with the other auxiliary archives and the primary archive to form a logical archive mailbox. Only Outlook
2016 (click to run version), Outlook 2016 for Mac, and OWA support auto-expanding archives. Older clients are only
able to access information in the primary archive and cannot access any data moved into an auxiliary archive.

You can see the details of the GUIDs by running the Get-Mailbox cmdlet to examine a mailbox’s properties. If you look
at the MailboxLocations property, you will see something like this:

[PS] C:\> Get-Mailbox –Identity TRedmond | Format-List MailboxLocations

MailboxLocations: {1;0370f354-2752-4437-878d-cf0e5310a8d4;Primary;eurprd04.prod.outlook.com; 353ce1b5-5044-4974-93f0-7b6f4a54edf8,

1;afc1e472-0826-498e-b990-85de223e809d;MainArchive;eurprd04.prod.outlook.com;353ce1b5-5044-4974-93f0-7b6f4a54edf8}

The information about mailbox locations reported by Get-Mailbox divides into two sections; one for the primary
mailbox and the second for the archive. Only the primary mailbox and the primary archive are shown here. If other
auxiliary archives are present, the archive section lists them as mailbox 2, 3, 4, and so on. The information for the two
mailboxes is:

Primary mailbox:

The ExchangeGUID (which ties the mailbox back to a user account).

“Primary” to show that this data refers to the user’s primary mailbox.
If, as in this case, the mailbox database is in Exchange Online, the name of the Exchange Online forest is noted
(eurprd04).
The GUID of the database holding the mailbox.

Archive mailbox:

The ArchiveGUID (which is only present when a mailbox is archive-enabled).

“MainArchive” to show that this data is an archive set. When auxiliary archives are part of the set, they are
tagged with “AuxArchive.”
The name of the Exchange Online forest that hosts the archive mailbox. This value is empty if on-premises.
The GUID of the database holding the archive.

In this case, the mailbox and the archive are in the same database (you can confirm this by using Get-Mailbox to
examine the Database and ArchiveDatabase properties). And, as you would expect, both the primary and archive
mailbox are in the same Exchange Online forest. Another way of accessing information about these archives is with
the Get-MailboxLocation cmdlet. For example:

[PS] C:\> Get-MailboxLocation -User TRedmond | Sort MailboxLocationType -Descending | Format-Table MailboxGUID, MailboxLocationType

MailboxGuid MailboxLocationType

----------- -------------------

0370f354-2752-4437-878d-cf0e5310a8d4 Primary

afc1e472-0826-498e-b990-85de223e809d MainArchive

bb131464-1461-147e-b774-41646ddadd11 AuxArchive

In this case, the MailboxGuid property for the user’s primary mailbox and all the parts of the archive are more
obvious. The MailboxGuid is needed if you want to check how much data Exchange has moved to an auxiliary archive.
For example:
[PS] C:\> Get-MailboxStatistics -Identity bb131464-1461-147e-b774-41646ddadd11 | Format-Table ItemCount, TotalItemSize

ItemCount TotalItemSize

--------- -------------

30225 4.398 GB (4,722,299,639 bytes)

We now know that Exchange has moved a certain amount of data to the auxiliary archive.

The Archive Folder

Exchange Online mailboxes include an Archive folder as one of the default set of folders created inside all mailboxes.
Apart from a user being able to apply a retention tag to the Archive folder to move items in the folder to the archive
mailbox after a period, the Archive folder has no relationship to the “online archive.” Instead, Microsoft envisages the
Archive folder as a convenient place to move items to from the Inbox after a user has finished processing the items
but wishes to keep them for a period.

The advantage of putting items in the Archive folder rather than removing them or having Exchange Online move
them to the archive mailbox is that items in the Archive folder are available to mobile clients. Items in the archive
mailbox are inaccessible to mobile clients because the protocols used by these clients only support primary mailboxes.
Items in the Archive folder are also available offline while items in the archive mailbox are only accessible when a
network connection is available. Sometimes that network connection is not fast enough to allow easy access to
archived items, including searching those items. Because local searches can process cached copies of the Archive
folder, searches complete faster and items are more accessible.

Outlook 2016 desktop, OWA, and the Outlook for iOS and Android apps support one-click (or swipe) options to allow
users to easily move items to the Archive folders. The idea is that a user can quickly move through their Inbox to
triage items by reading, removing, or archiving items as required. Because it is a mailbox folder, you can synchronize
the Archive folder to other mobile clients (like Outlook for Windows 10 Mobile) but one-click access will not be
available unless the developer of the client incorporates the necessary GUI.

Inbox Rules
Exchange Online supports both server-based rules and client-side rules. Server-based rules, or Inbox Rules as they are
referred to in the documentation, are kept in the user mailbox and are accessible to all clients. Inbox rules run
whether a client is connected to the mailbox or not and have priority over client-side rules (which are specific to a
client) because they are processed as part of the delivery pipeline. Inbox rules also have priority over server-based
processing like the Focused Inbox and offer the significant benefit of being able to process inbound messages no
matter what client is used. On the other hand, client-side rule implementations like the one found in Outlook differ
across client types and are highly specific to that client. In other words, you cannot take a rule created on one client
and expect to be able to use it with another.

You can also create Inbox rules with Outlook if the actions performed by the rule can be executed on the server. For
instance, a rule that calls for an item to be moved to a folder in a PST cannot be processed by Exchange Online
because the server does not usually have access to PSTs. In all cases, client-side rules can only process whatever
messages are left in the inbound mail stream after the transport system is finished its processing and Outlook is
running.

The processing steps that can be performed with Inbox rules include:

Delete an inbound message.

Move or copy the message to a folder.
Mark the message with a category, importance level, or as read.
Redirect or forward the message to another email address.

The conditions and exceptions that can be incorporated into an Inbox rule include if the message:

Was sent by a specific person or addressed to a specific user.

Has specific words in the subject, body, sender’s address, recipient’s address, or message header.
Includes the recipient’s name in the To: or CC: box or is the only recipient listed.
Is marked with a specific importance or sensitivity level.
Has an attachment, is of a specific type, is classified, or flagged as something.
Is a specific size.
Is received within a specific date range.

Users create and edit Inbox rules by selecting a message and using the “Create rule” option in OWA. Alternatively,
they can manage rules through the Automatic processing section of OWA options (Figure 6-11 ).

Both approaches end up using the same wizard to build the rule. OWA divides rules into Inbox rules, which check
inbound mail and apply processing based on the characteristics of the messages, and sweep rules, which run in the
background to move messages from a mailbox into the Deleted Items folder. Only the mailbox owner can create sweep
rules. To do this, select a message using OWA and click Sweep in the navigation bar (Figure 6-12 ). When saved, you
see the sweep rule in OWA options under Inbox rules. However, you cannot change sweep rules. To change a sweep
rule, you must remove and recreate the rule. In addition, sweep rules cannot be accessed with Outlook.

Figure 6-11: Creating a server-side rule with OWA

Figure 6-12: Creating a sweep rule

Using PowerShell to Manage Rules

If an administrator needs to set up rules for a user, they can do so logging into the mailbox to manage rules for a user
through OWA options. PowerShell can also be used by administrators to manage rules on behalf of users using the *-
InboxRule cmdlet set, including the ability to create new rules. For example, this command tells us whether any rules
exist for a mailbox and lists the rule name, the priority order, and whether rule processing stops when a rule fires:

[PS] C:\> Get-InboxRule –Mailbox "Kim Akers" | Format-Table Name, Priority, StopProcessingRules

–AutoSize

Given the complexity of the conditions and exceptions that can be defined for a rule, the easiest approach to creating
a rule is to use the OWA interface. For instance, you could create the rule for your own mailbox, make sure that the
rule works as expected, and then replicate it for a user mailbox. When the rule is ready, you can see all its properties
with a command like this:

[PS] C:\> Get-InboxRule –Mailbox "Kim Akers" –Identity "Remove Rubbish" | Format-List
You need to note all the conditions and exceptions for the rule and then use them as the basis for the parameter
values provided to the New-InboxRule cmdlet. For example, this rule looks for inbound messages that have “Hello
World” in their subject and forwards the messages to a different SMTP address, unless the message is marked with
high importance. In this case, we assign a name to the rule to show that this is an administrator-created rule. Using a
naming convention like this is not mandatory, but it is a good practice because, in the case of disputes, it helps to
isolate user-created rules from administrator-created rules. It is always a good idea to secure agreement with a user
before you create a rule on their behalf – or at least to tell them that a rule exists after creation.

[PS] C:\> New-InboxRule –Name "Process Email (Created by Admin)" –Mailbox "Kim Akers"

–StopProcessingRules $True –SubjectContainsWords "Hello World" –ExceptIfWithImportance High

–ForwardTo "Jill.Smth@Office365ITPros.com"

When creating the new rule, Exchange Online places it at the top of the rule priority order by assigning it a priority
value of 1. You might not want this to be the case as other rules should be processed first. If this is the case, you can
change the priority order with the Set-InboxRule cmdlet. Note that you always must tell Exchange Online to which
mailbox the rule belongs.

[PS] C:\> Set-InboxRule –Mailbox "Kim Akers" –Identity "Process Email (Created by Admin)"

–Priority 3

Outlook can also manage user rules. Because it spans actions and conditions that take advantage of the client’s
capabilities, Outlook supports a more comprehensive set of rule conditions and actions than OWA does. For this
reason, Outlook can be a better choice for rule creation. For instance, you can create an Outlook rule to have
Exchange Online send a template message in response to inbound mail (the “have server reply with a specific
message action”). This feature is not supported by the *-InboxRule cmdlet set. Outlook rules that use advanced
options cannot be edited through OWA.

The standard quota for the storage of inbox rules in a mailbox is 64 KB. This is usually enough, especially as anyone
who uses the Focused Inbox can probably remove many rules that filter unwanted messages. Once the quota is
reached, a user won't be able to create new rules. You can increase the rule quota to a maximum of 256 KB by
running the Set-Mailbox cmdlet. For example:

[PS] C:\> Set-Mailbox –Identity 'Kim Akers' –RulesQuota 256KB

PowerShell for Sweep Rules

You can retrieve a list of sweep rules for a mailbox with the Get-SweepRule cmdlet. For example, this command
returns a list of rules and tells you whether each rule is enabled and which sender the rule processes.

[PS] C:\ Get-SweepRule -Mailbox TRedmond | Format-Table Name, Enabled, Sender

You can also create and remove sweep rules. However, an administrator can only create a sweep rule for the mailbox
that they own. Here is an example of using the New-SweepRule cmdlet to create a new rule:

[PS] C:\> New-SweepRule -Name "Messages from Microsoft Tech Community" -Provider Exchange16 -KeepForDays 7 -SourceFolder "Kim
Akers:\Inbox" -Mailbox "Kim Akers" -DestinationFolder "Kim Akers:\Deleted Items" -Sender "notift@mstechcommunity.onmicrosoft.com" -
Enabled $True

On the other hand, an administrator can remove a rule from a mailbox, if they know the identity for the rule, which
can be retrieved with Get-SweepRule . The identity for a rule is composed of the mailbox name and a unique string.
For example:

TRedmond\CzzdSOaFuESmQGBLtw/yZw==

We can remove the rule as follows:

[PS] C:\> Remove-SweepRule -Identity TRedmond\CzzdSOaFuESmQGBLtw/yZw==

Checking for Rules that Forward Email

If a company assigns an Office 365 mailbox to a user, it is probably because the company wants that person to use the
mailbox to send and receive email, and to keep the email in the mailbox for compliance purposes. However, it is
possible that some users prefer to use another email system and will therefore use a rule to forward mail they receive
in Office 365 to that system. This action removes the email out of the control of retention and other data governance
policies, so companies often view allowing users to run rules to forward email outside the organization to be a bad
thing. The Forwarding report available in the Mail Flow dashboard in the Security and Compliance Center gives an
insight into mailboxes with forwards set from the tenant, and it is also possible to check this with PowerShell. The
steps in this example:

Creates a collection of user and shared mailboxes.

Checks if any rules exist in the mailbox.
If rules exist, check if any forward messages (including forward as an attachment).
Check the forwarding addresses to see if the recipient is known to the tenant directory (including guest
 accounts).
Report any forwarding address found.
Optionally, if the forwarding address is unknown (does not exist in the Exchange directory), remove the rule from
the mailbox (these lines are commented out).

[PS] C:\> Write-Host "Finding mailboxes to check..."

$Mbx = (Get-Mailbox -RecipientTypeDetails UserMailbox, SharedMailbox -ResultSize Unlimited)

Write-Host $Mbx.Count "user and shared mailboxes found. Now checking..."

$NumberMbxWithRules = 0

ForEach ($M in $Mbx) {

Write-Host "Processing" $M.DisplayName

$Rule = $Null

$InboxRules = (Get-InboxRule -Mailbox $M.Alias | ? {$_.ForwardTo -or $_.ForwardAsAttachmentTo })

If ($InboxRules -ne $Null) {

ForEach ($Rule in $InboxRules) {

$Ex = $Null

$ForwardTo = @()

$ForwardTo = ($Rule.ForwardTo | ? { ($_ -Match "SMTP") -or ($_ -Match "EX:") } )

$ForwardTo += ($Rule.ForwardAsAttachmentTo | ? {($_ -Match "SMTP") -or ($_ -Match "EX:")}

If ($ForwardTo.Count -gt 0) {

ForEach ($Recipient in $ForwardTo) {

If ($Recipient -Match "EX:") {

# Recipient known in Exchange directory

$Ex = (Get-Recipient -Identity ($Recipient-Split "Ex:")[1].trim("]}"))

$EmailAddress = $Ex.PrimarySmtpAddress }

Else {

# Simple SMTP address

$EmailAddress = ($Recipient -Split "SMTP:")[1].Trim("]") }

Write-Host $M.RecipientTypeDetails $M.DisplayName "is forwarding to" $EmailAddress

-ForegroundColor Red

# Remove the rule if the address is unknown to the directory

# If ($Ex -ne $Null) {

# Remove-InboxRule -Identity $Rule.Identity -Confirm:$False

# Write-Host "Rule" $Rule.Name "removed from mailbox!" }

$NumberMbxWithRules++ }

Write-Host $NumberMbxWithRules "found with forwarding rules"

Even if you did not remove the offending rules, you could run the script periodically to discover whether forwarding
email outside the organization is being done, and if so, the target domains for forwarding.
Calendar Sharing
Asking for the ability to share their calendar with external recipients is a common user request. This is possible with
Office 365, but only after you enable calendar sharing through the Office 365 Admin Center (Figure 6-13 ). To control
calendar sharing, go to Settings , then Services & Add-ins , and then Calendar .

Sharing can take three forms. Users control how much data is revealed when they issue a sharing invitation to
another person. The options are:

View when I am busy: The calendar shows slots to indicate when the person is free, busy, or has a tentative
appointment. This is the same free/busy data used to establish whether a potential attendee is available for a
meeting.
Titles and locations: The calendar shows booked time slots with some information – time, subject, and location.
All details: All the information held in calendar events is available for view.

After a tenant allows calendar sharing, users can click the Share button in the OWA calendar or Share Calendar in
Outlook to generate a sharing invitation to people within the tenant or in external organizations.

Figure 6-13: The choices to control how users share calendars

Within the tenant, individual users can assign delegate permission to other users to allow them to process calendar
events on their behalf. If you assign someone to be a delegate, you can restrict access to calendar events marked as
private.

Sometimes administrators like to know which users are sharing their calendars. This PowerShell command generates
a list of all calendar sharing, internal and external, and is one way to find out what is happening inside the tenant.

[PS] C:\> ((Get-Mailbox –RecipientTypeDetails UserMailbox).Alias) | % { Get-MailboxFolderPermission –Identity ($_+":\Calendar") } |

Where-Object { $_.User –notlike "Anonymous" } | Format-Table Identity, FolderName, User –AutoSize

Identity FolderName User

-------- ---------- ----

Deirdre Redmond:\Calendar Calendar Tony Redmond

TRedmond:\Calendar Calendar Deirdre Redmond

TRedmond:\Calendar Calendar ExchangePublishedUser.Someone@gmail.com

Any entry under “User” that starts with “ExchangePublishedUser ” indicates that a calendar is shared externally with
the email address that follows.

Strictly speaking, shared mailboxes do not support calendar sharing. However, you can set up calendar sharing for a
shared mailbox with these steps:

Grant an account Full Access to the shared mailbox.

Use OWA to log into the shared mailbox with the account.
 Set up sharing using the Calendar as normal.
Check that everything works.

A detailed discussion of how to convert a mailbox type from shared to regular and back again is later in the next
chapter.

The Default or My Organization calendar permission : The Default permission for calendars sets the level of
access that other users in the tenant or other federated organizations have to free and busy data. Typically, this
information is used by the Scheduling Assistant when it displays the set of available slots for a requested meeting.
The normal is to allow free and busy slots to be displayed, in which case the assistant can show when a person is
available, but it can be changed to display more information, such as the titles and locations for events. The default
permission is also used if a user opens another person’s calendar. As part of the new sharing model, the default
permission is renamed “My Organization” to clarify that this is how the permission is used. In addition, Microsoft is
removing the ability of clients to remove this permission from calendars.

Allowing Cross-Tenant Free/Busy Sharing

Being able to share calendars is one way to approach the need to enable cross-tenant scheduling. You can also create
a relationship with other Office 365 tenants to allow users to browse free/busy information for people in the other
tenant when they schedule meetings. To allow bi-directional access, you need to create an organization relationship in
both tenants. You can do this through the Organization section of the EAC. Navigate to Sharing and create a new
relationship under Organization Sharing . You can then enter the domains you want to share with, what degree of
calendar free/busy information you want to share, and whether to share for all users in the tenant or just those who
are members of a specified security group. Remember to have the other domain allow your users to access free/busy
information for their users in return.

You can also use PowerShell to create an organization relationship. This command in run in a tenant to create an
organization relationship with another tenant called “Domain1.” The free and busy access level is “Limited Details,
“meaning that users can see the name of a meeting, its location, and start and end date, but they cannot see the
attendees. Information about of meetings marked as private stays confidential.

[PS] C:\> Get-FederationInformation -DomainName Domain1.onmicrosoft.com | New-OrganizationRelationship -Name FreeBusyDomain1 -Enabled

$True -FreeBusyAccessEnabled $True

-FreeBusyAccessLevel "LimitedDetails" -FreeBusyAccessScope $Null

To complete the process, run the same command inside the other tenant, changing the domain name to point to the
first tenant and the name of the relationship appropriately. To test that everything works, create a new meeting and
use the scheduling assistant to browse the free and busy information for a user in the other tenant. If you can see
some details, you know that the relationship works. Alternatively, use the Test-OrganizationRelationship cmdlet to
verify that the configuration with another organization is correct.

The Search-Mailbox Cmdlet

The focus for eDiscovery searches within Office 365 is now on content searches and eDiscovery cases managed
through the Security and Compliance Center (Chapter 20). However, the need to search and remove content from
mailboxes existed long before eDiscovery searches. This simpler kind of search is still available in Exchange Online,
albeit only through PowerShell. Based on the number of appeals for help in constructing the correct syntax to scan
user mailboxes for inappropriate content or to remove copies of messages sent in error within the organization,
Search-Mailbox is an under-appreciated cmdlet. Search-Mailbox runs on both cloud and on-premises Exchange and
can:

Remove content from user mailboxes (including shared mailboxes). When items are removed from user
mailboxes, they are permanently removed. Permanent removal means that an item is irrecoverable, so you must
use Search-Mailbox with care as not even Microsoft can recover items removed by Search-Mailbox . Being able
to purge items permanently is useful when you need to do something like remove spam or malware from a set of
mailboxes or be sure that some objectionable content is eliminated. Office 365 content searches can have a
purge action added to remove items found by searches. However, a content search action cannot remove those
items permanently and is limited to removing 10 items at a time. See Chapter 20 for more information on content
searches.
Search against a wide variety of message properties using KQL syntax (the same search syntax used elsewhere in
Exchange Online and SharePoint Online). Note that searches conducted against on-premises Exchange 2010
servers with the Search-Mailbox cmdlet use a different syntax.
Copy messages from source to target mailboxes. This is often done to recover items from user mailboxes before
removing the items.
Search-Mailbox can search user and shared mailboxes. It cannot search group mailboxes or public folder
mailboxes. Use Office 365 content searches for this purpose. Search-Mailbox can find Teams compliance records
in user mailboxes, but not the compliance records for channel conversations stored in group mailboxes.

Only Exchange
Search-Mailbox hasn’t been updated by Microsoft for Office 365, so it can only handle mailbox content. If you need to
search for content across multiple Office 365 workloads, use a content search. Used wisely, Search-Mailbox is a
tremendously useful weapon for administrators, but you should always remember that fools rush in where cautious
administrators take their time to tune and test before they start removing items.

Restricting a search to specific folders : Search-Mailbox always searches the full mailbox. You can’t restrict the
search to specific folders. You can limit the search to the primary mailbox by passing the DoNotIncludeArchive
parameter. If you need a finer degree of control, you can use Exchange Web Services as this API allows the scope of a
search to be defined. An example of how to use EWS to perform mailbox searches can be found here .

RBAC Requirement for Search-Mailbox

Like other Exchange cmdlets, the ability for anyone to run Search-Mailbox is controlled by RBAC. Two RBAC roles
exist that include Search-Mailbox :

The Mailbox Search role allows users to search mailboxes. This role is included in the Discovery Management
role group.
The Mailbox Import Export role allows users to remove items from mailboxes. Removal is permanent, and the
items become irrecoverable. The Mailbox Import Export role is not included in any standard role group. To use it,
you must add the role to an existing role group (like Organization Management) or create a dedicated role group
and include the role. Only members of the role group that includes the Mailbox Import Export role can run
searches and remove the results of those searches. The restriction exists to ensure that only nominated
administrators can remove content from user mailboxes.

To discover who has access to the Mailbox Import Export role, we run the Get-ManagementRoleAssignment cmdlet:

[PS] C:\> Get-ManagementRoleAssignment -Role 'Mailbox Import Export' -GetEffectiveUsers | ? {$_.AssignmentMethod -eq "RoleGroup"} |
Format-Table EffectiveUserName, Name

EffectiveUserName Name

----------------- ----

TRedmond Mailbox Import Export-Organization Management-Delegating

James Redmond Mailbox Import Export-Organization Management-Delegating

Marc.Vigneau Mailbox Import Export-Organization Management-Delegating

Brian Weakliam Mailbox Import Export-Organization Management-Delegating

TempAdmin Mailbox Import Export-Organization Management-Delegating

Administrator Mailbox Import Export-Organization Management-Delegating

TRedmond Import Export Org Management

James Redmond Import Export Org Management

Marc.Vigneau Import Export Org Management

Brian Weakliam Import Export Org Management

TempAdmin Import Export Org Management

Administrator Import Export Org Management

In this case, the Mailbox Import Export role is part of the Organization Management role group and has been
delegated from that role group to the special Exchange Administrators role group that’s populated when users are
nominated to be an Exchange administrator through the Office 365 Admin Center. It is sensible to review the list of
those assigned the Mailbox Import Export role periodically to ensure that the right people have access.

Running Search-Mailbox
Although you can use Search-Mailbox to search a single mailbox, you can also pass a set of mailboxes retrieved with
the Get-Mailbox or Get-Recipient cmdlets to search up to a maximum of 10,000 mailboxes at one time. Up to 10,000
results can be returned for a query, which means that if more results are available, you need to split the search. For
example, if you need to remove 50,000 items from a mailbox, you would have to run the cmdlet to find and remove
items five times.

The goal is always to be as precise as possible with the search used to find messages. This can be a little challenging
at times because no GUI is available to help construct complex queries in KQL syntax, but you can build queries for
content searches in the Security and Compliance Center and use the same queries with Search-Mailbox . To help
understand what Search-Mailbox can do, let’s look at some common examples of its use.

Estimate Only
As you develop search queries to find, copy, and even remove mailbox content, you can use the EstimateResultOnly
parameter to estimate how many items a search is likely to retrieve. For example:

[PS] C:\> $Search = Search-Mailbox -Identity "Customer Services" -SearchQuery "My Best Customer" -EstimateResultOnly

Putting the output in a variable allows us to use the different results returned by the search.

[PS] C:\> Write-Host "The search found" $Search.ResultItemsCount "of size" $Search.ResultItemsSize

The search found 630 of size 32.33 MB (33,897,743 bytes)

Remember that the result returned is an estimate and that the actual results returned when you execute a search
might differ.

Find a Message Based on Subject

The need often arises to find and remove messages with specific subjects. In this example, we search a set of
mailboxes for messages that have “Kazuma” in the subject. This is the kind of search you might do if some malware
penetrated your defenses and arrived into user mailboxes.

[PS] C:\> Get-Mailbox -ResultSize Unlimited -RecipientTypeDetails UserMailbox | Search-Mailbox

-SearchQuery {Subject:"Kazuma*"} -TargetFolder "Malware Searches" -TargetMailbox MailboxSearches

-LogLevel Full -SearchDumpster

We fetch all user mailboxes in the tenant and pipe them to Search-Mailbox to search for messages with a string
starting with Kazuma in the message subject (the asterisk). Any messages found by the search are copied to the target
folder in the mailbox specified. We also pass the SearchDumpster parameter to force Search-Mailbox to look through
items in the Recoverable Items folder.

If we don’t want to copy found messages, we pass the -LogOnly parameter. This hasn’t been done in the example, so
any items matching the search criteria are copied to a folder structure under the target folder (see below).

If you only want to process a single mailbox, pass its identifier direct to Search-Mailbox :

[PS] C:\> Search-Mailbox -Identity James.Ryan

Building a Search Query to Find Messages

Another common reason for using Search-Mailbox is to rescue the situation when someone sends a message that they
didn’t intend to or sends it to the wrong set of people.

[PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"}| Search-Mailbox -TargetMailbox Searches

-TargetFolder "Retrieve Email" -SearchQuery {Body:"I have announced today the promotion of John Baker"} -LogLevel Full -
SearchDumpster

When people ask for messages to be “taken back” from recipients’ mailboxes, you should be able to find out a lot of
information about the problem message, including its originating email address, so we can improve the search query
by including the sender:

[PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"}| Search-Mailbox -TargetMailbox Searches

-TargetFolder "Retrieve Email" -SearchQuery {Body:"I have announced today the promotion of John Baker"
From:"Senior.Manager@outlook.com"} -LogLevel Full -SearchDumpster

In this case, we use the SMTP address of the sender. If we know what the display name (resolved name) of the sender
is, we can use it instead. If we are unsure of the exact SMTP address but know the domain the message came from,
we can use it instead, as in:

From:"*outlook.com"

If we know when the message was sent and its subject, we can include those pieces of information in the query:

[PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"}| Search-Mailbox -TargetMailbox Searches -TargetFolder "Retrieve Email" -
SearchQuery {Body:"I have announced today the promotion of John Baker" From:"Michael McArthur" Sent:"14-Aug-2018"
Subject:"Promotions"} -LogLevel Full

-SearchDumpster

When you use multiple keywords within a query, like the one above, Exchange combines them with AND operators
when it executes the search. You can include the OR operator in a query. For example:

From:"Michael McArthur" OR From:"Kim Akers"

In effect, what we’re doing here is building up a complex search query to focus in on the exact message we are
interested in. This might not be so important when you only search and copy messages, but it is critical when the time
comes to remove the messages from the source mailboxes.

Looking for Attachments

Another common scenario is when someone asks to find messages with an attachment or a certain attachment.
Perhaps the wrong file was circulated, or the attachment has a known virus lurking within it.

To find messages with an attachment, we include the HasAttachment keyword.

HasAttachment -eq $True

For example:

[{PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"} | Search-Mailbox -SearchQuery {Sent: "13-Aug-2018" HasAttachments -eq $True} -
TargetMailbox Searches -TargetFolder AttachmentSearch

-LogLevel Full -SearchDumpster

To look for messages with a specific attachment, specify the AttachmentNames keyword and the name of the
attachment in the search query. For instance:

[PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"} | Search-Mailbox -SearchQuery {Sent: "13-Aug-2018"
AttachmentNames:Sales_data_2018-7-1.CSV} -TargetMailbox Searches -TargetFolder AttachmentSearch -LogLevel Full -SearchDumpster

Keyword queries also support wildcard matching for attachment names. For example, you can look for all
spreadsheets starting with "SalesForecast" by including "AttachmentNames:SalesForecast* " in the search query. A
variant is to look for messages that have an attachment of a specific type. For instance, to look for messages with PDF
attachments, use “AttachmentNames -like ":*.pdf. " You can even look for messages of a certain size. For example, to
look for messages that have .DOCX attachments and are larger than 10 MB, you’d use

{AttachmentNames -like "*.DOCX" Size -gt 10 MB}

Searching by Date Range

Because Search-Mailbox uses KQL to interrogate the content indexes for mailbox databases, the queries that can be
constructed can be complex. Because dates are often used in searching, KQL includes some interesting date-based
ways to look for items. In this example, we look for items received from 1 June through 14 August 2018 (U.S. date
format).

[PS] C:\> Get-Mailbox –Filter {Office -eq "Dublin"} | Search-Mailbox –SearchQuery {Received:"06/01/2018..08/14/2018"
From:"*outlook.com"} -TargetMailbox Searches -TargetFolder DateSearch -LogLevel Full -SearchDumpster

The date format used depends on the default format used on your workstation. You can spell the dates out if you want
to avoid problems with different date formats. For example

Received:"1-Jan-2018..1-Sep-2018"

Here’s an example of specifying a canned period in a search. In this case, we want to look for items received in the
previous year:

[PS] C:\> Search-Mailbox –SearchQuery {Received:"last year"}

Other date intervals supported by KQL include "today", "yesterday", "this week", and "this month." Note that although
KQL supports ISO 8601 DateTime data types in searches, you cannot use date/time values when searching against
fields such as “Read” and “Sent” because the parser used by Exchange Online drops the time segment and uses only
the date. This means that you cannot search for a message sent or received at a particular time using a value such as
“Read:”20-July-2015T19:53.”

KQL Properties for Exchange

KQL queries can run against both Exchange Online mailboxes and SharePoint Online/OneDrive for Business sites.
However, each repository has a different set of searchable properties that match their usage. For example, Exchange
messages might have attachments, but SharePoint documents never do. For more information about the properties
you can use with content searches, see this support article . You can use any of the Exchange properties in a search
performed with Search-Mailbox .

Output from Search-Mailbox

Because searches are run interactively, it can be quite slow to search large mailboxes. Three factors contribute to the
search time: the numbers of mailboxes to be searched, the size of those mailboxes, and the number of items copied
from the mailboxes to the target mailbox. When it completes a search, Search-Mailbox reports its results for each
mailbox:

Identity : TRedmond

TargetMailbox : Search Investigations

Success : True

TargetFolder : \Search es\Tony Redmond-16/05/2018 15:16:08

ResultItemsCount : 3514

ResultItemsSize : 188.1 MB (197,276,026 bytes)

This output tells us the following:

1. A search results folder (Searches ) is created in the target mailbox (Search Investigations ). Under the search
root, you find an entry for each mailbox scanned by the search and the folders where items were found. In this
instance, the search scanned a single mailbox, and the results of the search are in the folder “Tony Redmond –
16/05/2018 15:16:08 ” (the date and time when the search started). A separate folder is created for each mailbox
scanned, if some results are found in that mailbox. Under this folder you’ll find sub-folders for the primary
mailbox and archive mailbox (if one exists and the search processes it), and then the set of folders to hold copied
items.
2. The search copied 3,514 items totaling 188.1 MB that matched the search criteria. The search results include
items found in the system Files folder and the Recoverable Items folder (if you specify the SearchDumpster
parameter). Search-Mailbox copies found items to folders in the target mailbox as it processes the source
mailbox, so you can check the target mailbox as the search proceeds to see what it finds. The names of the
folders holding the copied items are the same as those in the source mailboxes where the items were found. For
instance, if items are found in the Inbox and Sent Items folders in the source mailbox, the copied items are in the
Inbox and Sent Items folders.

In addition, the search generates a log report in the top-level target folder. If you specify “Full” for the LogLevel,
Exchange generates a ZIP file holding a spreadsheet (CSV) detailing the results for individual mailboxes and attaches
it to the log report.

Removing Mailbox Content

You can remove items from mailboxes with the Search-Mailbox cmdlet by specifying the DeleteContent parameter.
Because the items found by the search can be permanently removed and cannot be recovered afterwards, it’s easy to
imagine the potential havoc that might be wreaked on user mailboxes, so you need to be careful that the correct items
are found before you try to remove anything. You should make sure to take account of the following points.:

You cannot use the DeleteContent parameter unless your account is part of a management role group that holds
the RBAC “Mailbox Import Export” role.
Items subject to a litigation or in-place hold or have not exceeded the retention period specified for Single Item
Recovery are kept in the Recoverable Items structure of their host mailboxes. Users cannot access these items,
but the items still are available for search and compliance purposes. If you need to remove items under hold,
follow the guidance in this article .
If you need to keep a copy (for instance, to allow their source and content to be investigated), you can specify the
name of a target mailbox in the TargetMailbox parameter. In this case, the items are copied to the target mailbox
before they are removed from the source mailboxes. Note that if the logging level for the search is set to Basic or
Full, you must supply the name of a target mailbox and folder to store the log file. To remove messages without
copying them, omit the LogLevel , TargetMailbox , and TargetFolder from your command.
Have management backing in place and have approval clearly documented before any content is removed from
user mailboxes. Make sure that any removal follows the data governance policy for your organization.
Test, test, and test again before you begin to remove content. The WhatIf switch for the Search-Mailbox cmdlet is
especially useful here, as is the EstimateResultOnly parameter, which gives an estimate of the total number and
size of found items. Obviously, any search that returns thousands of items needs to be checked before you use it
for deletions. Make sure that the search criteria that are specified work. It would be a shame to clean out the
CEO’s mailbox just because of a slip in a PowerShell command. Remember that Exchange Online does not use
backups, so if you permanently remove an item, it is irrecoverable.

For example, this command searches all user mailboxes and permanently removes the items found by the search
query. You’ll be asked to confirm that deletions can occur and can respond for each mailbox or “A” for all.

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox | Search-Mailbox -SearchQuery {Subject: "Spam Email" AND Received:1-Apr-
2018..1-Aug-2018} -DeleteContent

Auto-Expanding Archives : Microsoft recommends that if you have mailboxes with auto-expanding archives, you
should be very careful when removing items using the Search-Mailbox cmdlet. You can certainly remove items from
primary mailboxes, but if you remove items from archives that are in the middle of the expansion process, a small
chance exists that some data loss might occur.
Audit Records for Search-Mailbox
Exchange captures two types of audit records for Search-Mailbox and writes them into the Office 365 audit log:

Records for each mailbox processed by the Search-Mailbox cmdlet: This record is captured even if no data is
removed. It reveals the search query and other parameter used. These records are found using the “Search-
Mailbox” operation.
Records logging the removal of items : The record reveals who ran the command and what items were removed
from a mailbox. If more than a few items are removed and the details of the items cannot fit in the audit data part
of a record, Exchange captures a series of records. These records are found using the “HardDelete” operation.

See Chapter 21 for details about how to search the Office 365 audit log.

Recovering Deleted Mailboxes

When you remove an Office 365 user account, a clock starts ticking and the Azure Active Directory object for the user
account stays in a soft-deleted state for a 30-day retention period. During this time, you can recover and restore the
data belonging to the user, including their mailbox. Once the retention period elapses, Office 365 removes all traces of
the user account and the account is irretrievable. You can use the Deleted users view in the Office 365 Admin Center
(Figure 6-14 ) to see the set of deleted users that are still within the 30-day retention period. As you can see, the list
includes guest accounts. You can select a user from this list and restore them at any time until their retention period
expires.

To restore a user account, select their entry to have Office 365 display the details of the account. If you’re sure that
you want to restore it, click Restore and give details of what the password should be for the restored account (you
can assign one, force the user to create one when they first sign in, or have a password auto-generated). Office 365
will then restore the user account to its pre-deletion state, including the mailbox (if they had one) as well as
memberships of any distribution and Office 365 Groups to which the account belonged. It takes between 15 minutes
and 30 minutes before Office 365 fully restores an account. After the restore process completes, the user should be
able to log-on and use their account as before. You can also restore deleted accounts with PowerShell. To return a list
of deleted Azure Active Directory accounts, use the command:

[PS] C:\> Get-MsolUser –ReturnDeletedUsers

To restore a deleted account, use the Restore-MsolUser cmdlet:

[PS] C:\> Restore-MsolUser –UserPrincipalName "Jack.Jones@Office365ITPros.com"

If the user principal name for the deleted account has been reused since the account was removed, you can assign a
new user principal name by passing it in the NewUserPrincipalName parameter to the Restore-MsolUser cmdlet.

Figure 6-14: Viewing a list of deleted users in the Office 365 Admin Center

Mailbox Recovery Troubleshooter : Microsoft obviously meets many situations when administrators have
discovered that they have removed the wrong mailbox and do not know what they should do to execute a recovery. To
reduce issues in this area, Microsoft created a mailbox recovery troubleshooter , a walk-through guide as to what an
administrator needs to do to recover a mailbox. It is not a wizard and processing a successful recovery needs some
skill in PowerShell and knowledge of the various PowerShell modules that are involved, but at least it’s a step
forward towards automated recovery.
Inactive Mailboxes
Sometimes you want to remove a user account because that person does not work for the company any longer, but
you need to keep their information for an extended period for legal or regulatory purposes or because it is a source
needed for an eDiscovery case. In the on-premises world you might take a backup of the user mailbox or export it to a
PST. Another way of carrying out the goal is to disable the user account in Active Directory and leave the account and
mailbox alone until needed by eDiscovery investigators.

However, these options are not as useful within Office 365 as they are in an on-premises deployment, which is why
inactive mailboxes exist. Removing items from Office 365 and exporting them to a PST takes time and removes the
data from indexes to make the mailbox items invisible to eDiscovery. Keeping disabled accounts around is a practical
choice, but you must pay the monthly cost of a license. Neither approach is particularly attractive. An inactive mailbox
belongs to an account that has been removed from Office 365 and has some held content. Because an inactive mailbox
is no longer used, you do not need to assign it a license.

Finding Inactive Mailboxes

You can use the Get-Mailbox cmdlet to discover whether any inactive mailboxes exist in a tenant. This command
retrieves the list of inactive mailboxes and reports the date when they were removed as shown by the last changed
timestamp for the object (the WhenChanged property is also updated if the mailbox comes under the scope of an in-
place hold). As you can see, some of the inactive mailboxes have been kept for quite a while.

[PS] C:\> Get-Mailbox –InactiveMailboxOnly | Sort WhenChanged | Format-Table DisplayName, WhenChanged

DisplayName WhenChanged

----------- -----------

Steve Smith 29-Mar-15 1:40:11 AM

Tom Perham 20-Apr-15 6:23:44 AM

David Keane (Inactive - under in-place hold) 01-Feb-16 9:14:50 AM

Jill Smith 01-Feb-16 9:14:50 AM

Like any other deleted mailbox, Exchange Online considers inactive mailboxes to be in a soft-deleted state. Therefore,
if you check for soft-deleted mailboxes, you will also see the inactive mailboxes. The difference between the two sets
is that Office 365 permanently removes soft-deleted mailboxes 30 days after they are first removed while the inactive
mailboxes persist until the last hold is removed. This command generates a list of soft-deleted mailboxes for a tenant:

[PS] C:\> Get-Mailbox –SoftDeletedMailbox | Format-Table DisplayName, WhenChanged

Although inactive mailboxes serve an extremely useful purpose, they need to be managed with care. Even though they
would have to ignore a warning prompt, it is entirely possible that an administrator might remove the last hold that is
securing some inactive mailboxes and remove the mailboxes in error, losing all the data contained within. Inactive
mailboxes are not visible within EAC and it is easy to miss the fact that a license has been removed from a mailbox or
that a hold covers some deleted mailboxes. For this reason, it is sensible to edit the account properties of inactive
mailboxes so that they become more obvious to administrators. For instance, you could update the display name
property to include a sign that the user account has an inactive mailbox that is on hold. Hopefully such a visual
reminder would be enough to stop a potentially embarrassing administrator mistake.

Making Mailboxes Inactive with Holds

The data is kept within Office 365 by placing the mailbox on a litigation or an in-place hold before it is removed. You
can place a hold on a mailbox as follows:

Put the mailbox on litigation hold. All the contents of the mailbox are held.
Include the mailbox in an Office 365 retention policy that keeps content for a set period.
Include the mailbox in an eDiscovery case that includes an in-place hold that includes the mailbox within its
scope. eDiscovery cases are managed through the Security and Compliance Center (see Chapter 20). Older in-
place holds belonging to Exchange eDiscovery searches are also respected. However, you cannot create new
Exchange eDiscovery cases.

Both eDiscovery searches and cases allow you to input a query to select and hold a subset of the content in a mailbox.
Do not input any search conditions if you want to hold everything.

You can even put an inactive mailbox on litigation hold to keep its contents. This is useful when a hold associated with
an eDiscovery search or case is released and you do not want the inactive mailbox to be removed from Exchange
Online. To put an inactive mailbox on litigation hold, use the command:

[PS] C:\> Set-Mailbox -Identity "Jill Smith" -InactiveMailbox -LitigationHoldEnabled $True

Checking Holds
The holds that might keep an inactive mailbox within a tenant include both org-wide holds (that apply to all
mailboxes) and specific holds. The difference between the two types are explained in the section about Office 365
retention policies in Chapter 19. Exchange Online holds some information about org-wide holds in its organizational
configuration, meaning that we can gain some insight into the holds that might be keeping inactive mailboxes by
running the Get-OrganizationConfig cmdlet. In the following output, we can see that five org-wide holds are in place
for mailboxes (“mbx”).

PS C:\> Get-OrganizationConfig | Select -ExpandProperty InPlaceHolds

mbx15382841af9f497c83f9efe73e51888d:1

mbx9696959111f74ecda8a40aef97edd2c2:1

mbx703105e3b8804a1093bb5cb777638ea8:1

grp703105e3b8804a1093bb5cb777638ea8:1

mbxc1e2d6f1785d4bf8a7746a26e58e5f66:1

grpc1e2d6f1785d4bf8a7746a26e58e5f66:1

mbxf6a1654abdba4712a43c354e28a4d56c:2

grpf6a1654abdba4712a43c354e28a4d56c:2

More specific holds are registered to the mailboxes that they cover. To see this information, we would have to run Get-
Mailbox like this:

[PS] C:\> Get-Mailbox -InactiveMailboxOnly | Format-Table DisplayName, InPlaceHolds, LitigationHoldEnabled

DisplayName InPlaceHolds LitigationHoldEnabled

----------- ------------ ---------------------

Emma Robinson {} False

Jodie Smith (Program Manager) {} False

Rob Young {d9eb7052cc0f4200b6a1ad0d6f2171ed...} True

Ed Banti {skp748f77b020124e6e8304e66021fb297b:3} False

Tom Perham {UniH4001d1c2-9438-4e46-9d14-80207a8099e9} True

David Keane (Inactive) {UniH4001d1c2-9438-4e46-9d14-80207a8099e9} False

Brian Jones {UniH1f53db1f-bb22-4748-bdad-192ff63e41c2} True

The top two users have no specific holds and are not on litigation hold, so we can then assume that these inactive
mailboxes are kept because of one of the org-wide holds. A more comprehensive exploration of how to interpret holds
is in Chapter 20.

When it detects that one or more holds exist for a mailbox, Exchange Online knows that the mailbox must be kept,
even if its user account has long disappeared. Although an inactive mailbox does not need an Office 365 license, a
suitable license must be assigned to the mailbox to allow the hold to be enforced before the user account is removed
from Office 365. The minimum license is Exchange Plan 2 or a separate Exchange Online Archiving license. As
mentioned earlier, user accounts are kept for 30 days after deletion and can be recovered through the Office 365
Administration Center (Figure 6-14 ) during this time. After this time, the user account disappears, and the mailbox
enters the inactive state, kept there by the in-place hold.

Exchange Online keeps an inactive mailbox until the last hold covering the mailbox is released or lapses. When this
happens, the Managed Folder Assistant permanently removes the mailbox when the 30-day retention period expires.
The retention period begins when the user account is first removed so an inactive mailbox that is newly released from
a hold might be removed very soon afterwards. All the information stays in the mailbox and can be accessed by
executing a content search and exporting the search results to a PST. See Chapter 20 for more information about
content searches.

Making a Mailbox Inactive Immediately

Normally you must wait 30 days for a deleted mailbox to become inactive because its matching user accounts stays in
the Microsoft Online Services database to allow the account to be recovered during that period. However, you can
remove a user account to create an immediate inactive mailbox by setting the RemoveFromRecycleBin switch when
running the Remove-MsolUser cmdlet. For example:
[PS] C:\> Remove-MsolUser –UserPrincipalName 'Kim.Akers@Office365ITPros.com'

–RemoveFromRecycleBin

Alternatively, you can remove a soft-deleted account through the Users section of the Azure Active Directory portal.
Select the Deleted Users view, then select the account you want to remove, and then click the Delete permanently
button.

Usually it's best to let nature (or Office 365) take its course and let user accounts degrade into inactive mailboxes, but
you never know when you might want to accelerate the process.

Restore or Recover Inactive Mailboxes

In addition to being able to export the contents of inactive mailboxes through eDiscovery searches, you can also
retrieve information by restoring or recovering an inactive mailbox . When you restore an inactive mailbox, Exchange
Online merges the contents of the mailbox into another mailbox. This might be done when a user needs to work with
the information contained in a mailbox that belonged to an ex-employee. Restoring data from an inactive mailbox
leaves the inactive mailbox intact and still available for eDiscovery.

When you recover an inactive mailbox, you bring it back to life and prepare it to be used by another user (or the user
who left and has now returned to work with the company). Recovery takes the inactive mailbox and transforms it into
a fully operational mailbox that is connected to the account of the user to whom the mailbox now belongs. No trace of
the old inactive mailbox is left after the recovery is complete.

Restore an inactive mailbox

The first step is to run the Get-Mailbox cmdlet to return a list of inactive mailboxes and identify which mailbox is to be
restored. The InactiveMailboxOnly switch tells Exchange Online that we only want to see inactive mailboxes that are
recoverable.

[PS] C:\> Get-Mailbox –InactiveMailboxOnly | Format-Table DisplayName, PrimarySMTPAddress

DisplayName PrimarySmtpAddress

----------- ------------------

Melissa Travers MTravers@office365itpros.com

Jill Smith Jill.Smith@office365itpros.com

It's possible that several of the inactive mailboxes might share the same display name or other attributes, so we need
to retrieve a unique value for the mailbox to pass to Exchange Online when the mailbox is restored. The Distinguished
Name is best for this purpose, so we'll use that. The value returned will be like:

CN=JS,OU=Soft Deleted Objects,OU=tenantname.onmicrosoft.com,OU=Microsoft Exchange Hosted

Organizations,DC=EURPR04A002,DC=prod,DC=outlook,DC=com

[PS] C:\> $Inactive = (Get-Mailbox –InactiveMailboxOnly –Identity "Jill Smith").DistinguishedName

We can now set up the restore with the New-MailboxRestoreRequest cmdlet to ask Exchange Online to fetch the data
from the inactive mailbox and move it into a target mailbox. Remember, a restore operation leaves the inactive
mailbox intact, so this is in effect a copy operation. The AllowLegacyDNMismatch switch allows the New-
MailboxRestoreRequest cmdlet to process the restore request even though the distinguished names (DN) of the
inactive and target mailboxes do not match. Normally, to safeguard against items being misdirected to a mailbox that
they don't belong to, New-MailboxRestoreRequest would refuse to copy items into a target mailbox if a DN mismatch
existed. Obviously, we want to proceed in this situation even though we know a mismatch exists, so we override the
natural caution of the cmdlet by telling it that it's OK to go ahead and copy the items:

[PS] C:\> New-MailboxRestoreRequest –SourceMailbox $Inactive –TargetMailbox Abrus@Office365ITPros.com –TargetRootFolder "Jill Smith
Old Mailbox" –AllowLegacyDNMismatch

Name TargetMailbox Status

---- ------------- ------

MailboxRestore Abrus Queued

The restore operation continues in the background. You can check it by running the Get-MailboxRestoreRequest
cmdlet. When the status is "Completed," all the data from the inactive mailbox should be in the target mailbox. The
target root folder specified in the restore request ("Jill Smith Old Mailbox") is in the target mailbox with all the folders
and items that belonged to the inactive mailbox underneath that root.

If the inactive mailbox owns an archive, you can restore items out of the archive and direct them to either the archive
of a target mailbox or to the target mailbox itself. New-MailboxRestoreRequest supports the SourceIsArchive switch
to control whether to copy items from the primary (the default) or archive mailbox and the TargetIsArchive switch to
control whether to restore items to the primary mailbox of the target or its archive. Restoring items to an archive
mailbox has the value of creating a clear separation between the restored items and the owner's own items in the
target mailbox.

Recover an Inactive Mailbox

Recovering an inactive mailbox means the transformation of a mailbox into one that is usable by a new user. To make
this happen, we run the New-Mailbox cmdlet to create a new mailbox and instruct Exchange Online to populate that
mailbox with the inactive mailbox.

[PS] C:\> $Inactive = (Get-Mailbox –InactiveMailboxOnly –Identity "Jill Smith").DistinguishedName

[PS] C:\> New-Mailbox –InactiveMailbox $Inactive –Name "Joe Healy" –FirstName Joe

–LastName Healy –DisplayName "Joe Healy"

–MicrosoftOnlineServicesID "Joe.Healy@Office365ITPros.com"

–Password (ConvertTo-SecureString –String "Testing123!" –AsPlainText –Force)

–ResetPasswordOnNextLogon $True

[PS] C:\> Set-MsolUserLicense –UserPrincipalName "Joe.Healy@Office365ITPros.com"

–AddLicenses "Office365ITPros:EnterprisePack"

In this case, we take the inactive mailbox of Jill Smith and use it to create a new mailbox under the control of Joe
Healy, a new Office 365 account created using the Office 365 provisioning workflow. After New-Mailbox completes,
the old inactive mailbox is gone, and its content is in the mailbox belonging to Joe Healy. To complete the process and
make the mailbox fully operational, you must assign an Office 365 license to the new mailbox by running the Set-
MsolUserLicense cmdlet as shown in the example.

You cannot use the recover method for an inactive mailbox while its user account still exists in Azure Active Directory.
This object is kept for 30 days after the account is removed. During this period, you can use the standard Office 365
Recover Deleted Users option to restore the account and reactivate the mailbox, but you can't recover the data to a
new mailbox. To test whether the user object still exists for an inactive mailbox, run Get-Mailbox as shown below. In
this case, Exchange returns a GUID, so you know that the object still exists.

[PS] C:\> Get-Mailbox –InactiveMailboxOnly –Identity 'Jill Smith' | Select ExternalDirectoryObjectID

ExternalDirectoryObjectId

-------------------------

636578d1-89fd-42e6-8b1d-237c96635a95

If you recover an inactive mailbox, any holds that existed on the mailbox when it was removed are not in place.
Instead, Exchange enables single item recovery for the mailbox and a retention hold is put in place for 30 days to be
sure that nothing is removed through the application of retention policies in that time.

Preserving a Tenant
Sometimes it is necessary to preserve a complete tenant, usually in the context of an investigation of fraud or similar
irregularity. Although not formally endorsed by Microsoft, it is possible to do this with a single license assigned to the
tenant administrator by making all the mailboxes in the tenant inactive. Make sure that the license assigned to the
administrator account is at least an Office 365 E3 plan (E5 is better as it allows access to advanced compliance
features). The most effective way to do this is:

1. Create an Office 365 eDiscovery case as explained in Chapter 20.

2. Create a hold in the eDiscovery case.
3. Add all the user mailboxes, SharePoint sites, and Exchange public folders as locations covered by the hold. It is
easiest to use a distribution list to add large numbers of mailboxes to the hold.
4. Make the query blank to keep all content.
5. Save the hold.
6. It takes a little while for the hold to apply to all workloads. When the status of the hold changes to On (Success),
you know that the hold is in force.
7. Delete the user accounts. Exchange recognizes that the hold exists and makes the mailboxes inactive.
The content in the inactive mailboxes stays indexed and accessible for searches. You can recover or restore selected
mailboxes at any time.

It is natural that employees will leave a business over time. Some are planned departures, some are amicable, and
some will be immediate and potentially traumatic. For legal reasons, you might need to preserve the employee's
mailbox in the latter case. In an on-premises environment, you can preserve mailboxes by disabling the user account.
If needed, you can reenable the account to restore a mailbox to full running order. The same is true for Office 365, but
you probably do not want to pay the monthly license for an unused account only kept in case the organization needs
some of the information in its mailbox in the future. Removing the need for licenses is why inactive mailboxes are so
valuable.

Securing the Data of Ex-employees

When you remove a user through the Office 365 Admin Center (Figure 6-15 ), the following happens:

The Office 365 license assigned to the account can be cancelled or kept for reassignment to another account.
Access to the user’s OneDrive for Business account is given to another user so that they can retrieve information
in that account after the person leaves.
The user’s mailbox is changed into a shared mailbox and access to the mailbox is assigned to another user. The
display name can be changed to reflect that the mailbox is now shared and an auto-reply created to inform others
that the person is no longer with the organization. Finally, you can remove email proxy addresses to free them up
for assignment to another recipient.

The focus of the workflow is to preserve the personal information of the deleted user. Later, after recovering whatever
information is available in the user’s mailbox, you could make the mailbox inactive to preserve the remaining
information for compliance purposes. The process is reasonable and suits the needs of many organizations. However,
sometimes you might want to use a different process, especially if some element of discord is present when someone
leaves. For this reason, we should discuss the different steps you can take to preserve all the data that a person might
have access to when the time comes for them to leave the organization.

Only cloud accounts can be removed with the Office 365 Admin workflow. If user accounts are hosted on-premises,
they can only be removed using an on-premises administration tool. In addition, the workflow does not handle other
Office 365 workloads. For instance, if the user you want to delete is the sole owner of some Office 365 Groups or
Teams, it does nothing to make sure that a new owner is added to the groups/teams. The same is true for traditional
SharePoint site collections where the user to be removed might be the sole administrator. In other words, treat the
Office 365 user removal workflow as something that deals with cloud mailboxes and OneDrive for Business accounts
but ignores other remnants of the user’s Office 365 activity. We’ll come back to this topic shortly.
Figure 6-15: The steps taken to delete a user when removed through the Office 365 Admin Center

Blocking an Office 365 User Account

The first course of action is to secure access to the user’s account to prevent any unauthorized removal of data. We
can then put their mailbox into a suitable status for long-term preservation. Changing the password for the account
prevents further access by the user. These commands block the account credentials and then change its password:

[PS] C:\> Set-MsolUser -UserPrincipalName Rob.Young@Office365ITPros.com -BlockCredential $True

[PS] C:\> Set-MsolUserPassword –UserPrincipalName Rob.Young@Office365ITPros.com

–NewPassword "Testing123" –ForceChangePassword $False

You can also block sign-ins and reset account passwords from the Office 365 Admin Center. To block an account,
select the account from the list of active users, edit its settings, and select Block Sign-in . The same effect is gained
by going to Edit sign-in status and changing the sign-in status from Allow the user to sigh in to Block the user
from signing in (Figure 6-16 ).
Figure 6-16: Blocking a user from signing into Office 365

To block the account from signing in, Office 365 will not renew the tokens needed by the sessions the user has open
with Office 365 applications and the sessions eventually stop because they cannot authenticate. After a short delay,
any sessions that the user has with Office 365 will end, including Outlook desktop, web sessions such as OWA and
OneDrive, and mobile clients like Outlook for iOS. It can take up to 60 minutes before all sessions disconnect.

You can also stop an account connecting by selecting Initiate in the Sign-out section in the OneDrive Settings of an
account properties. Alternatively, you can use PowerShell to do the same thing by running the Revoke-
AzureADUserAllRefreshToken cmdlet, For example:

[PS] C:\> Revoke-AzureADUserAllRefreshToken -ObjectId Jim.Smith@Office365ITPros.com

The cmdlet works by invalidating all the refresh tokens used to obtain new access tokens for Office 365 applications
by setting their expiry to the current date and time. When a user authenticates to connect to an Office 365
application, they create a session with that application. The session receives an access token and a refresh token from
Azure Active Directory. An Office 365 access token is valid for an hour. When that period elapses, an automatic
reauthentication process kicks in to get a new access token to allow the session to continue. This exchange can
happen if the refresh token is still valid and the account credentials are the same.

Because the forced sign-out invalidates the refresh tokens, the next time a session to an Office 365 application tries to
use its refresh token to renew its access, it discovers that the token has expired and so forces the user to
reauthenticate. As you have already changed the account password and blocked access, the user cannot
reauthenticate.

The exact time when an application enforces the requirement to reauthenticate depends on how much longer the
access token for the session is valid when you start the sign-out process and the actions taken by the user. If they stay
in the same page, the sign-out happens when the access token expires. On the other hand, the sign-out happens at
once if they move to another page within the application, refresh the browser, or open another Office 365 application.
This support article has more information about Office 365 access and refresh tokens .

Blocking an account from signing in through the Office 365 Admin Center is the right approach when you manage the
departure of a single employee. Using PowerShell commands is better when you need to secure a group of accounts
quickly or you want to script the process to secure an account for a departing employee.

Putting the Mailbox on Litigation Hold

After blocking the user account, the next step is to place its mailbox on litigation hold to ensure that no-one can
remove information from the mailbox. See Chapter 20 for more detail on how to apply litigation holds. As an example,
this command places a mailbox on litigation hold and updates the RetentionComment property with details of who
applied the hold and the date of its application.

[PS] C:\> Set-Mailbox –Identity 'Bad Employee' –LitigationHoldEnabled $True

–RetentionComment ("Employee Terminated on " + (Get-Date) + " by " [System.Security.Principal.WindowsIdentity]::GetCurrent().Name))

Chapter 20 also describes how to create eDiscovery cases in the Security and Compliance Center. You can create an
eDiscovery case whose only purpose is to manage a hold on mailboxes for ex-employees. Create a hold within the
eDiscovery case and add the mailboxes that you want to keep to the hold. You can also add the OneDrive for Business
sites belonging to ex-employees to the hold if you want to preserve their contents.

A risk always exists that a soon-to-be-terminated employee might hear of their impending fate in advance and not
react well to the news. They might then decide to remove information from their mailbox and other data repositories
(shared mailboxes, Office 365 group mailboxes, Teams conversations, SharePoint Online document libraries, their
OneDrive for Business personal site, and so on) before management makes the formal announcement. This is a tricky
situation because of the lack of backups for Office 365, but it is somewhat mitigated by the ability to assign retention
policies to SharePoint Online sites to stop people removing information.

Placing the mailboxes of affected employees on hold as soon as possible after management decides that the employees
are leaving will preserve the mailbox contents no matter what steps their owners take to remove information, but it
does create the possibility that employees will learn about their impending departure when HR asks IT to place
mailboxes on hold. One way to fix this issue is to create a special RBAC role that allows HR representatives to place
mailboxes on hold. This will not stop a maliciously-minded individual from erasing data from SharePoint Online or
their OneDrive for Business account, but you can ask Microsoft support to help recover information removed from
SharePoint using the backups that they take.

Long-term Mailbox Preservation

Two choices exist as to how to preserve the mailbox for the long term. You can convert it into a shared mailbox or
make it an inactive mailbox.

Convert to a shared mailbox : An EAC option exists for this purpose. Converting to a shared mailbox removes
the need for an Office 365 license (unless the new shared mailbox has an archive mailbox, you place the shared
mailbox on hold, or assign it a larger quota than the default 50 GB) and keeps the mailbox contents online so that
whoever needs to access the mailbox can open it. You should check that full delegate auditing is enabled for the
mailbox (see Chapter 21) so that Exchange records whatever actions the delegates take with items in the
mailbox. Because the purpose of the shared mailbox is to preserve information, you should prevent people
sending messages it by changing its SMTP address and hiding it from the GAL. You also need to remove the
mailbox from the membership of any distribution groups and Office 365 Groups to which it belongs and revoke
its access to other SharePoint Online sites. Converting the mailbox of a departed employee to a shared mailbox is
a simple and effective way to preserve mailbox contents that is preferable if you think that someone will need to
access the information in the mailbox in the short term.
Convert to an inactive mailbox : If no need exists for short-term access to the mailbox contents, you might
prefer to warehouse the mailbox by making it inactive. Because the mailbox is on hold, it is safe to remove the
Office 365 user account in the knowledge that Exchange will keep the mailbox until the hold lapses or you
remove the hold on the mailbox. No one will be able to log into the account and no need exists for an Office 365
license. When an account becomes inactive, Exchange removes it from address lists and distribution groups. A
mailbox can stay in the inactive state for a sustained period, but you can recover or restore an inactive mailbox if
needed. Information held in the mailbox still is available for eDiscovery searches.

You can run the Remove-MsolUser cmdlet to remove an Office 365 user account. In this example, the
RemoveFromRecycleBin switch tells Office 365 that it can remove the account at once and not keep it for 30 days.

[PS] C:\> Remove-MsolUser –UserPrincipalName 'BadEmployee@Office365ITPros.com' -RemoveFromRecycleBin

Removing a user account releases the Office 365 license that the account had. You can either reassign the license to a
new user or reduce the number of licenses that you pay for each month.

The steps needed to disable and preserve an employee's account in a hybrid deployment are different. Remember that
in this scenario, the on-premises Active Directory is the master and all changes to accounts and mailboxes must
happen on-premises and then synchronize to Office 365, which usually creates a delay before a guaranteed block
exists for an account.

Dealing with Other Office 365 Information

Of course, because Office 365 information exists in many other places than the user’s mailbox, administrators need to
do some added work to look for and recover anything considered valuable. These include:

The equipment issued to the employee. PSTs and documents might be on the hard drive of the user’s PC and
have information that invisible to administrators unless they can gain physical access to the drive. In terms of the
potential for information leakage, PSTs are especially vulnerable as users can remove them from the organization
on easily portable USB thumb drives along with copies of documents downloaded from SharePoint and OneDrive
for Business sites. You can scan the Office 365 Audit Log to figure out whether the ex-employee downloaded an
excessive number of files during their last weeks of employment, but it is impossible to know whether they
copied messages from Outlook to a PST and took that PST with them.
Given the widespread use of mobile devices, a terminated employee probably used a personal mobile device to
access their mailbox. Because of caching, it might be possible to use cached credentials to connect to the mailbox
with a mobile device even after the employee leaves. Given that a user might have multiple mobile devices, it is
best to issue a remote wipe for all ActiveSync devices registered with the mailbox. Any future attempt to access
the mailbox via ActiveSync will wipe the device that issues the connection request. Depending on how the mobile
device vendor has implemented the ActiveSync protocol within the email client, the wipe might affect some or all
personal data. To wipe all the ActiveSync devices connected to a user account, run the following PowerShell
command:

[PS] C:\> Get-MobileDevice –Mailbox 'Bad Employee' | Clear-MobileDevice

If BlackBerry devices are in use, you need to take steps to disable any of these devices used by ex-employees.
 InTune has more selective wipe capabilities for its managed devices than ActiveSync has. See this page for
information about how to perform a full or selective wipe of a mobile device with InTune.
The employee’s OneDrive for Business site. Because this site stores personal information belonging to the
removed user, its contents are out of sight to administrators unless you take steps to recover data after the
removal of a user account from Office 365. By default, unless the site is on hold, after the removal of a user’s
account, their OneDrive for Business site becomes a candidate for removal after 30 days by a background
process (the My Sites Clean Up timer job). A message goes to the user’s manager to warn that this will happen,
and a second reminder goes 3 days before SharePoint Online removes the data. However, if the information in
Azure Active Directory does not allow SharePoint Online to decide whom is the user’s manager, SharePoint
obviously cannot send the notification. If 30 days is too short a retention period, you can increase it to anything
up to 3,650 days. For instance, this command sets a retention period of one thousand days.

[PS] C:\> Set-SPOTenant -OrphanedPersonalSitesRetentionPeriod 1000

You can ensure continued access to the OneDrive sites owned by ex-employees by
configuring the SharePoint Online settings to automatically assign a secondary owner to
OneDrive for Business sites. Go to SharePoint Admin , User Profiles , My Site
Settings and then Setup My Sites , and select a user account to be the secondary
owner in the My Site Cleanup section. If Azure Active Directory does not hold details of
a manager for the removed user, SharePoint sends the reminders to preserve data to the
secondary owner, who can then arrange for a review of the data and the capture of
anything that needs preservation to another location (for example, another user’s
OneDrive for Business site or a SharePoint Online team site).

SharePoint Online team sites that the user manages. The other users who have access to these sites can continue
to work with the information held in the libraries and lists and the SharePoint Online administrator can grant
ownership to these sites to a different user.
Office 365 Groups and Teams. If the user is the sole owner of a group or team, you should assign that role to
another user. See Chapter 11 for more information about how to add an owner to a group or Chapter 13 for how
to change owners for a team. If you want to capture information about the conversations that an employee has
participated in, you can run a content search against all the groups that they belong to and their own mailbox for
items of type “MicrosoftTeams.” Once the search completes, you can then export the results to a PST or ZIP file.
The user might have created some Sways that the organization might want to keep for reuse. If you remove their
account, you will lose access to these Sways.

It is sensible to review the list of data sources annually as the potential places where users can store valuable
corporate data might grow as the feature set available within Office 365 expands.

Removing Calendar Events : When someone leaves the organization, it is possible that their mailbox is be the
organizer of events that exist in other peoples’ calendars. In many cases, you might want to remove those events
from calendars to allow someone else to reschedule replacement events (or not, as the case might be). The Remove-
CalendarEvents cmdlet cancels future events organized by a specific mailbox and sends out cancellation notices. For
example, this command cancels all meetings organized by Nancy Anderson from today’s date, which is probably what
you would do for an employee leaving the organization.

[PS] C:\> Remove-CalendarEvents -Identity "Nancy.Anderson@Office365itpros.com"

-CancelOrganizedMeetings

On the other hand, if the user is going away for an extended period and will eventually return, you can run the cmdlet
to cancel events for a date range. See this page for more information about the cmdlet.

Handling Inbound Email for Ex-Employees

When you convert a user mailbox to a shared mailbox, the mailbox keeps all the assigned email addresses and can
continue to receive email sent to the mailbox. Obviously, someone will need to process messages that arrive into the
mailbox to let the senders know that the intended recipient no longer works at the company. Alternatively, you can
change the assigned email addresses so that Exchange will “bounce” (send a non-delivery notification) any new
messages sent to the mailbox.

You can let people who try to contact the now-departed employee receive the normal non-delivery notification or you
can create a better experience by telling them why the person they tried to contact is no longer available. This
PowerShell code sets up internal and external auto-replies for the mailbox and adds a MailTip that is visible to
internal users. We also hide the mailbox from all address lists and enable mailbox auditing to track any access that
occurs to the mailbox.

[PS] C:\> Set-MailboxAutoReplyConfiguration –Identity "Terminated Employee" –InternalMessage "The person you are emailing no longer
works for us. Please refer communications to Mr. Manager"

–ExternalMessage "Mr. Terminated Employee is no longer an employee of this company."

–AutoReplyState Enabled


[PS] C:\> Set-Mailbox -Identity "Terminated Employee" -MailTip "Terminated Employee has left the company. Please do not send any more
mail to their mailbox" –HiddenFromAddressListsEnabled $True

–AuditEnabled $True

Auto-replies and MailTips will keep external and internal users informed about people that are no longer with the
company if the mailbox still exists in an active state. However, we might want to remove the mailbox completely after
harvesting any useful data in it. Exchange Online does not have a way to inform correspondents that a mailbox is no
longer in use, but we can do this by creating a shared mailbox to hold the email addresses previously assigned to
removed mailboxes or the old addresses of user mailboxes that have been converted to shared mailboxes. In effect,
you use the shared mailbox (which does not need an Office 365 license) to redirect inbound messages so that the
senders will receive some information to inform them that their correspondent is now longer available.

As explained in the chapter on addressing in the Companion Volume, although Exchange Online limits the number of
SMTP proxy addresses that you change assign to a mail-enabled object, a shared mailbox can easily hold 400 email
addresses. If you need to keep a higher number of addresses for departed employees, you can spread the addresses
over several mailboxes. One technique is to create a shared mailbox for each department so that external senders can
receive an auto-reply giving them details of a new contact within the department.

Of course, the shared mailbox will act as a black hole if you simply add the email addresses of departed employees to
it. To complete the process, you should create an auto-reply for the shared mailbox so that Exchange Online will
respond to the senders after it delivers inbound messages for the addresses assigned to the shared mailbox. Ideally,
the auto-reply will tell the sender that they should not use the address in the future.

Although there might be some value gained by receiving email in the shared mailbox, often companies do not want to
accumulate messages for people who have left. It can be an onerous task, not to mention a potential breech of
personal privacy, if someone accesses the shared mailbox to process and respond to the messages found there, so the
best solution is often to institute an automatic bounce mechanism to suppress inbound messages. This can be
achieved with the combination of a distribution group and a transport rule. Here's how:

Create a normal distribution group and add the shared mailbox (or mailboxes if necessary) to its membership.
Make sure that the group owner (by default, the administrator who creates the group) is not added to the
membership as they will not be able to receive new mail once the transport rule created in the next step
implements a block for group members.
Now create a transport rule (see below) to intercept messages sent to the members of the distribution group and
return a rejection notice to the senders. We provide suitable text to explain why messages are rejected by the
rule.

[PS] C:\> New-TransportRule "Block Email to Disabled Mailboxes" -SentToMemberOf "Disabled Mailboxes" -RejectMessageReasonText
"Unfortunately the person with whom you attempted to communicate is no longer with our company." -Enabled $True

Once enabled, the rule will reject any message sent to one of the SMTP addresses assigned to the shared mailboxes in
the distribution group. It is a simple but effective way to provide a better user experience.

When Someone Dies

All of us will die someday. In large organizations, statistics show that the likelihood exists that one or more employees
will die per 10,000 annually, depending on the average age of employees. Given the era we live in, when we do pass
on, we will leave behind many digital assets, among which might be a corporate mailbox. It’s a good idea to have a
procedure to preserve the mailboxes and other information belonging to dead employees. The steps that you might
take are like those that you use to preserve content in mailboxes for terminated employees and include:

Disabling or blocking the Office 365 account.

Decide if the mailbox should be made into an inactive mailbox or kept online. Alternatively, if you run a hybrid
deployment, you could move the mailbox back to an on-premises database that is reserved for this purpose. The
purpose of keeping the mailbox is to allow for the retrieval of any valuable information from the mailbox within a
retention period decided by HR and/or the legal department and in compliance with applicable regulations such
as GDPR. Some arrangement should be put in place to extract personal information from the mailbox and give it
to the employee’s family. For instance, some people store passwords and other valuable information in their
mailboxes that might be needed by their family following their death. Personal and corporate data might also
need to be recovered from the user’s OneDrive for Business site. After retrieving whatever information is
considered valuable from the mailbox, you might move it into an inactive state and keep the mailbox for a further
period.
Removing the mailbox from any distribution lists and Office 365 Groups/Teams to which it belongs. You can
discover what groups a mailbox belongs to by looking at the mailbox properties via EAC or by running some
PowerShell code (an example can be found in Chapter 7). If you make a mailbox inactive, its membership of
distribution lists and groups is automatically canceled.
If the mailbox is kept online, you should remove the mailbox’s access to any shared mailboxes. You can consider
adding the mailbox to a special “Departed Employees” distribution group, which is hidden from the GAL. Some
companies like to use a transport rule that blocks any incoming messages sent to the members of the “Departed
Employees” group. The rule might also generate a customized NDR back to senders to inform them of the sad
demise of the intended recipient. As described above, you can use the RejectMessageReasonText parameter for
the New-TransportRule cmdlet to create a thoughtful response to messages sent to a deceased employee. An
 alternative is to set an auto-reply message on the user’s mailbox by either logging onto the mailbox or using the
Set-MailboxAutoReplyConfiguration cmdlet (explained earlier in this chapter). An auto-reply allows for a more
obvious, human-friendly, and customized message that might, for instance, tell the sender whom they should
contact if the message related to company business. Another potential advantage of using the auto-reply
approach is that the inbound messages will be delivered to the mailbox. Of course, this creates another problem
in that someone must review the messages and decide which to keep and which to erase.
Eventually, the retention period will elapse, and you will either remove the user’s Office 365 account or cancel
the hold that keeps their mailbox inactive.

Remember that an Office 365 user is likely to have documents and other information in other places across the service
and that some arrangement must be made to track down this information and transfer it to the safe keeping of
another user.

Humane effectiveness : Although it is good to have a well-documented process to handle what happens when users
die, it is also good if an organization can show some humanity. Some large businesses delegate the authority to
manage the process of securing the digital assets of deceased employees to joint HR/IT teams who can flex and alter
the process as necessary to meet the needs of any specific circumstances that might arise. The IT members of the
team ensure that the technical processes are followed while HR ensures that everything is done in a humane and
thoughtful manner. It is a good approach to follow.

Compromised Accounts
A compromised account is one where someone outside the tenant (an attacker or hacker) manages to gain access to
the account resources. Usually this is because the attacker obtains the credentials necessary to sign-into the account,
perhaps because their username and password is compromised through a breach of another site (see the discussion
about “Have I been Pwned” in Chapter 8 of the companion volume).

Signs of unusual activity such as missing data, rules or a forwarding address appearing in the mailbox that the user
can’t remember setting up, or strange email from the mailbox might be indications that an account is compromised.
The steps necessary to secure the account and prevent further unauthorized access are:

Block the account.

Reset the account password and enable it for multi-factor authentication, if this is not already done.
Check all the mailbox settings to ensure that inbox rules, sweep rules, and forwarding addresses are valid (if a
forwarding address exists outside the organization, ask if a business purpose exists for forwarding email to that
address). Steps to check these elements are described earlier in this chapter.
If the account has administrative access, check what data might have been compromised through this access,
validate that the access is needed and remove it if not.
The Office 365 audit log (see Chapter 21) and Office 365 Cloud App Security can help you find what data the
account has accessed recently. You should check any documents the account uploaded to SharePoint or OneDrive
for Business to ensure that they don’t contain malware.
When the account is completely checked out, unblock it and give the new password to the user.

Microsoft’s advice on the topic is explained in this support article .

Even More Email

Individual user mailboxes take a lot of attention, but they are not the only kind of mailboxes and mail-enabled object
that exists within Exchange Online. Chapter 7 takes us into the domain of other types of mail-enabled objects,
including shared mailboxes and distribution groups.
Chapter 7: Managing Shared
Mailboxes and DLs
Tony Redmond

User mailboxes are a critical part of how people interact with Office 365, but they are not the only kind of mail-
enabled objects used by tenants. This chapter covers shared mailboxes and distribution lists. You can find coverage of
public folders and other email-enabled objects in the companion volume.

When Microsoft launched Office 365, they referred to distribution lists as distribution groups. This was fine because
there were not as many groups available inside Office 365 as there are now, especially following the introduction of
Office 365 Groups. In May 2018, to reduce the potential for confusion, Microsoft decided to revert to the older name
for the groups owned by Exchange Online and call them distribution lists again. We use this term in this chapter.
However, some documentation, terms used in the EAC and other GUIs, and cmdlet names still refer to distribution
lists as groups, which accounts for why you might see both terms used here and in Office 365.

Shared Mailboxes
Shared mailboxes have been part of the Exchange product since Exchange 2000. They meet the need to have a
mailbox to handle messages that a team of people might process, such as the team staffing a support desk. The
implementation of shared mailboxes in Exchange Online is like that found on-premises, with the disabled user object
used to instantiate the shared mailbox created in Azure Active Directory. Shared mailboxes have many uses, including:

To allow groups of users shared access to functional email. For example, all the support agents who staff a help
desk can use a shared mailbox to review and respond to messages sent to the help desk by users asking for help.
Sometimes these mailboxes are called functional mailboxes and are often used to ensure that incidents can be
managed effectively by multiple people across shift boundaries. A major attraction of this use is that messages
sent in response come from the shared mailbox rather than the individual user.
To allow users access to the mailbox of a colleague who has left the company. This is done by changing the
personal mailbox into a shared mailbox and assigning access to the mailbox to those who need access. This
technique is used extensively in on-premises organizations.

Mailbox delegation goes together with shared mailboxes because there is not much point in creating a shared mailbox
if you cannot then access the mailbox. Because a shared mailbox is linked to a disabled user object in Azure Active
Directory, access to its contents must be gained by delegating or granting permissions over the mailbox to other
users. The Get-Mailbox cmdlet enables us to discover the set of shared mailboxes known in a tenant:

[PS] C:\> Get-Mailbox –RecipientTypeDetails SharedMailbox

Name DisplayName Alias

---- ---------- -----

Customer Services Customer Services CServices

Office 365 Book Feedback Office 365 Book Feedback BookComments

Help Desk Corporate Help Desk HelpDesk

A shared mailbox can be hosted by Office 365 or, in a hybrid environment, on-premises. In this instance, the shared
mailbox is known as a remote shared mailbox when accessed from the other platform. It is best to keep shared
mailboxes on the same platform as used by the accounts that have delegated access to the mailboxes. In other words,
if you want to move some shared mailboxes to Exchange Online, move the mailboxes that access those shared
mailboxes to Exchange Online too. Office 365 places no limit on the number of shared mailboxes that you can create
within a tenant.

Licensing, Quotas, and Limitations of Shared Mailboxes

By default, a shared mailbox does not need an Office 365 license. However, a license is needed under the conditions
described in Table 7-1 . To assign a license to a shared mailbox, select the mailbox in the list of users in the Office 365
Admin Center, edit its properties, and assign the license under Product licenses .

Feature License

Enable an archive for a shared Exchange Online Plan 2 or Exchange Online Plan 1 with Exchange Online
mailbox. Archiving.

Place a hold on a shared mailbox. Exchange Online Plan 2.

Exceed the default 50 GB mailbox Exchange Online Plan 2.

quota.

Table 7-1: Licensing requirements for shared mailboxes

Microsoft documentation has always said that the default quota for shared mailboxes is 50 GB. Because of some
errors in the provisioning process, many shared mailboxes received a 100 GB quota without the need for a license.
These mailboxes keep their 100 GB quota unless the mailbox state changes. For example, if you convert a shared
mailbox to be a user mailbox and then convert it back again, the shared mailbox object then needs a license to keep
its 100 GB quota. New shared mailboxes created after the end of July 2018 will receive the correct 50 GB quota.

Here's a PowerShell script to report on the set of shared mailboxes in a tenant. The report shows the current number
of items in each mailbox, the size of the mailbox, the assigned quota, whether it is licensed, and if it has an archive. In
this case, only one shared mailbox is licensed, one was previously licensed but had the license removed, and the
others have never been licensed. All the mailboxes existed before Microsoft put the new provisioning process in place,
so they all have 100 GB quotas.

[PS] C:\> $SMbx = Get-Mailbox -RecipientTypeDetails SharedMailbox -ResultSize Unlimited

$Report = @()

ForEach ($S in $SMbx) {

$Archive = “Disabled”

If ($S.ArchiveName -ne $Null) { $Archive = “Enabled” }

$Quota = $S.ProhibitSendReceiveQuota.Split("(")

$Stat = (Get-MailboxStatistics -Identity $S.Alias | Select ItemCount, TotalItemSize)

$ShMbxSize = $Stat.TotalItemSize.Value

$Size = $ShMbxSize.ToString().Split("(")[0]

$ReportLine = [PSCustomObject][Ordered]@{

Mailbox = $S.DisplayName

TotalItems = $Stat.ItemCount

MailboxSize = $Size

Quota = $Quota[0]

Licensed = $S.SkuAssigned

Archive = $Archive }

$Report += $ReportLine }

$Report | Format-Table Mailbox, TotalItems, MailboxSize, Quota, Licensed, Archive -AutoSize

Mailbox TotalItems MailboxSize Quota Licensed Archive

------- ---------- ----------- ----- -------- -------

Customer Services 213 7.484 MB 100 GB False Enabled

Office 365 Book Feedback 301 7.278 MB 100 GB Enabled

Redirect for Removed Mailboxes 198 814 KB 100 GB Disabled

Redmond Shared Events 3777 190.5 MB 100 GB Disabled

Company Information 252 897.4 KB 100 GB Disabled

If a shared mailbox without a license exceeds 50 GB, Exchange stops delivering email to the mailbox. You can assign a
license to the mailbox to restore delivery. Because Exchange caches mailbox information, the newly licensed status
and 100 GB quota takes about 15 minutes to become effective, after which email delivery recommences.

If you convert a user mailbox holding more than 50 GB to a shared mailbox (often done to keep mailboxes for ex-
employees), Exchange continues to deliver mail to the mailbox while the mailbox is licensed. In most cases, people
convert user mailboxes to shared mailboxes to free up Office 365 licenses, so if you go ahead and do this, Exchange
detects that the mailbox holds more than 50 GB and ceases email delivery. You can restore delivery by removing
content from the mailbox to get it under the 50 GB quota or by assigning a new license.

Moving shared mailboxes from on-premises Exchange organizations will fail if they hold more than 50 GB and the
MailUser object for the target s hared mailbox in Exchange Online is unlicensed. To allow migrations to succeed (up
to the 100 GB quota), assign a license to the MailUser object. This limitation exists for all migrations performed by the
Mailbox Replication Service (MRS) or third-party migration utilities.

An unlicensed shared mailbox can be placed on hold through PowerShell or the EAC as no check is performed to
ensure that a license is in place, but it is a technical breach of Office 365 licensing.

Apart from making sure that you meet licensing requirements, some limitations exist for shared mailboxes. The most
important limitation is that the Exchange ActiveSync (EAS) protocol does not support shared mailboxes nor do the
Outlook for iOS and Android clients, meaning that you must use IMAP to access shared mailboxes from mobile
devices. We will cover how to do this shortly. Although companies often use shared mailboxes for functional purposes,
apart from inbox rules, Exchange doesn’t come with any out-of-the-box way to automate processing of messages that
land into a shared mailbox, so manual intervention is usually needed to handle new email. Performance for a shared
mailbox can be problematic if more than twenty users try to access it concurrently. Finally, users who have an
Exchange Online kiosk license cannot access a shared mailbox.

Creating a New Shared Mailbox

Figure 7-1: Creating a new shared mailbox with the Office 365 Admin Center

New cloud-based shared mailboxes are created through the Groups section of the Office 365 Admin Center
(previously, you could only create shared mailboxes through the EAC). Not much information is needed to create a
shared mailbox (Figure 7-1 ). After you click Add, Office 365 needs a short delay to provision the mailbox and make it
ready to add the users who will access and use the mailbox. Remember that users do not log into a shared mailbox
like you do with a regular mailbox and that access to its contents occurs by opening the mailbox as a secondary
resource within Outlook or OWA.

Mailbox Delegates and Permissions

After creating the new shared mailbox, you must assign permission to the mailbox to allow access to those who need
to work with it (its delegates). You can assign permissions when creating a new shared mailbox or by editing the
mailbox object in the Office 365 Admin Center or through PowerShell. In most cases, you deal with two separate
permissions:

SendAs permission allows a user to send messages from the shared mailbox that appear as if the messages came
from the shared mailbox rather than the user. This is the best situation when you want replies to conversations
started from the mailbox to flow back to the shared mailbox.
FullAccess permission allows a user to open the shared mailbox and access its contents. Full Access does not
mean that a user has SendAs permission. Sometimes Office 365 refers to this permission as “Read and manage
mail to this mailbox”.
After you create a new shared mailbox with the Office 365 Admin Center, you can Add members to this mailbox ,
which is the way that Office 365 refers to the action of giving users permission to access the shared mailbox. Figure 7-
2 shows the addition of two users as members for a shared mailbox. When someone is a member of a shared mailbox,
it means that they have both FullAccess and SendAs permissions. Because Exchange caches permissions for better
performance, it can take up to an hour for the new permission to be effective. In addition to a refresh of the server
caches, clients need to be updated that a mailbox has newly-assigned access to a shared mailbox. For example,
Outlook clients learn about access to a shared mailbox through its Autodiscover process, which runs every 15 minutes
for this purpose.

Figure 7-2: Assigning permissions to a shared mailbox

The FullAccess permission allows a user to work with all folders and items inside the shared mailbox, and the SendAs
permission allows them to send messages for the shared mailbox. The SendAs permission exists for shared mailboxes
because they have no personality of their own. Therefore, you always want to send a message as if you had assumed
the identity of the shared mailbox. These permissions are the same that you can assign to personal mailboxes and
users need both permissions to work with a shared mailbox as if it was their personal mailbox.

As we will discuss in a little while, Exchange Online also supports the Send on Behalf Of permission, which also allows
a user to send a message for the mailbox owner (or, as explained later, for the group mailbox owned by an Office 365
group).

If you need to assign access to a shared mailbox to multiple users, it is often more convenient to assign permission to
a distribution list, which must be a security-enabled group. You cannot assign permissions for a shared mailbox to a
normal distribution list because these groups are not security principals and you cannot assign permissions to an
Office 365 group or a dynamic distribution list either. Remember that assignment is a one-time operation and that you
grant permission to the current membership of the distribution list. Users who join or leave the group will not gain
permission or have their permission revoked.

At any point, you can edit a shared mailbox to add or remove members or work with individual permissions. Select the
mailbox in the Office 365 Admin Center to reveal its current properties. Then select Edit in the Members section to
add or remove members or Customize permissions to assign or remove individual permissions (FullAccess (Read
and manage), SendAs , and Send on Behalf of ) to users or groups (Figure 7-3 ). Once again, it will take a little while
before the amended permissions become fully active across the tenant.
Figure 7-3: Customizing permissions for a shared mailbox

Shared mailbox or Groups : On the surface, Office 365 Groups (Chapter 11) seem like a better and more modern
choice as the basis for team sharing than a shared mailbox. That statement is often true, if the requirement is to
collaborate based on shared documents; calendar, and conversations with perhaps a link to a plan managed by
Microsoft Planner (Chapter 15). However, shared mailboxes still have some unique strengths. For instance, shared
mailboxes allow access to a full range of folders rather than the limited set used in a group mailbox. In addition,
shared mailboxes support shared contacts and tasks while these are unavailable to groups, an issue that can be very
important to customer support or sales teams. The point is that the two types of mailboxes are suitable for different
purposes. Think about how people need to share information and what type of information they need to work with
before you select which type to use.

Handling Messages Sent from Shared Mailboxes

If you use a shared mailbox for customer communications or a similar purpose, you probably want to keep any replies
sent by people using the SendAs or Send on Behalf of permission in the mailbox. The default behavior is that
Exchange keeps messages in the mailbox of the person who sends a message, even if they are replying as or on behalf
of the shared mailbox. This is fine for personal mailboxes, but not so good for shared mailboxes. Consider the example
of a mailbox used to receive customer comments or complaints where a team of support agents are members. If an
agent answers a message, their mailbox stores the reply and none of the other team members know that the customer
has received a response (or what that response said). Accordingly, you can edit the properties of the mailbox to
change the behavior for Sent Items. In Figure 7-4 , we see that the sliders controlling how Exchange handles
messages sent by members using both the SendAs and Send on Behalf of permissions are set to On, meaning that
Exchange will now store copies of messages in the Sent Items folder in the shared mailbox. Copies also exist in the
mailbox of the account who replies.
Figure 7-4: Controlling the Sent Items behavior for a shared mailbox

You can also control these settings with PowerShell by running the Set-Mailbox cmdlet. For example:

[PS] C:\> Set-Mailbox -Identity SharedMailbox -MessageCopyForSentAsEnabled $True

-MessageCopyForSendOnBehalfEnabled $True

To ensure consistency and enable these settings on for all shared mailboxes in the tenant, the command is:

[PS] C:\> Get-Mailbox -RecipientTypeDetails SharedMailbox | ? {$_.MessageCopyForSendAsEnabled -eq $False -or

$_.MessageCopyForSendOnBehalfEnabled -eq $False} |

Set-Mailbox -MessageCopyForSendOnBehalfEnabled $True -MessageCopyForSentAsEnabled $True

If you run a hybrid environment, some added considerations exist when interoperating with on-premises Exchange
servers.

Somewhat along the same vein is the situation that occurs when a member removes items from a shared mailbox.
Logically, you might think that these items would go into the Deleted Items folder of the shared mailbox. However,
Outlook moves such items into the Deleted Items folder of the delegate's mailbox. To change this behavior, update the
system registry by creating a new DWORD value at:

HKEY_CURRENT_USER\Software\Microsoft\Office\[xxx]\Outlook\Options\General\DelegateWastebasketStyle

Replace [xxx] with 14.0 for Outlook 2010, 15.0 for Outlook 2013, and 16.0 for Outlook 2016. Set the value to 4 (four)
and restart Outlook. Deleted items will now stay in the Deleted Items folder of the shared mailbox, which is really
where they should be.

Creating and Managing Shared Mailboxes with PowerShell

The PowerShell command to create a new shared mailbox is straightforward as all you should specify is the name,
alias, and primary SMTP address (which must belong to the tenant domain). The other properties that you would
expect to assign to a mailbox such as the MRM policy, RBAC policy, and mailbox quotas are set automatically. This
command is an example of how to create a new shared mailbox.

[PS] C:\> New-Mailbox –Shared –Name "End User Services" –Alias EndUserServices

–PrimarySmtpAddress "EndUserServices@Office365ITPros.com"

–UserPrincipalName "EndUserServices@Office365ITPros.com"

After creating the new shared mailbox, we need to add some members. Somewhat confusingly, we need to run two
different cmdlets to grant the full set of permissions over the mailbox. The reason why this situation exists is that we
need to grant a user permission to open the mailbox and work with folders and then the right to send as (impersonate)
the mailbox. In other words, the first is an access right, the second, is the right to do something for the mailbox. In
this example, we use the Add-MailboxPermission and Add-RecipientPermission cmdlets to grant Kim Akers FullAccess
rights to the mailbox and then the SendAs permission.

[PS] C:\> Add-MailboxPermission –Identity 'Office 365 for IT Pros eBook Feedback' –User 'Kim Akers'

–AccessRights FullAccess

[PS] C:\> Add-RecipientPermission –Identity 'Office 365 for IT Pros eBook Feedback' –Trustee 'Kim Akers' –AccessRights SendAs –
Confirm:$False

You can remove any extraneous permissions with the Remove-MailboxPermission or Remove-RecipientPermission
cmdlets. For example:

[PS] C:\> Remove-MailboxPermission –Identity 'Office 365 for IT Pros eBook Feedback' –User 'Kim Akers' –AccessRights FullAccess -
Confirm:$False

[PS] C:\> Remove-RecipientPermission –Identity 'Office 365 for IT Pros eBook Feedback' –Trustee 'Kim Akers' –AccessRights SendAs –
Confirm:$False

It is sensible to conduct periodic reviews of the permissions that are assigned to shared mailboxes and to remove
permissions that are no longer needed. You can use the Get-MailboxPermission and Get-RecipientPermission cmdlets
to check the existing permissions. Here is an example of using the Get-RecipientPermission cmdlet to view details of
accounts that have the SendAs permission for a mailbox. The same command works for both shared and personal
mailboxes.

[PS] C:\> Get-RecipientPermission –Identity 'Office 365 for IT Pros eBook Feedback'

Identity Trustee AccessControlType AccessRights Inherited

-------- ------- ----------------- ------------ ---------

Book Feedback NT AUTHORITY\SELF Allow {SendAs} False

Book Feedback Tony Redmond Allow {SendAs} False

Book Feedback Michael Van Horenbee Allow {SendAs} False

Book Feedback Paul Cunningham (Boo Allow {SendAs} False

Book Feedback Kim Akers Allow {SendAs} False

The Get-MailboxPermission cmdlet returns the access permissions for a mailbox. In this example, we check the
permissions for a mailbox and trim the returned set to only report user accounts. If the full set is shown, it will include
many system accounts and management role groups that have access to mailboxes.

[PS] C:\> Get-MailboxPermission –Identity 'Office 365 for IT Pros eBook Feedback' | ? {$_.User –like "*@*"} | Format-Table User,
AccessRights

User AccessRights

---- ------------

Tony.Redmond@office365itpros.com {FullAccess}

MVH@office365itpros.com {FullAccess}

PCunningham@office365itpros.com {FullAccess}

We can also use the permissions on shared mailboxes to discover the list of mailboxes that a user can access:

[PS] C:\> ForEach ($Mbx in Get-Mailbox -RecipientTypeDetails "SharedMailbox") {

ForEach ($Users in Get-MailboxPermission -Identity $Mbx.Alias | Where {$_.User -eq

"Tony.Redmond@Office365itpros.com"}) { $Mbx.DisplayName }}

Send on Behalf Of permission

Exchange Online also supports the ability for someone to hold the Send on Behalf Of permission for a mailbox. The
difference in terms of functionality between the SendAs and Send on Behalf Of permissions is the degree of
impersonation implied. When a user sends a message on behalf of another person, the message is clearly marked as
such and you know that someone else has taken responsibility for composing the message. The Send On Behalf Of
permission is intended to cover scenarios such as when an assistant processes email on behalf of an executive. You
know that the executive did not personally send the message (because that fact is clear in the message header), but
the message has still come from their mailbox. In the world of letter-writing, using the Send on Behalf Of permission
to send a message is the equivalent to signing a letter for someone and adding “pp” (per pro) beside your signature.

You can use the Set-Mailbox cmdlet to grant the Send on Behalf Of permission to a shared mailbox, just like you can
grant the permission to send messages on behalf of distribution lists or dynamic distribution lists with the Set-
DistributionGroup and Set-DynamicDistributionGroup , or indeed, for an Office 365 Group using the Set-UnifiedGroup
cmdlet. The same syntax is used in all cases. For instance, to grant the permission for the Customer Services shared
mailbox to Jill Smith:

[PS] C:\> Set-Mailbox –Identity "Customer Services" –GrantSendOnBehalfTo "Jill Smith"

You can even pass a comma separated list of user accounts to grant permission at the same time, as in this example:

[PS] C:\> Set-Mailbox –Identity "Customer Services"

–GrantSendOnBehalfTo "Jill Smith", "Jack Jones", "Paul Cunningham"

To see the accounts that have hold Send on Behalf Of permission to mailboxes, you can run this command:

[PS] C:\> Get-Mailbox -RecipientTypeDetails Shared | ? {$_.GrantSendOnBehalfTo -notlike ""} |

Format-Table Name, GrantSendOnBehalfTo

Even if you grant an account the FullAccess and Send on Behalf Of permissions to a mailbox, this is different from
giving the account the Send As permission. Send As is a separate permission to assign users the ability to send
messages as if the sender is the mailbox owner. If a user has both the Send As and Send on Behalf Of permissions for
a mailbox, Exchange applies the Send As permission when they send a message.

Remember to change the behavior for Sent Items storage as described earlier if you want to keep copies of messages
sent using the Send As or Sent on Behalf Of permissions in the mailbox.

Disappearing messages? Sometimes a delegate who has full access to a mailbox will not see the same messages as
the mailbox owner sees. The reason might be simple. If the mailbox owner marks messages as private or confidential
(set through message properties (click ALT+Enter) with Outlook or through Show Message Options in OWA), then
Exchange Online will not reveal those messages to delegates. They are, after all, private. If a mailbox owner wants to
reveal private items to their delegates, they must set the necessary permission. To do this with Outlook 2016, select
the Inbox and then File, Account Settings, Delegate Access, select the delegate, then Permissions, and set the
"Delegates can see my private items" checkbox. Your choice applies to all folders in the mailbox and not just the
Inbox.

Who Sent That Mail?

One of the questions that often occurs is who sent a certain message from a shared mailbox. The question does not
arise when someone uses the Send on Behalf Of permission because their name appears in the message header, but
SendAs takes the name of the shared mailbox and does not tell you who created and sent the message. If you enable
mailbox auditing for the shared mailbox, Exchange captures audit records for actions such as SendAs and you can
search those records to discover the sender. You must enable mailbox auditing for shared mailboxes after creating
them as otherwise Exchange does not generate the audit records. See Chapter 21 for details about Exchange mailbox
auditing and how to enable auditing by running the Set-Mailbox cmdlet. Be sure to enable auditing for delegate
activities as these are the most relevant for shared mailboxes.

In this example, we run the Search-MailboxAuditLog cmdlet to find the audit records for the Customer Services
shared mailbox for a five-hour period and then filter the records to focus in on those for SendAs events. We then
display the name of the user who signed into the shared mailbox and the date and time of the event. By comparing the
date and time with the timestamp in the message header, you know who sent the message.

[PS] C:> Search-MailboxAuditLog -Identity "Customer Services -LogonTypes Delegate -StartDate "6-Aug-2017 12:00" -EndDate "6-Aug-2017
17:00" -ShowDetails | ? {$_.Operation -eq "SendAs"} | Select LogonUserDisplayName, LastAccessed

The Office 365 audit log ingests the mailbox audit records from Exchange alongside audit data generated by other
applications. You can search these records using the audit log search in the Security and Compliance Center or by
running the Search-UnifiedAuditLog cmdlet. If you suspect you know who sent the message, you can refine the search
by passing their email address as the value for the UserIDs parameter. Otherwise Office 365 returns all audit records
for all mailboxes where SendAs events occurred. See Chapter 21 for an example of how to search the Office 365 audit
log for SendAs events.
Accessing Shared Mailboxes
Once created, users can access the new shared mailbox through Outlook or OWA. If a user has FullAccess permission
for a mailbox, they do not have to do anything specific as an auto-mapping process kicks in to inform Outlook to
include the shared mailbox in the list of resources that it opens. Auto-mapping, which you need if you want to access
an archive belonging to a shared mailbox, occurs every time Autodiscover refreshes information for Outlook, so it will
occur shortly after Outlook starts and every 60-90 minutes thereafter. After Outlook refreshes its set of resources, the
shared mailbox appears in the set of resources just like any other mailbox. For more information on opening and using
shared mailboxes in Outlook, see this support article .

You can override auto-mapping by including a shared mailbox in an Outlook profile. This gives an explicit instruction
to open the mailbox for every Outlook session and is the older method used to access a shared mailbox. In this case,
you click More Settings when editing the profile, go to the Advanced tab and click Add to enter the names of the
shared mailboxes that you want Outlook to open. You can use the SMTP address, alias, or display name to tell
Exchange which shared mailbox to use.

If you use Outlook in cached Exchange mode, Outlook depends on the OAB to check unknown addresses (except when
a message is addressed to a fully-formed SMTP address). The new shared mailbox will not be present in the OAB until
after Exchange Online has updated the OAB files and distributed them to clients. This process can take 24 hours or
more, so if you want to send messages as the shared mailbox in the interim, use the online Global Address List (which
has the shared mailbox because the mailbox joins the address list once after its creation) to lookup the name that you
enter in the “From:” field. If you do not do this, Exchange Online will not be able to deliver the messages because the
address that you enter will not have the necessary information to allow it to be checked and you will receive a non-
delivery notification containing the following error:

This message could not be sent. Try sending the message again later, or contact your network administrator. Error is [0x80070005-
00000000-00000000].

Opening Shared Mailboxes in OWA

Although OWA does not use Autodiscover or have a profile, the client can also open and use shared mailboxes. To
access a shared mailbox, expand the full set of folders in your mailbox and right click the name of your own mailbox in
the folder list. Then select the “Add shared folder ” option from the menu. Then input the name of the shared
mailbox that you want to open (Figure 7-5 ). OWA validates that you have the correct permissions and if all is in order,
will open the mailbox and display it in the folder list. This operation will not affect the set of resources shown in
Outlook, just like auto-mapping does not influence the set of folders available to OWA. Both clients function
independently of each other.

Figure 7-5: Opening a shared mailbox with OWA

Another way of gaining access to a shared mailbox is to click the mailbox icon in the top right-hand corner of the
menu bar and then Open another mailbox . This method opens another session with the shared mailbox. Once a
shared mailbox is open, you can access its contents in the same way as any other mailbox. You can send messages as
if they came from the mailbox if you have the SendAs permission. You can select the mailbox to use by inputting the
name of the shared mailbox into the “From:” field of the message so that the message comes from the shared mailbox
rather than you.

Logging on to a shared mailbox : The usual course of events is to grant access to a shared mailbox and have users
open the shared mailbox as an added resource in either Outlook or OWA. However, at the time of writing, it is
possible to change the password for the account for the shared mailbox and log on to the mailbox directly, without
going anywhere near a user's personal account. Using this method is frowned upon before several reasons. First, it
depends on password sharing, which is never a good thing, and second, according to Office 365 Terms and
Conditions, once you perform a direct logon to an account, you need to license that access because the mailbox is
considered as no longer shared.

Mailbox Automapping
Like their on-premises counterparts, Exchange Online shared mailboxes can be automapped as an Outlook resource
who have full access to the mailboxes. Automapping means that Autodiscover returns details of the shared mailbox in
the XML manifest generated when an Outlook client queries the server to discover the set of resources available to it.
Because the shared mailbox is part of its resource set, Outlook automatically opens the shared mailbox. An example of
the XML from an Autodiscover manifest to define a shared mailbox as a resource is below.

<AlternativeMailbox>

<Type>Delegate</Type>

<DisplayName>Editing Team</DisplayName>

<SmtpAddress>E ditingTeam@Office365ITPros.com</SmtpAddress>

<OwnerSmtpAddress>E ditingTeam@Office365ITPros.com</OwnerSmtpAddress>

</AlternativeMailbox>

Including a shared mailbox in the Autodiscover manifest means that Outlook desktop opens the mailbox no matter
what workstation the user uses to run Outlook. Although automapping usually is convenient, you might not always
want to automap a shared mailbox for all users. If so, you must remove the FullAccess permission for the mailbox
from that user’s account and then grant it again, this time specifying that auto-mapping should not occur. No options
are available for this operation in the EAC so it has to be done through PowerShell (an interesting example of how to
remove automapping for all shared mailboxes is available online ). Another way of handling the issue is to use the
Remove-MailboxPermission cmdlet. For example, this command removes all automapping for the shared mailbox
passed in the identity parameter. After running the command, users who have Full Access to the shared mailbox will
continue to have the access but automapping will not happen.

[PS] C:\> Remove-MailboxPermission –Identity 'Customer Services' -ClearAutoMapping

You can clear the complete set of mailbox permissions from a mailbox by running Remove-MailboxPermission with the
ResetDefault switch. This instructions Exchange to reset the mailbox back to its default state by removing all
delegated permissions. Because they are folder-level permissions, the Send As and Send on Behalf Of permissions are
unaffected.

[PS] C:\> Remove-MailboxPermission –Identity 'Customer Services' -ResetDefault

Note that if you cannot simply flip the automapping switch from $True to $False once a set of permissions exist. If you
make a mistake, you must remove the permissions and then add them back to the mailbox, making sure that the
correct automapping choice is in place this time. In this example, we use the Remove-MailboxPermission cmdlet to
remove the permission from the mailbox and then the Add-MailboxPermission cmdlet to add the permission back
again:

[PS] C:\> Remove-MailboxPermission –Identity "Shared Mailbox" –User "UserWithAccess"

–AccessRights FullAccess

[PS] C:\> Add-MailboxPermission –Identity "Shared Mailbox" –User "UserWithAccess"

–AccessRights FullAccess –AutoMapping:$False

The next time Outlook refreshes its list of resources from Autodiscover, it will remove the shared mailbox.
Alternatively, you can close and re-open Outlook to accelerate the process.

Converting a Shared Mailbox to a Regular Mailbox

The EAC includes a method to convert a shared mailbox to a regular mailbox (and vice versa). Once the conversion is
complete, you must assign a license and reset the password. The license is necessary to allow the newly regularized
mailbox to have full functionality while the reset password allows the person who takes ownership of the mailbox to
access it. Shared mailboxes have disabled user objects, so they do not need a password. Instead, access to the shared
mailbox is gained by giving permissions to the users who need to access the mailbox.

The EAC converts a shared mailbox by running the Set-Mailbox –Type ‘Regular’ command. The reverse is also
possible, in which case the EAC runs the Set-Mailbox –Type ‘Shared’ command. In neither instance does the EAC do
anything to assign a license or remove a license, nor does the EAC handle conversions of several mailboxes at one
time. If you want to script the entire process, including the granting or removal of a license, you need to run some
PowerShell commands. To begin, let's convert a regular mailbox to a shared mailbox and grant an Office 365 license:

[PS] C:\> Set-Mailbox –Identity "Shared@Office365ITPros.com" –Type Shared

[PS] C:\> $License = (Get-MsolUser –UserPrincipalName "Shared@Office365ITPros.com").Licenses[0].AccountSkuID

[PS] C:\> Set-MsolUserLicense –UserPrincipalName "Shared@Office365ITPros.com"

–RemoveLicenses $License

If you want to convert a shared mailbox to a regular mailbox and assign it an Office 365 license, you would use
commands like those shown below. In this instance, one of the enterprise licenses available to the tenant is assigned
to the user account and the usage location for the license is shown as Ireland. Different licenses might be available to
your tenant.

[PS] C:\> Set-Mailbox –Identity "Shared@domain.com" –Type Regular

[PS] C:\> Set-MsolUser –UserPrincipalName Shared@Office365ITPros.com -UsageLocation IE

–Country Ireland

[PS] C:\> Set-MsolUserLicense –UserPrincipalName "Shared@Office365ITPros.com" –AddLicenses "Contoso:EnterprisePack"

Redirecting Email
For historic reasons, Exchange Online supports two methods to forward email from a mailbox. The methods use
different mailbox attributes, but both instruct the transport service to redirect messages to another SMTP address
with the option to also deliver a copy to the original mailbox.

A user can set the ForwardingSmtpAddress through OWA options or an administrator can set email forwarding
for a mailbox through the Office 365 Admin Center. Setting the ForwardingSmtpAddress attribute is the
preferred approach within Office 365. Because the mailbox owner can set up email forwarding through OWA, if
an administrator creates a forward for a mailbox, its existence is known to the mailbox owner.
An administrator can set the ForwardingAddress attribute through EAC (edit the mailbox properties and set the
forwarding address through Mailbox Features > Mail Flow > Delivery Options) or by running the Set-Mailbox
cmdlet. This redirect is invisible to the mailbox owner. As we will see shortly, this approach is useful for
managing the email traffic for shared mailboxes.

A significant difference between the two attributes is that ForwardingSmtpAddress supports any valid SMTP address,
including those belonging to external domains. ForwardingAddress only supports addresses that are known to the
tenant, including mail-enabled contacts pointing to external addresses. It is possible for a mailbox to populate the two
forwarding attributes with different SMTP addresses. When this happens, Exchange Online will forward copies of all
inbound messages to both addresses.

Below is an example of using the Set-Mailbox cmdlet to set a forwarding address for a mailbox. In this instance, the
DeliverToMailboxAndForward property is set to $True to instruct Exchange Online to forward a copy of any inbound
messages to the supplied address and keep a copy in the mailbox.

[PS] C:\> Set-Mailbox -Identity "Andy Ruth" -ForwardingSmtpAddress Andy.Ruth@yandex.com

-DeliverToMailboxAndForward $True

Going forward, Microsoft says that tenants should use the ForwardingSmtpAddress attribute whenever possible.
Administrators can set the ForwardingAddress attribute through PowerShell, but the belief is that transport rules are
a more effective mechanism for administrator-controlled forwarding of email. It is probable that Microsoft will
deprecate the ForwardingAddress attribute at some time in the future and then remove it from Exchange Online.

When you set up email forwarding for a user through the Office 365 Admin Center, the forwarding address is written
into the ForwardingSmtpAddress attribute and any value found in the ForwardingAddress attribute is cleared. Apart
from checking that the input address is formatted properly, no attempt is made to confirm that messages can be
redirected to the email address input as the forwarding destination. The administrator is notified that “the mailbox
owner will be able to view and change these forwarding settings ” (because they will be able to see the redirect
address through OWA Options). If the need exists to hide forwarding, the administrator should use EAC or PowerShell
to set up the redirect through the ForwardingAddress attribute.

Both the Set-Mailbox cmdlet and the Office 365 Admin Center flag warnings if redirect addresses are detected in both
attributes. Set-Mailbox will allow you to write redirect addresses into the two attributes, but the Office 365 Admin
Center will insist on removing the redirect contained in the ForwardingAddress attribute before it will update the
email forwarding settings.
Redirecting Mail Sent to a Shared Mailbox
Companies often use shared mailboxes as the basis for customer communication. In some scenarios, any email
communication with a customer must go from a shared mailbox rather than someone’s personal mailbox. This
functionality is easily achieved by granting users SendAs permission to the shared mailbox. Things are a little more
complicated when you have different teams of customer support agents, each having its own shared mailbox, but you
still want to have all communications recorded in a central mailbox for compliance or other purposes, such as
integration with a CRM system that archives and categorizes customer interactions. You can achieve the goal by using
a couple of mailbox properties.

Our scenario is as follows: our customer services team handles many different products. We want each product to
have its own identity when email links are published on web pages and in other communications, so we create a set of
shared mailboxes, one for each product, like this:

Diapers.
Cosmetics.
Cars.

In addition, we have a central Customer Services shared mailbox.

Taking care of outbound email is simple. The customer service agents simply make sure that they are sending
messages as the Customer Services mailbox and that Outlook is configured to store the message in the shared
mailbox rather than their personal mailbox. The email address of the central mailbox will be stamped into the
outbound message header so that any response from the customer will flow back into the central mailbox.

Inbound mail is redirected by setting two properties on each of the product mailboxes. The forwarding address is set
to the central mailbox while the deliver and forward property is set to $True to instruct Exchange Online to both
forward a copy of the message to the central mailbox and keep a copy in the brand mailbox. The address selected for
the forwarding address must be a known mail-enabled object belonging to the tenant, including other mailboxes,
public folders, mail contacts, shared mailboxes, and distribution lists. You can use an alias, SMTP address, or display
name (if unique) to identify the mail-enabled object to receive the forwarded messages and will see a “couldn’t find
object ” error If you attempt to use a recipient that does not belong to the tenant. Here’s an example of using the Set-
Mailbox cmdlet to set a forwarding address for a mailbox:

[PS] C:\> Set-Mailbox –Identity Cosmetics –ForwardingAddress "Customer Services"

–DeliverToMailboxAndForward $True

A case can be argued not to forward a copy of the message to the brand mailbox (by setting the
DeliverToMailboxAndForward flag to $False ) but this means that someone must process all the new mail arriving into
the central mailbox and forward it on for attention. It is much better to have one copy captured centrally and another
going direct to the people who must deal with the customer. In this scenario, the central copy acts as a customer
contact record while the copy delivered to the "action" mailbox initiates the response.

Sending an acknowledgement from a shared mailbox : It is a common requirement to want to issue an

acknowledgement for messages that arrive into a shared mailbox. For example, when people send email to a
customer services mailbox, it is good to have them receive a response to say that their email will be answered soon
(or whatever is the right text). There is no obvious way to set up an auto-reply message for a shared mailbox, but it is
easily done:

Click your photo in the OWA menu bar.

Select Open another mailbox .
Enter the name of the shared mailbox and click OK.
When OWA opens the shared mailbox, click Options (cogwheel) and select Mail , then Automatic processing
and create the automatic reply.

Another way to connect is to include the email address of the shared mailbox in the URL for OWA. For example:
https://outlook.office.com/owa/bookcomments@office365ITPros.com/. You can then go to Options as before. This
procedure is the best approach if you want to use a text editor to format the content of an auto-reply. Alternatively, if
you are only interested in basic text auto-replies, you can use the Set-MailboxAutoReplyConfiguration to set internal
and external auto-replies for shared mailboxes. See Chapter 6 for more information about the cmdlet. The thing
about auto-replies is that Exchange only sends a single response per correspondent. If you want responses for every
email, you must use Outlook to create an Inbox rule that forces the server to create a reply to every message using a
template.

You can also configure a redirect for a shared mailbox by editing its properties with the Office 365 Admin Center.
Select the mailbox, edit its properties, and select Email Forwarding . You can set the slider to on and enter the
forwarding address. Note that you also have the choice to keep copies of forwarded email in the mailbox. (Figure 7-6
). The same choice exists to configure forwarding for a user mailbox.

When an administrator configures a forwarding address for a user or shared mailbox, the people who access the
mailbox do not know that the forwarding is in place. Because email arrives into the mailbox as normal, forwarding is
especially invisible when the option is selected to deliver messages to both the forwarding address and the mailbox.
For this reason, it is wise to inform people when forwarding is set up on a mailbox unless it is obvious that they should
not be told (for instance, if HR and Legal have advised the IT department that it is necessary to gather copies of all
messages sent to a mailbox).

Figure 7-6: Setting a forwarding address for a mailbox

In general, it is a bad idea to allow users to forward email outside Office 365 as this means that the email is not
subject to compliance policies. It is also the case that some hackers set forwarding on mailboxes to learn about the
way people work and how they interact with co-workers before launching a phishing or impersonation attack. For this
reason, it is wise to check what accounts forward email on a regular basis and understand the business reasons why
the forwards are in place. You can scan user and shared mailboxes to discover whether either type of forwarding is in
place with this command:

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox, SharedMailbox | ? {$_.ForwardingAddress –ne $Null -or
$_.ForwardingSmtpAddress -ne $Null} | Format-Table DisplayName, ForwardingAddress, ForwardingSmtpAddress, DeliverToMailboxAndForward

DisplayName ForwardingAddress ForwardingSmtpAddress DeliverToMailboxAndForward

----------- ----------------- --------------------- ---------------------------

Andy Ruth (Director) Flayosc 18922635 True

Office 365 Feedback TRedmond True

Vasil Michev Vasil2@gmail.com True

Personal Forwarding
As noted earlier, users can also decide to forward email to another SMTP address, which can belong to a recipient
outside the tenant such as a user’s personal Gmail account. The setting is available through OWA. Go to the Options
menu and navigate through Mail, Accounts, and Forwarding (Figure 7-7 ).

As with administrator-initiated forwarding, a user can opt to keep copies of received messages when email is
forwarded. Experience proves that most people don't bother. For this reason and because the forwarding of email
outside the organization makes messages inaccessible to the compliance features of Office 365, you might want to
remove the ability to forward messages from users. Or at least check when forwarding happens to ensure that users
keep copies of email inside the organization. One of the standard alert policies available in the Office 365 Security and
Compliance Center checks for the creation of a forwarding address. You can use this alert policy to send email to
tenant administrators when someone forwards their email and then take whatever action is necessary to check out
what happened. See Chapter 21 for more information on alert policies.
Figure 7-7: Using OWA options to set up a forward to another recipient

You can’t remove the temptation for users to forward email outside the tenant, but you can make the option unusable
by taking three steps to stop users.

1. Use the command shown below to check for users who are forwarding messages and remove any forwards that
are in place. Note that the property checked is ForwardingSmtpAddress , which can redirect to an external
address, rather than ForwardingAddress (which only accepts internal addresses). An alternative is to create a
report and use that information to contact people who use forwarding to request them to stop because it’s
against company policy to forward email outside the tenant. If they continue to forward after the warning some
harsher action might be appropriate.

[PS] C:\> Get-Mailbox –Filter { RecipientTypeDetails –eq 'UserMailbox' –and ForwardingSmtpAddress

–ne $Null } | Set-Mailbox –ForwardingSmtpAddress $Null

2. Remove the ability of users to create Inbox rules to forward messages to external domains. When forwarding is
disabled for a domain, Exchange Online detects that this is the case and delivers new email to the original
mailbox. You can block forwarding for all external domains as follows:

[PS] C:\> Get-RemoteDomain | Set-RemoteDomain –AutoForwardEnabled $False

Because it is possible that good business reasons exist to warrant forwarding to certain
external domains, a more granular approach can be taken to disable the ability for users
to forward messages to selective external domains. This approach is often used to stop
users forwarding messages to consumer email services. For instance, let’s assume that
you don’t want users to be able to forward messages to Yahoo.com accounts. You need to
define Yahoo.com as a new external domain and then block forwarding to that domain.
Here’s how.

[PS] C:\> New-RemoteDomain -DomainName "*.yahoo.com" -Name "Yahoo.com"

[PS] C:\> Set-RemoteDomain -Identity "Yahoo.com" -AutoForwardEnabled $False

3. Remove the ability of users to set forwarding through OWA options. This approach needs you to update the
default user role assignment policy because the options are controlled by the policy. If you'd like the challenge of
figuring out how to make the necessary changes, much the same technique is used later to allow users to edit but
not create distribution lists. However, you can also find a complete description of the required steps online.

Transport rules can also be used to block forwarded messages from reaching their destination (here’s an example how
to implement such a rule ). Given that transport rules execute in strict priority order, any use of transport rules to
block messages creates the potential that the processing done by one rule might interfere with other transport rules,
especially when Office 365 generates transport rules automatically for features such as Data Loss Prevention (DLP) or
Supervisory Review policies. Tenants use transport rules for different purposes, so some testing is necessary to figure
out the right solution to block email forwarding.

Naturally, as in the case of any action that removes a facility that some people might be using, it’s a good idea to
capture the business reasons why a policy exists for email forwarding and to communicate why blocks are being
enforced and how the changes might affect users before anything is done.

More forwarding options : In addition to the ability of users to set up forwarding for their mailbox via OWA options
and the ability of an administrator to create a forwarding address for a user’s mailbox, Exchange Online supports
three more methods to forward email. First, a user can create a rule to forward a message after it arrives. Second, an
administrator can create an Inbox Rule with the New-InboxRule cmdlet (Chapter 6). Third, a transport rule can
redirect messages sent to a mailbox.

Using Mobile Devices with Shared Mailboxes

Shared mailboxes are a very convenient method for teams to have access to information on which they need to work
together. Given the highly mobile nature of today’s workforce, it is natural to assume that you will be able to use
shared mailboxes on smartphones or tablets, but this isn’t really the case. The problem is that Microsoft designed the
shared mailbox access mechanism for Outlook desktop: Autodiscover detects the shared mailboxes that you have
access to and loads these mailboxes into its resource list. Exchange ActiveSync (EAS) is the most common protocol
used by mobile devices but EAS can only open personal mailboxes and does not accommodate shared mailbox access.
Another issue is that Microsoft does not support mobile connections to Office 365 shared mailboxes as no licenses
exist to cover this functionality.

It is possible to use the IMAP protocol to open a shared mailbox from a mobile client. IMAP is a now-venerable mail
retrieval protocols that knows nothing about shared mailboxes. The trick is to use the credentials of a user who has
full access to a shared mailbox together with the mailbox’s alias to logon to the shared mailbox. For example,
specifying the username Kim.Akers@Office365ITPros.com\Accounts means that you want to use the credentials
belonging to Kim Akers to access the shared mailbox pointed to by the alias Accounts. To access a shared mailbox
with Outlook for iOS, follow the normal steps to add a new IMAP account and select Advanced Settings. Input the
following:

Name : Give the shared mailbox a name to show its purpose.

Email : Enter the primary SMTP address of the shared mailbox.
IMAP host : Enter outlook.office365.com
IMAP username : Enter the User Principal Name of an account that has full access to the shared mailbox and
the alias of the shared mailbox. Do not use an account protected by multi-factor authentication because IMAP
does not support this feature. For example:

Kim.Akers@office365itpros.com\SharedMailboxAlias

IMAP password : Enter the password to the account you specified in the IMAP username.
SMTP host : Enter smtp.office365.com:587
SMTP username: Enter the User Principal Name of the account that has full access to the shared mailbox.
SMTP password: Enter the password to the account you specified in the SMTP username.

While not recommended, you can use different accounts for IMAP and SMTP. However, before you rush to deploy this
method, consider these issues:

Because Microsoft does not support this mechanism for access to shared mailboxes, they might remove or block
access at any time.
The technique works with IMAP clients running on iOS and Android. My experience is that it is easy to download
the Inbox from a shared mailbox to a mobile device but much harder to get the device to send mail. The trick
here is to tell the device to use SMTP on port 587.

If you want a safe way to access a shared mailbox on a mobile device, it’s better to use a browser and include the User
Principal Name of the shared mailbox in the URL passed to OWA as described earlier. This approach will work
because all the browser needs to do is display the data provided by Office 365.

Because the protocols used by mobile devices do not support non-personal mailboxes, the same problem exists with
other forms of non-personal mailboxes such as resource mailboxes. The exception is the group mailboxes used by
Office 365 Groups, which have their own app. That app uses the Office 365 REST API rather than EAS, IMAP4, or
POP3, which is why it works.
Recipient Moderation
Moderation (or “message approval”) means that a message must first be reviewed and approved by a nominated
moderator before Exchange Online can deliver it to the addressee. Up to ten moderators can be assigned
responsibility to control messages sent to the following recipient types:

User Mailboxes.
Shared Mailboxes.
Distribution Lists.
Dynamic Distribution Lists.
Mail Contacts.
Mail Users.
Mail-enabled Public Folders.

Moderation is not supported for group mailboxes. Moderation for individual recipients (mailboxes, mail contacts, or
mail users) is intended to “protect” the recipients against inappropriate email, usually because the recipients are
sensitive in some way. For example, it’s reasonably common to impose moderation on the mailboxes of senior
executives so that an executive assistant or other moderator can review inbound email before passing it to the
executive. Moderation for distribution lists is often imposed to ensure that posts to very large groups only appear
when it is appropriate to share the content with the group. In other words, you might not want to have everyone in the
company be allowed to send messages to the 50,000-recipient “All Company” group. In all cases, you can arrange for
select users to bypass the requirement for moderation so that their messages are delivered to the recipient without
going through the approval process.

A moderated recipient can have multiple moderators, in which case all the nominated moderators receive copies of
messages for approval (Figure 7-8 ). However, no quorum must be available before a message can be delivered as a
simple approval from a single moderator suffices. If a message is declined by a moderator, Exchange Online returns it
to the original sender. Logically, a moderator can send a message to the recipient that they are moderating without
gaining further approval as can the owner of a distribution list.

A message awaiting moderation is held in an arbitration mailbox (you have no control over which arbitration mailbox
is used) and should be processed within two days. However, the process that expires messages waiting in the
arbitration mailbox runs on a weekly workcycle, which means that senders will receive notification that moderation
has not occurred any time between two and nine days after the original message is sent – a tenant administrator can
do nothing to accelerate the approval process. It is also interesting to note that Exchange Online throttles the number
of expired moderation notifications to 300 per hour , which means that in periods when heavy usage is made of
message moderation, some senders might not receive notifications that their messages have expired while awaiting
moderation.
Figure 7-8: A message arrives for moderation

To set up moderation for a group through EAC, edit the object’s properties and go to the “Message approval” tab
(Figure 7-9 ). Here you can set the flag to enable moderation and input the names of the users to act as moderators.
Notice that you also can input the names of groups or users who will bypass moderation and select the kind of
notification sent to users when their messages await moderation.
Figure 7-9: Enabling moderation for a distribution list

The EAC does not display the message approval UI for mailboxes, contacts, mail users, or public folders, so you must
set up moderation for these objects using PowerShell. For example, this command enables moderation for a mailbox
and sets up three moderators. We also select a group that can bypass moderation.

[PS] C:\> Set-Mailbox –Identity "CEO Mailbox" –ModerationEnabled $True

–ModeratedBy "Steve Smith", "Bill Jones", "CEO Assistant"

–BypassModerationFromSendersOrMembers "Executive Committee"

Similar properties to enable moderation can be set through the Set-DistributionGroup , Set-DynamicDistributionGroup
, Set-MailContact , and Set-MailUser cmdlets. When an object is enabled for moderation, Exchange Online creates an
automatic MailTip that is displayed to users to inform them that a message will be moderated and might be delayed
when they address a message to the object.

Moderation for nested groups : Distribution lists (but not dynamic distribution lists) support the
BypassNestedModerationEnabled parameter. If set to $True , any nested groups that also need moderation are
governed by the decision of the moderator of the group to which a message is addressed. If a moderator approves a
message, it will be delivered to the members of all the nested groups too. The default value of the flag is $False ,
meaning that delivery to each moderated group needs approval, but in most cases it’s reasonable to set the flag to
$True and allow the first moderator to control approval.

Mail Contacts and Mail Users

Exchange Online supports both mail contacts and mail users and includes the two object types in the GAL so that they
can be addressed by users. The difference between the two is:

A mail contact is a pointer to a user of an external email system.

A mail user has an external email address but also has Office 365 credentials and is therefore able to logon to
Office 365 to access resources such as SharePoint Online or OneDrive for Business sites or Skype for Business
 Online.

Remember that in a hybrid environment, mail contacts and mail users are created on-premises and then synchronized
to Exchange Online.

Mail contacts are usually created to provide users with a GAL (and OAB) entry that points to a known, valid email
address. They are often used to include entries for external business partners, such as a Public Relations agency, in
the Global Address List. You can create a mail contact with the EAC by completing six fields:

First Name: Optional.

Last Name: Optional.
Initials: Optional.
Display Name: The name that appears in the GAL. It’s a good idea to mark the object as being external to the
company and to include a visual blue about which organization the contact belongs.
Alias: The alias must be unique, and it can’t have any spaces.
External email address: An SMTP address that is external to the Office 365 tenant.

The New-MailContact and Set-MailContact cmdlets are also available and allow a greater range of control over the
properties of mail contacts. In this example we first create a new mail contact, setting their preferred mail format to
be HTML, and then run the Set-MailContact cmdlet to enforce moderation and to inform senders that any message
sent to this address will be moderated.

[PS] C:\> New-MailContact –ExternalEmailAddress "Danny.Flowers@contoso.com" –LastName "Flowers"

-DisplayName "Danny Flowers (Contoso)" –FirstName "Danny" –Name "Danny Flowers"

–MessageBodyFormat HTML –Alias Dflowers

[PS] C:\> Set-MailContact –Identity DFlowers –ModeratedBy "Kim Akers" –ModerationEnabled $True

–MailTip "Message will be moderated before dispatch"

Some of the mail contact settings supported by Exchange on-premises, such as the ability to set the maximum
message size, are unavailable in Exchange Online.

Although you can create the new mail user object by running the New-MailUser cmdlet, the easiest way to create a
mail user is with EAC. As shown in Figure 7-10 , you must give a valid SMTP email address to point to the user’s
mailbox on an external email system.

Behind the scenes, Office 365 creates a user object in Azure Active Directory that will show up in the Office 365
Admin Center. However, the new object needs to be assigned an Office 365 license before they will be able to use any
of the Office 365 services.

Mail user objects can be manipulated with the Set-MailUser cmdlet. For example:

[PS] C:\> Set-MailUser –Identity "David Pelton" –CustomAttribute10 "External Recipient"

Despite the fact that mail users are also Office 365 user objects, when you delete a mail user from the EAC or by
running the Remove-MailUser cmdlet, the object is permanently removed from Office 365 and cannot be recovered.
Figure 7-10: Creating a new mail user

Distribution Lists
Distribution lists or distribution groups have existed in email systems since the early days of the technology. A
distribution list is a collection of known email recipients which provide users with the ability to address them as a
single recipient. The membership of a distribution list is composed of mail-enabled recipients registered in EXODS
and accessible through the GAL:

User Mailboxes.
Site Mailboxes (now deprecated).
Mail-enabled public folders.
Other distribution lists.
Dynamic distribution lists.
Mail contacts.
Mail users.

The Exchange Online transport system takes care of expanding the membership of the group into the individual
recipients and figuring out how best to route a copy of the message to each recipient. Normally, a distribution list is a
universal distribution group (UDG), in which case the sole purpose of the group is to act as an address for email.
Alternatively, a distribution list can be a universal security group (USG), which means that you can use the group to
send mail and to control access to resources, such as access to a shared mailbox. The “universal” part of the name is a
legacy of on-premises Active Directory where it means that the group is available throughout a forest. This does not
have much meaning within an Office 365 tenant, but the name persists. You cannot change a UDG to make it a USG or
vice versa.

Technically, you can add an Office 365 Group to the membership of a distribution list. You cannot do this through the
Office 365 Admin Center or the EAC because the “pickers” used in these interfaces do not show Office 365 Groups.
However, you can run the Add-DistributionGroupMember cmdlet to add an Office 365 Group. Although Microsoft does
not say that this action is unsupported, it is hard to recommend. In most cases, you should keep distribution lists apart
from Office 365 Groups until the time comes to convert them.

Microsoft has a very clear strategy to encourage tenants to move from distribution lists to Office 365 Groups as
quickly as possible. Microsoft’s view is that Office 365 Groups are more functional and productive than distribution
lists will ever be and that users will gain more benefit if the tenant upgrades older distribution lists into modern Office
365 groups. It is for this reason that Microsoft offers multiple methods to convert distribution lists. Notwithstanding
the undoubted added functionality available through Office 365 Groups, there are still reasons to continue using
distribution lists, including:
 Ability to include different mail-enabled recipient types in group membership.
No licensing requirement to use dynamic distribution lists.
Easier interoperability with other third-party email systems.
Ability to nest distribution lists.

Microsoft is aware of where Office 365 Groups are less functional than distribution lists and is working to close the
gap, so this is an area to keep an eye on as time progresses. See Chapter 13 for a more extensive discussion about
how to convert distribution lists to Office 365 Groups.

Creating a New Distribution List

Figure 7-11: The UI used in EAC to create a new distribution list

The preference shown in the EAC is to create a new Office 365 Group. If you want to create a distribution list, you
must explicitly select to create a distribution list, a security group, or a dynamic distribution list. The group type tells
EAC which UI to display to gather information about the properties of the new group. Even when you opt to create a
distribution list, the interface displayed (Figure 7-11 ) includes text to convince you to create a new Office 365 Group
instead. Because of the importance assigned by Microsoft to Office 365 Groups and the significant role they occupy
within the service, it is sensible to create these groups rather than traditional distribution lists if you can.

Between Figure 7-11 and Figure 7-12 , we see the fields commonly populated to create a new distribution list. The
fields that we need to pay attention to are:

Display Name : This is the name that shows up in the GAL and you should give some thought should to its
composition. Ideally, the name should convey the reason why the group exists, how to use it, and what its
membership might be. Groups created by administrators are not subject to the organizational group naming
policy as this only applies to user-created groups. We will discuss how to create a group naming policy and how
to restrict users from being able to create groups later.
Owners : A distribution list must have at least one owner. This is more important with a USG because only
owners can add or remove members from a group. You can choose to have owners added to the group
membership so that they receive messages sent to the group, but this is not necessary. The ownership of a group
can include other groups as well as individual users.
Membership : Each member appears separately. You can add mailboxes, mail contacts, mail users, shared
mailboxes, public folders mail contacts, and other distribution lists. You cannot add an Office 365 Group to the
membership of a distribution list through this UI, but you can do this with PowerShell.
Joining Approval : You can control how members join a UDG by setting its property to be Open, Closed, or
Owner Approval. Any one of the group owners can approve a request to join a UDG.
Leaving Approval : Similar control is available over how members can leave a UDG. Open allows members to
leave at will; Closed means that a group owner must remove users from the membership.

Users can browse available groups that they belong to or which they own through the Distribution groups section of
OWA options. Options are also available to join a group and to leave a group. Joining a group involves a user first
searching the directory to find the group and then creating a request to join it. Exchange automatically approves
requests to join open groups and rejects them for closed groups. Otherwise Exchange sends requests to join these
groups to the group owner(s) for approval.

The MemberJoinRestriction and MemberDepartRestriction properties control how users can join or leave UDGs. You
can update a UDG with the Set-DistributionGroup cmdlet. This example sets the group properties so that an owner
must give approval to join but a user can leave without approval.

[PS] C:\> Set-DistributionGroup –Identity ‘All Company’ –MemberJoinRestriction ‘ApprovalRequired’

–MemberDepartRestriction ‘Open’

The options to control user-initiated joining and leaving of a group do not exist for either USGs or dynamic
distribution lists. USGs are security principals that can grant access to other resources. It therefore makes sense that
only the group owners should be able to control who enjoys that access. Exchange Online calculates the membership
of dynamic distribution lists by running a query against Azure Active Directory. An administrator can decide whether
an object should be found by that query by adjusting the object’s properties, but unless the owner is also an
administrator, they cannot influence the query results.

Figure 7-12: Creating a new distribution list

To simplify the creation of distribution lists, the EAC UI only reveals the essential properties of a group when you
create a new group. You can then complete several other interesting properties using PowerShell to update the group
or by editing the group through EAC. These properties include:

RequireSenderAuthenticationEnabled : The default is $True , meaning that Exchange Online will only allow
messages to be delivered to the group if the sender can be authenticated. In other words, the sender belongs to
the tenant. Messages generated by senders who do not belong to the tenant will be rejected unless this property
is set to $False.
RejectMessagesFrom, RejectMessagesFromDLMembers, RejectMessagesFromSendersOrMembers AND
AcceptMessagesOnlyFrom, AcceptMessagesOnlyFromDLMembers,
AcceptMessagesOnlyFromSendersOrMembers: Exchange Online allows for reasonably granular control over
who can send to a distribution list. You can block specific users or members of other distribution lists with the
Reject* properties or block everyone except those specified in the Accept* properties. The EAC populates the
Accept* properties when you edit a distribution list to update its delivery management parameters. All the
 objects specified in these lists must be known to Exchange Online and can include mail contacts and mail users.
If you try to add a user via their SMTP address, Exchange Online checks the address against Azure Active
Directory and rejects it if the address does not match a mail-enabled object.
When updating a reject or accept list for a group, you must update the relevant *SendersOrMembers property
and Exchange Online will sort out the individual object types and update the properties accordingly. For instance,
here is how to update a group so that it will only accept messages from two mailboxes and the members of
another group:

[PS] C:\> Set-DistributionGroup –Identity "Executive Committee"

–AcceptMessagesOnlyFromSendersOrMembers "Ben Owens", "CEO Mailbox", "Vice Presidents"

GrantSendOnBehalfTo: Controls who has the right to send messages on behalf of the distribution list.
You can also assign the SendAs right to a user for a distribution list. However, you can do this with the Add-
RecipientPermission cmdlet instead of Set-DistributionGroup . For example, here is how to assign the SendAs
permission for the Executive Committee distribution list to the CEO Mailbox user.

[PS] C:\> Add-RecipientPermission -Identity "Executive Committee" -Trustee "CEO Mailbox"

-AccessRights SendAs

Note : To protect resources, Exchange Online enforces limits on the number of recipients that can exist for an
individual message (500) and the total number of recipients that a mailbox can send messages to in a single day
(10,000). These specific limits exist to prevent tenants using Exchange Online from as a bulk email service. When
Exchange Online checks messages against these limits, distribution lists and dynamic distribution lists count as a
single recipient, even if the use of these groups means that a message addresses many more recipients than allowed
by the limit after the transport service expands groups into individual addressees.

Updating Group Membership with PowerShell

Exchange Online holds the membership of distribution lists as backward links to the EXODS objects for the mail-
enabled recipients that make up the membership of groups. EXODS is used instead of Azure Active Directory because
mail-enabled public folders and mail contacts, which can be members of distribution lists, do not exist as Azure Active
Directory objects. No trace of membership is visible if you examine a group’s properties using PowerShell. Instead,
you must use the EAC to view the properties of a group or run the Get-DistributionGroupMember cmdlet to extract
and display its membership. Here is an example:

[PS] C:\> Get-DistributionGroupMember –Identity "Exchange Connections Speakers"

Name RecipientType

---- -------------

TRedmond UserMailbox

Paul Robichaux MailContact

Jeff Guillet (MVP) MailContact

Michel de Rooij (MVP) MailContact

Paul Cunningham (MVP) MailContact

Nathan O'Bryan (MVP) MailContact

So much for reporting. The interesting thing is when you want to update group membership. We can use the Update-
DistributionGroupMember cmdlet to update the complete group membership at one time, overwriting anything that
might already exist. As shown below, the input to the cmdlet is a comma-separated list of aliases for the group
members.

[PS] C:\> Update-DistributionGroupMember -Identity "Exchange Connections Speakers"

-Members @("TRedmond", "Bowens")

Alternatively, you can use the Add-DistributionGroupMember cmdlet to add a single object to a group or Remove-
DistributionGroupMember to remove a member. For example:

[PS] C:\> Add-DistributionGroupMember –Identity "Exchange Connections Speakers"

–Member VanHybrid

[PS] C:\> Remove-DistributionGroupMember –Identity "Exchange Connections Speakers"

–Member VanHybrid –Confirm:$False

Users who belong to cloud-only tenants can edit the membership of the distribution lists that they own using Outlook
or OWA. However, distribution list owners should select the online Global Address List rather than the Offline Address
Book (OAB) as this will ensure that up-to-date group membership is available and that updates occur quickly. You can
still update group membership through the OAB, but everything seems to happen much slower.

Things are much trickier in a hybrid environment where those with Office 365 mailboxes cannot manage on-premises
groups, even if they are a registered owner. The same is true for those who have on-premises mailboxes as they
cannot edit distribution lists created in the cloud. Permissions and the ability of Outlook to support cross-platform
editing of the membership of distribution lists are the main reasons. In hybrid environments, you are better off using
Outlook to edit distribution l membership unless you are positively sure that the mailbox and group are on the same
platform.

Using Office 365 Groups with Distribution Lists

The EAC UI does not allow you to include an Office 365 Group in the membership of a distribution list. This is largely
because the EAC UI comes from the on-premises version of Exchange where Office 365 Groups do not exist. However,
with a little PowerShell, you can sometimes overcome the restrictions of the UI. For example, you can an Office 365
Group to the membership of a distribution list, you can do this with PowerShell. For example:

[PS] C:\> Add-DistributionGroupMember -Identity GDPRWorkingGroup -Member MyOffice365Group

Running Get-DistributionGroupMember afterwards will show that the Office 365 Group appears in the member list as
a mail universal distribution list. Exchange will deliver sent to the distribution list to the Office 365 Group as normal.
You can remove the group with Remove-DistributionGroupMember .

[PS] C:\> Remove-DistributionGroupMember -Identity GDPRWorkingGroup -Member MyOffice365Group

-Confirm:$False

You can also update a distribution list so that it will only accept messages from members of an Office 365 Group:

[PS] C:\> Set-DistributionGroup -Identity GDPRWorkingGroup -AcceptMessagesOnlyFromSendersOrMembers @{add="MyOffice365Group"}

You can even grant the send on behalf of permission to members of an Office 365 Group to send messages from the
distribution list:

[PS] C:\> Set-DistributionGroup -Identity GDPRWorkingGroup -GrantSendOnBehalfOfTo MyOffice365Group

Including a Teams channel in a Distribution List

As discussed in Chapter 13, the email integration for Teams creates email addresses that can be used to send email to
a channel in a team. Messages sent to a channel appear as new conversations in the channel and are also captured in
the SharePoint site belonging to the team. To add a team channel email address to a distribution list, create a new
mail contact with the address and add the mail contact to the distribution list.

Including a Guest Account in a Distribution List

Guest accounts are created to share resources in Office 365 Groups, Teams, Planner, and SharePoint. Exchange
Online represents guest accounts as mail users, but the mail user objects for guest accounts don’t show up in address
lists, so you can’t add them to a distribution list through EAC or a client UI. Instead, you can add them to a
distribution list via PowerShell. This code adds the mail user object for the guest account JohnSmith@outlook.com to a
distribution list.

[PS] C:\> Add-DistributionGroupMember -Identity MyDL -Member JohnSmith_outlook.com#EXT#

Discovering the groups someone belongs to

You can discover the groups to which a user belongs by examining their mailbox properties through EAC. Or you can
run some PowerShell! The code to do the trick is:

[PS] C:\> ForEach ($Group in Get-DistributionGroup) { ForEach ( $Members in Get-DistributionGroupMember -Identity $Group.Alias|
Where-Object { $_.PrimarySmtpAddress -eq "Kim.Akers@Office365ITPros.com"}) { $Group.DisplayName}}

We can make the code quicker by using the ability of Exchange Online to execute server-side filters. Here is how:

[PS] C:\> $Dn = (Get-Mailbox –Identity Kim.Akers).DistinguishedName

[PS] C:\> Get-DistributionGroup –Filter "Members –eq '$Dn'"

If you want, you can replace Get-DistributionGroup with the Get-Recipient cmdlet to report the different types of
group to which someone belongs.

[PS] C:\> Get-Recipient -Filter "Members -eq '$Dn'" -RecipientTypeDetails GroupMailbox, MailUniversalDistributionGroup,
MailUniversalSecurityGroup, DynamicDistributionGroup | Format-Table DisplayName, RecipientType

You can use a variation of this command to focus on membership of a specific group type. For instance, this command
creates a list of Office 365 Groups to which the user belongs.

[PS] C:\> Get-Recipient -Filter "Members -eq '$Dn'" -RecipientTypeDetails GroupMailbox | Format-Table DisplayName

Dynamic Distribution Lists

Traditional email distribution lists have functioned very well for years. However, some well-known deficiencies exist.
An ongoing issue that persists is the maintenance of group membership. Exchange 2010 introduced the ability for
users to manage the groups that they own and for other users to join and leave groups as they wish, but all these
actions need some form of user intervention. Time being precious, the need to update group membership is unlikely to
have a high priority for most.

Exchange 2003 introduced the concept of query-based distribution lists and this concept exists today in the form of
dynamic distribution lists. The idea behind both is the same. Any time that the server needs to determine the
membership of a group to route messages, the transport system executes a query against the directory to identify
recipients that match the query criteria such as “all the users in the London office” or “everyone who works for the
Accounting department” or “all full-time employees”. Exchange Online defines the queries (also known as a recipient
filter) in OPATH format instead of LDAP, the reason being that OPATH is the query format used by PowerShell. In
addition, Azure Active Directory does not support LDAP!

Any group that depends on directory queries for their membership have a weak point in that the success of the
queries depends on the quality of the directory. If the objects held in Azure Active Directory do not hold the values
used by the queries, dynamic distribution lists will address messages to incomplete or erroneous sets of recipients. On
the other hand, if you make sure to populate and maintain Azure Active Directory with the information required to
support effective queries, dynamic distribution lists work very well.

Once you know the query to find the desired recipient set, creating a new dynamic distribution list is not particularly
difficult. The EAC has a reasonable interface to allow administrators to compose the query used to figure out group
membership. Each condition that you add to the query is a rule that checks an attribute of the objects that fall under
the scope of the query (all recipient types, user mailboxes, and so on) against one or more text values. In Figure 7-13
the word "Ireland" appears in a rule to filter users from the GAL. The filter can find specific objects based on the
presence of the words or phrases supplied here in standard attributes such as “Department” or the custom attributes
available for organization-specific data. You can enter anything you like here, including values that do not exist in any
Azure Active Directory object.

Figure 7-13: Specifying a rule for the query used by a dynamic distribution list

The EAC has no preview to check the effectiveness of the query that you establish when you create or edit a new
dynamic distribution list. You can view the filter for a dynamic distribution list with the Get-DynamicDistributionGroup
cmdlet. For instance, this command reveals that the filter looks for user mailboxes that have “Exchange” in
CustomAttribute1 and excludes system, discovery, and arbitration mailboxes

[PS] C:\> Get-DynamicDistributionGroup –Identity "Office 365 Gurus" |

Format-List RecipientFilter, RecipientFilterType

RecipientFilter : ((((RecipientType -eq 'UserMailbox') -and (CustomAttribute1

-like 'Exchange'))) -and (-not(Name -like 'SystemMailbox{*')) -and (- not(Name -like 'CAS_{*'))

-and ( -not(RecipientTypeDetailsValue -eq 'MailboxPlan')) -and

(-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) -and

(-not(RecipientTypeDetailsValue -eq 'ArbitrationMailbox')))

RecipientFilterType: Custom

The “Custom” value in RecipientFilterType tells us that Exchange Online generated the OPATH query for the group
because an administrator ran the New-DynamicDistributionGroup or Set-DynamicDistributionGroup cmdlets to create
a complex recipient filter. Dynamic groups that use recipient filters based on the pre-prepared queries (like “all mail-
enabled recipients”) presented in the EAC GUI run “precanned” rather than “custom” queries to calculate their
membership. Only groups with precanned filters can have their recipient filters updated through the EAC.

It is good to know that a dynamic group has a recipient filter and what that filter is, but we do not know whether the
filter will return any useful results when someone uses the group to address messages. The easiest way to test a
recipient filter is to input it to the Get-Recipient cmdlet. Here is how we would input the query contained in the
“Exchange Gurus” group:

[PS] C:\> Get-Recipient –RecipientPreviewFilter (Get-DynamicDistributionGroup

–Identity “Office 365 Gurus").RecipientFilter

Name RecipientType

---- -------------

Jeff.Guillet UserMailbox

If we only expect the query to return a single mailbox, then we know that the query works. If not, the fault is either in
the query (it searches for the wrong data) or the underlying data (does not contain the right values). Testing the
results of a query is important not only because the results will reveal any flaw in your logic and identify whether the
filter returns the correct recipient set when the query executes against the directory. It will also tell you how many
recipients are in the selected set. Equipped with that knowledge, we can construct some guidelines to help
administrators maintain dynamic distribution lists. For example, your organization might use the following guidelines
for both standard and dynamic distribution lists:

Any group that has more than 200 recipients must have a MailTip to highlight the fact to users. Exchange Online
automatically displays a MailTip to tell the sender how many people will receive their email, but you can add
another MailTip to urge additional caution when sending to very large groups.
Any group that has more than 500 recipients must be moderated to ensure that people do not inadvertently
broadcast to groups such as “The Entire Company”. Moderated groups are also a good way to stop email storms
that result when users reply-all to very large groups.

Adjusting Dynamic Filters

The recipient filter shown in the example above comes from a dynamic distribution list created in 2014. At that time,
the set of system mailboxes (like arbitration mailboxes) used by Exchange Online was smaller than it is today, so the
conditions in the recipient filter to exclude system mailboxes are less complex than the filter generated for new
dynamic groups. These conditions ensure that Exchange does not try to deliver messages sent to the group to system
mailboxes If you compare a new filter, you see that it covers more types of system mailboxes in the queries that
Exchange Online generates automatically for new groups. No one expects you to type in this query as it is much easier
to let Exchange generate the code for you, so including it here is just to show the kind of complicated query that’s
used:

-and (Alias -ne $null))) -and (-not(Name -like 'SystemMailbox{*')) -and (-not(Name -like 'CAS_{*')) -and (-
not(RecipientTypeDetailsValue -eq MailboxPlan')) -and (-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) -and (-
not(RecipientTypeDetailsValue -eq 'PublicFolderMailbox')) -and (-not(RecipientTypeDetailsValue -eq 'ArbitrationMailbox')) -and (-
not(RecipientTypeDetailsValue -eq 'AuditLogMailbox')) -and (-not(RecipientTypeDetailsValue -eq 'AuxAuditLogMailbox')) -and (-
not(RecipientTypeDetailsValue -eq 'SupervisoryReviewPolicyMailbox')))
Exchange Online does not update filters in dynamic distribution lists retrospectively. The old filters continue to work,
even if there is a slight chance that a copy of a message might find its way to a newer form of system mailbox, like the
ones used for supervisory review policies. You can update the recipient filters for older dynamic distribution lists if
you want to, but there is really no need to do this, unless a specific need arises.

Once such need is when you allow external guests to access Office 365 Groups. Office 365 creates guest accounts in
Azure Active Directory allow external people to join Office 365 Groups. Because they have email addresses, Exchange
can route email to the guest accounts. Therefore, because the default recipient filter for a dynamic distribution list
includes these objects, they will receive copies of all messages sent to the group. EAC does not include the necessary
filter to exclude guests, so if you want to ensure that these addresses do not receive copies of messages, you must
amend the filter to include an extra filter to exclude guests, like this:

and (-not(RecipientTypeDetailsValue -eq ‘GuestMailUser’))

To do this for a group, first capture the existing filter into a text file.

[PS] C:\> (Get-DynamicDistributionGroup -Identity "Office 365 Gurus").RecipientFilter > Filter.txt

Now edit the filter in the text file using NotePad or a similar editor and add the clause to the end of the filter. Next,
copy the new recipient filter to the clipboard and then paste it into PowerShell as the value passed to the Set-
DynamicDistributionGroup cmdlet. Make sure to enclose the recipient filter in curly braces. You should end up with a
command like that shown below:

[PS] C:\> Set-DynamicDistributionGroup -Identity "Office 365 Gurus" -RecipientFilter { ((((((Department -eq 'Flayosc') -or
(Department -eq 'MVP'))) -and (((StateOrProvince -eq 'Var') -or (StateOrProvince -eq 'Dublin') -or (StateOrProvince -eq 'United
States'))) -and (Alias -ne $null))) -and (-not(Name -like 'SystemMailbox{*')) -and (-not(Name -like 'CAS_{*')) -and

(-not(RecipientTypeDetailsValue -eq 'MailboxPlan')) -and (-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) -and (-

not(RecipientTypeDetailsValue -eq 'PublicFolderMailbox')) -and

(-not(RecipientTypeDetailsValue -eq 'ArbitrationMailbox')) -and (-not(RecipientTypeDetailsValue -eq 'AuditLogMailbox')) -and (-

not(RecipientTypeDetailsValue -eq 'AuxAuditLogMailbox')) -and

(-not(RecipientTypeDetailsValue -eq 'SupervisoryReviewPolicyMailbox')) -and

(-not(RecipientTypeDetailsValue -eq 'GuestMailUser'))) }

To check that the new filter works as expected, send a message to the group and check that those you expect to
receive the message do so while guests do not. Unfortunately, the technique explained above of using the Get-
Recipient cmdlet to check the set of objects returned by a recipient filter does not work with guest accounts.

Remember that once you use PowerShell to update a recipient filter for a dynamic distribution list, you must update
the filter with PowerShell thereafter. This makes it slightly more complicated to change a recipient filter, but it is
usually possible to arrive at the right query through some trial and error.

Room Lists
Figure 7-14: Using a room list to schedule a meeting with Outlook 2016

A room list is a special form of distribution list composed exclusively of room mailboxes. No other type of recipient
(including resource or equipment mailboxes) can be a member of a room list. The idea behind room lists is that they
are a convenient way to segregate the different conference or meeting rooms available within an organization so that
you have a room list per building or location. The room lists can then be used to select the best location when
scheduling a meeting from Outlook or OWA. This is probably not a feature that is of much interest to small companies,
but it can be valuable in large campus scenarios (like Microsoft’s own Redmond HQ) where many multi-floor buildings
can host meetings. Figure 7-14 shows how users interact with room lists when scheduling a meeting. In this case, the
user selects the “HQ Rooms” list to reveal that it includes four rooms. The user can then select one of these rooms to
add it to the meeting.

You cannot manage room lists through the EAC (the EAC does not even include room lists when it lists available
distribution lists). Instead, you must manage room lists with PowerShell. Here is an example of how to create a new
“HQ Rooms” list. Note the use of the IgnoreNamingPolicy parameter to override the group naming policy in force for
the organization.

[PS] C:\> New-DistributionGroup –Name "HQ Rooms" –Members "Room 101", "Room 102", "Room 103"

-RoomList -IgnoreNamingPolicy

You can use this command to discover the room lists that already exist within a tenant:

[PS] C:\> Get-DistributionGroup –RecipientTypeDetails RoomList

Editing the membership of a room happens as described before using the Update-DistributionGroupMember , Add-
DistributionGroupMember , and Remove-DistributionGroupMember cmdlets. You will see an error if you try to add a
recipient that is not a room mailbox to a room list. The Remove-DistributionGroup cmdlet removes a room list.

[PS] C:\> Remove-DistributionGroup –Identity "Old HQ Rooms"

Reporting Distribution List Membership

You can use the EAC, the People app, or Outlook to view the membership of a distribution list, but neither interface
does a good job with groups of more than 20 or so recipients. The best experience is through the People interface in
OWA, which makes it easy to scroll through large groups. However, none of the interfaces provided by Microsoft allow
you to print off the membership of a group, so unless you invest in a third-party reporting product (see the discussion
about Office 365 reports in Chapter 21), you must create your own reports using PowerShell. For example, this one-
liner extracts all the members of the Executive Committee group and outputs some of their details into a CSV file.

[PS] C:\> Get-DistributionGroupMember –Identity "Executive Committee" |

Select-Object Name, DisplayName | Export-CSV "c:\temp\Execs.csv" -NoTypeInformation

It is a little more difficult to report the membership of a dynamic distribution list because the membership is unknown
until you run the query against Azure Active Directory. However, we can access the information easily using another
PowerShell one-liner. In this example, we retrieve the recipient filter from the group, evaluate the filter with the Get-
Recipient cmdlet, and use the output to create a CSV file.

[PS] C:\> Get-Recipient –RecipientPreviewFilter (Get-Dynamic-DistributionGroup

–Identity "Office 365 Gurus").RecipientFilter | Select-Object Name, DisplayName, Department, City |

Export-CSV C:\Temp\O365Gurus.CSV -NoTypeInformation

As this command returns different types of object, some of them are likely to not have all of the properties populated
with the same information.

Another common request is to know what distribution list are inactive and are therefore a candidate for removal from
the directory. Exchange Online does not provide an option to find inactive groups. One way to approach the problem is
to search the message tracking logs on a regular basis to identify the groups that are being used. Exchange Online
does not support the Get-MessageTrackingLog cmdlet found in on-premises versions, so searches must be performed
through the message trace section of EAC or by running the Get-MessageTrace cmdlet. Alternatively, you can look at
a list of groups and make an informed guess about which are disused, or you can invest in a reporting product that
includes the ability to locate these groups.

Finding Inactive Distribution Lists

Another common request is to know what distribution list are inactive and are therefore a candidate for removal.
Exchange Online does not include a way to find and report inactive groups, so we must create one with PowerShell.
The key points to remember are:

A distribution list is active when people use it to address messages.

Evidence of distribution list activity can be found in the message tracking logs by running a message trace to find
events noting the expansion of distribution list memberships.
Exchange Online keeps message tracking logs online for up to 10 days, after which the information is moved into
Office 365 data repositories and kept there for an extra 80 days. Thus, online searches can only look back 10
days to find expansion events. See Chapter 17 for more information about running message traces from the
Security and Compliance Center.

With these points in mind, we can write a script to collect expansion events from the message tracking logs for the
last 10 days and store the results in a table. We can then check the distribution lists in the tenant against the table to
discover if we find a match. If we do, we know that the distribution list was used in the last ten days. If not, it’s was
inactive in that time. Apart from reporting each list as it is checked, the script also outputs the results to a CSV file.

[PS] C:\> $EndDate = Get-Date

$StartDate = $EndDate.AddDays(-10)

$Messages = $Null

$Page = 1

Write-Host "Collecting message trace data for the last 10 days"

Do

$PageOfMessages = (Get-MessageTrace -Status Expanded -PageSize 5000 -Page $Page -StartDate $StartDate -EndDate $EndDate | Select
Received, RecipientAddress)

$Page++

$Messages += $PageOfMessages

Until ($PageOfMessages -eq $Null)

$MessageTable = @{}

$Messagetable = ($Messages | Sort RecipientAddress -Unique | Select RecipientAddress, Received)

$DLs = Get-DistributionGroup -ResultSize Unlimited

Write-Host "Processing" $DLs.Count "distribution lists..."

$Results = ForEach ($DL in $DLs) {

If ($MessageTable -Match $DL.PrimarySMTPAddress) {

 [pscustomobject]@{Name = $DL.DisplayName ; Active = "Yes"}

Write-Host $DL.DisplayName "is active" -Foregroundcolor Yellow }

Else {

[pscustomobject]@{Name = $DL.DisplayName ; Active = "No"}

Write-Host $DL.DisplayName "inactive" -Foregroundcolor Red }

$Results | Export-CSV c:\Temp\ListofDLs.csv -NoTypeInformation

Given that message traces give us a limited ten-day window to detect inactive distribution lists, this is not a practical
technique for a production-quality solution. Nevertheless, the method gives us the basis to develop the technique
further into something that might work. For instance, you could run a script every ten days and merge the results over
a period of a few months to give a more precise view of inactive and active lists.

Controlling User-Created Distribution Lists

The set of user-controlled OWA options includes a “distribution groups” link to allow users to manage distribution lists
they belong to and those that they own. To see the set of distribution lists owned by a user, select OWA Options ,
then General , and then Distribution groups . OWA decides what options are available for a user to work with
distribution lists by evaluating if the “MyDistributionGroups ” and “MyDistributionGroupMembership ” roles are in
the user role assignment policy assigned to their mailbox.

In Figure 7-15 we can see that both roles are present in the default role assignment policy used by Office 365. Note
that users can only create standard distribution lists – they cannot create security groups or dynamic distribution lists.
As explained in Chapter 12, the ability for a user to create a new Office 365 Group is controlled by a different policy.

Even if your tenant deploys a distribution list naming policy, to prevent “directory anarchy”, many organizations also
like to control the process who can create new groups and do not want users to create new groups without oversight.
You can remove the ability of users to create new distribution list by unchecking the MyDistributionGroups option in
the default role assignment policy as OWA will not then display the UI to reveal groups. However, a subtler approach
is also possible that allows users to see the groups to which they belong and to edit the membership of groups that
they own. Given that some organizations are migrating distribution list to Office 365 Groups, it makes sense to leave
users with the ability to work with groups that they already have while blocking their ability to create any more.

Figure 7-15: Distribution list entries in the default user role assignment policy

To achieve this goal, you must create a new user role assignment policy and amend it to remove the part that allows
users to create new groups. Here are the steps:

Go to the Permissions section of EAC, click user roles and then [+] (the plus sign) to create a new user role
assignment policy. Give the policy a suitable name and description and check all the boxes to allow users access
to the various items of functionality. In effect, you start off by cloning the default user role assignment policy. In
this example the new policy is “Restricted Group Management”.
Using PowerShell, define a new management role that is based on the existing MyDistributionGroups role. We
will call the new management role MyGroupsNoCreate .

[PS] C:\> New-ManagementRole –Name "MyGroupsNoCreate" –Parent MyDistributionGroups

The new management role still allows users to create new groups. To remove that ability, we have to remove the
reference (or role entry) to the New-DistributionGroup cmdlet from the MyGroupsNoCreate management role.
This command breaks the link.

[PS] C:\> Remove-ManagementRoleEntry MyGroupsNoCreate\New-DistributionGroup –Confirm:$False

A user role assignment policy consists of a set of connections between management roles and the policy. The new
user role assignment policy that we created in the first step still contains a reference to the standard
MyDistributionGroups role. We need to remove it before we can add the altered management role that we have
just created.

[PS] C:\> Remove-ManagementRoleAssignment "MyDistributionGroups-Restricted Group Management"

–Confirm:$False

Now we add a management role assignment to link the MyGroupsNoCreate role to the “Restricted Group
Management” policy.

[PS] C:\> New-ManagementRoleAssignment –Name "MyGroupsNoCreate-Restricted Group Management"

–Role MyGroupsNoCreate –Policy "Restricted Group Management"

We can then check that the correct roles exist in the “Restricted Group Management” policy. You should find a
link to the MyGroupsNoCreate in the list and no reference to MyDistributionGroups. Remember that
MyGroupsNoCreate has all the functionality of MyDistributionGroups except the link to the New-
DistributionGroup cmdlet. Because they cannot run the cmdlet, a user to whom we assign this policy can do
everything except create a new group.

[PS] C:\> Get-ManagementRoleAssignment –RoleAssignee "Restricted Group Management" |

Format-Table Name, Role -AutoSize

To prove that everything works, we apply the Restricted Group Management policy to a mailbox:

[PS] C:\> Set-Mailbox -Identity ‘Nancy Anderson’ -RoleAssignmentPolicy "Restricted Group Management"

Figure 7-16: The effect of the restricted user role assignment policy
Then, after waiting ten minutes or so to ensure that any cached permissions are clear, we log in to validate that when
the user accesses the distribution groups section of OWA options, they still see the set of groups that they belong to in
addition to the groups that they own. What we want to see is that the plus sign (+) icon that allows them to create a
new distribution list is missing, as it is in Figure 7-16 .

Distribution List Naming Policy

If you decide to allow users to create new distribution list, it is logical to expect that some users will not be as
disciplined as you would like them to be when it comes to creating logical and sensible names for their groups. No one
likes to see entries such as “Billy-Bob’s Fishing Aficionados” show up in the GAL as the group name does not really
convey a sense of the true business purpose behind its existence.

To help organize the GAL, you can impose a group naming policy for Exchange Online. The policy can define:

A prefix to apply to the user-supplied name for a new group. The prefix can be multi-part and consist of text
strings and the values of attributes for the user account stored in Azure Active Directory. The policy forces the
insertion of a blank character if the user account does not store a value for the chosen attribute.
A suffix to add to the user-supplied name for a new group. Again, you can use both text and attribute values.
Barred or forbidden words that users cannot include in the name of a new group. For instance, you can prohibit
potentially offensive terms by including them in the policy.

Figure 7-17: Creating a distribution list naming policy

To configure the group naming policy, go to the Groups section of the EAC and click the ellipsis […] button and select
Configure group naming policy. Figure 7-17 shows the dialog used to create the general part of a group naming policy.
The “blocked words” section of the policy allows an administrator to define a set of words that cannot appear in the
name of a distribution list. These might be words that you consider offensive or words that have no business meaning.
The parameters selected by the administrator indicate that all group names should be prefixed with “DG “ (the text
string DG plus a space) followed by the value of the Department attribute extracted from the user’s account and then
another string value containing a space. The policy should generate names for distribution lists like DG Sales
Members, DG Accounts Team 1, DG Senior Executives, DG Volleyball team, a nd so on.

The logic behind a policy for group names is to ensure that all groups exist in a single section within the GAL. Adding
the department name ensures that all the groups belonging to a department are found together. Behind the scenes,
the group naming policy exists as settings in the tenant organization configuration for Exchange Online. You can view
the settings with the Get-OrganizationConfig cmdlet:

[PS] C:\> Get-OrganizationConfig | Format-Table Distr* -AutoSize

DistributionGroupDefaultOU DistributionGroupNameBlockedWordsList DistributionGroupNamingPolicy

-------------------------- ------------------------------------- -----------------------------

{XXX, Cheese} DG <Department> <GroupName>

These properties have the following meanings:

The DistributionGroupBlockedWordsList property stores a set of words that cannot be used in a group name.
The DistributionGroupNamingPolicy property holds the format defined for group names. In the example shown
below, the policy consists of four elements:

A text string “DG “ (including a space).

<Department> means that the Department value stored in the user’s Azure Active Directory account is inserted.
Another space is then inserted.
<GroupName> means the name of the group as supplied by the user.

The DistributionGroupDefaultOU property is only used in on-premises deployments

It is possible to create or amend the naming policy through PowerShell. For instance:

[PS] C:\> Set-OrganizationConfig -DistributionGroupNamingPolicy "DG <Department> <GroupName>"

If you need to remove the group naming policy, set the value of DistributionGroupNamingPolicy to $Null :

[PS] C:\> Set-OrganizationConfig -DistributionGroupNamingPolicy $Null

The group naming policy only applies to user-created groups and has no retrospective effect. You must update the
names of existing distribution lists if you want them to follow a new policy, probably by issuing a PowerShell query to
locate non-compliant groups and then update the names of those groups according to the naming policy. Any group
created by an administrator through the EAC will not have the naming policy applied to it because Exchange assumes
that an administrator knows what the group name should be. However, if you create a distribution list with
PowerShell, you must pass the IgnoreNamingPolicy parameter to instruct Exchange Online to not apply the policy. For
example:

[PS] C:\> New-DistributionGroup –Name "Tiger Team" –Members JSmith, Bowens, KAkers

-IgnoreNamingPolicy

The group naming policy defined in the Exchange configuration does not apply to dynamic distribution lists or security
groups because users cannot create these types of groups. The policy does not apply to Office 365 Groups, which have
their own naming policy explained in Chapter 14.

Defining a distribution list naming policy for Exchange Online can also affect the way that on-premises distribution
lists synchronize in hybrid deployments. To make everything consistent, Exchange applies the naming policy to groups
that originate in the on-premises environment when they synchronize with Azure Active Directory objects with
AADConnect. Thus, an on-premises group might have a different name in the on-premises directory to that shown in
the cloud unless you make sure to apply the same naming policy in both environments.

Migrating On-Premises Distribution Lists to Exchange Online

Office 365 includes no out-of-the-box method to migrate a distribution list from an on-premises Exchange organization
to Exchange Online. When an on-premises organization is synchronized with Exchange Online in a hybrid deployment,
the suggested method is to remove the distribution list from on-premises and recreate it as a brand-new object in the
cloud. For anything but simple lists with just a few members, this is a tiresome process, but it reflects the fact that
transferring a distribution list to the cloud can be quite complex because:

The owner(s) of the distribution list might not have their account(s) in the cloud. An on-premises user cannot
manage a cloud-based distribution list.
Objects for mail-enabled members of the distribution list might not exist in the cloud. For example, a mail contact
in the on-premises environment might not be synchronized to the cloud.
The distribution list might hold other distribution lists.
The proxy addresses for the on-premises distribution list must be transferred to the new list.
Some properties of distribution lists refer to other directory objects that must exist in the target environment
before they can be used. For example, the property controlling the ability of users to send email on behalf of the
list.

It is possible to write a PowerShell script to concurrently connect to Exchange on-premises and Exchange Online and
perform the processing to transfer a distribution list. The script must:

Check that all the prerequisites are satisfied for the transfer to go ahead. For example, are all the members of
the list known in the cloud.
Create the target distribution list in Exchange Online.
Read information about the source distribution list from Exchange on-premises and update the properties of the
target distribution list.
Assign a new proxy address to the source distribution list and transfer it to the target distribution list.
 Update the membership of the target distribution list with the membership of the source list.
Hide the source distribution list from address lists so that the only list that is visible to users is the target.
Eventually, if the transfer worked and no problems are found, the old list is removed.

An example of a migration script for distribution lists is available in GitHub (the documentation is here ). As with any
script, you should test it carefully and adapt where necessary to meet the needs of your deployment.

On to SharePoint
Exchange Online is a very important base workload for Office 365, but so is SharePoint Online. In fact, given the
amount of new Office 365 Groups and Teams being created, all complete with SharePoint site collections, SharePoint
Online is more popular now than ever before. We find out about its capabilities in Chapter 8.
Chapter 8: SharePoint and OneDrive
For Business
Juan Carlos González & Gustavo Vélez

SharePoint Online and OneDrive for Business are part of the core workloads present in Office 365, giving the ability
not only to store documents and information, but also to build modern workplace solutions and services on top of both
platforms. This chapter focuses on describing how to manage both services using their admin centers, PowerShell and
available APIs. We also describe some basic SharePoint concepts and how it differs from its on-premises counterpart.
Finally, we cover some points to consider when migrating from SharePoint on-premises and on-premises file servers to
SharePoint Online and OneDrive For Business.

SharePoint Online
Two kinds of SharePoint people exist in the world of Office 365. The first know SharePoint from its on-premises
version; the second only know SharePoint from its contribution to applications like Teams, Groups, and Planner. The
first view SharePoint as a large, complex, and powerful document-centric ecosystem capable of delivering great
applications to companies. The second consider SharePoint to be the document management system for Office 365.
Part of the charm of Office 365 is that both viewpoints are correct. On the one hand, both platforms (SharePoint on-
premises and Online) include out of the box a bunch of collaboration, communication, document management and
business processes modeling features; and on the other hand, they include the building blocks for many kinds of
Modern Workplace solutions through a set of rich APIs and extensibility mechanisms.

When the product first appeared as SharePoint Portal Server 2001, it shared the same ESE-based database used by
Exchange 2000. The vision for that version of SharePoint was a departmental document management application and
was very different to what we see in SharePoint today. SQL is now the underlying database and SharePoint has
evolved to become the fulcrum of a web-based ecosystem that does much more than document sharing. According to
statements made by Microsoft VP Jeff Teper at the SharePoint Virtual Summit 2018 , SharePoint and OneDrive are
used by more than 400,000 organizations, 70 % of which use SharePoint Online. Much of that success is driven
through a thriving developer community and the many applications they create to run on top of SharePoint. Some of
those applications are available for SharePoint Online and more will come, which in turn creates further opportunities
for tenants to exploit the platform.

SharePoint Online shares much of the same codebase with the on-premises versions of SharePoint 2016 and
SharePoint 2019. Indeed, SharePoint 2016 was the first version of the product defined by Microsoft as “cloud-born .”
Having a common foundation allows Microsoft to deliver some of the technical innovations developed in the cloud to
SharePoint on-premises customers at a more rapid cadence through product Feature Packs and the product updates
released monthly.

In the past, it was difficult to configure the on-premises version of SharePoint integrated with other Office
applications such as Exchange. Microsoft laid down the procedures for integration in the form of multi-step checklists,
but in practical terms, the process was never straightforward and was prone to error. These challenges evaporate in
the cloud because Microsoft does the work to connect Exchange Online and SharePoint Online together in a way that
makes the combination more agreeable and easier to work with, even if the join between the two products sometimes
shows. We do not have to worry about the finer points of APIs and how SharePoint Online and Exchange Online
communicate (REST APIs in one direction, Exchange Web Services and the Outlook REST API in the other and of
course the possibilities provided by the Microsoft Graph). Instead, we can focus on using the platform for productive
purposes.

Stitching together two products to deliver functionality is one way to extract more value, but that approach limits
developers to whatever combining applications can produce. It is also limited by the capacity of on-premises IT
departments to be able to exploit new capabilities by deploying the necessary components into production, which is a
common problem shared in both the Exchange and SharePoint installed base.

Introduction to SharePoint Online

Although we cannot deny that the cloud versions of Exchange and SharePoint are not as flexible in terms of
customizability as their on-premises counterparts, as Office 365 matured, Microsoft has taken a more holistic view of
the value that they can extract by treating the different applications within the suite as a kind of software parts bin.
Because all the applications run inside Office 365 and Microsoft can update them quickly to meet new requirements,
software designers can come up with ideas for new functionality that draw together different elements from all parts
of Microsoft’s cloud services portfolio. Office 365 Groups and Microsoft Teams (Chapters 12 and 13) are great
examples where applications combine SharePoint Online and other components to create some new collaborative
experiences for teamwork. The way that Planner (Chapter 15) builds on Groups to deliver task management
functionality for teams is another example.

Further proof is in the way that SharePoint is consumed by other applications. For example:

Delve : aka “Search for Office 365” (Chapter 9), provides users with a way to see the most relevant information
 drawn from sources across Office 365. SharePoint Online and OneDrive for Business are the sources for most of
the information exposed through Delve (Exchange Online is less of a source as Delve only exposes attachments
emailed to a user). Delve also serves as the landing point for user profile information.
Office 365 Video : A combination of SharePoint Online (to store metadata about video content) and Azure Media
Services (to transcode and stream video content). This portal is a good example of how to assemble components
from various parts of the Microsoft cloud to deliver new functionality. Office 365 Video and its replacement,
Microsoft Stream, are presented in Chapter 9 of the Companion Volume.

Documents and other content stored in SharePoint Online site collections is the common thread running through the
new modern SharePoint site (Modern Team Sites and Communication Sites). In fact, SharePoint has returned to its
roots and moved away from an attempt to be an application platform. Office 365 and its growing collection of REST-
based APIs is the platform; SharePoint Online contributes the ability to manage large collections of documents and
metadata in that platform and offers native extensibility through building blocks such as the SharePoint Framework
(SPFx) PnP extensibility model, Site Designs and Site Scripts, PowerApps or Flow. SharePoint Online is a critical piece
of the overall Office 365 story. Any Office 365 deployment that seeks to move workload into the cloud therefore needs
to incorporate SharePoint Online into the plans to extract full value from Office 365.

Although some will feel that the importance of SharePoint is reduced because it is not as obvious as Exchange appears
to be within Office 365, they are wrong. SharePoint is a critical part of Office 365 and Office 365 could not function
without SharePoint. It is as simple as that.

Differences Between SharePoint On-Premises and SharePoint Online

The most significant difference between SharePoint on-premises and SharePoint Online is that the former is a
standalone platform while the latter is a core workload within an integrated environment. Many Office 365 tenant
administrators and users interact with SharePoint Online daily without knowing that their information is managed by
SharePoint. Compared to the tweaks that administrators might have to perform to make Exchange Online handle
issues such as anti-malware settings, administrators have relatively few interactions with the SharePoint Admin
Center or the SharePoint Online Management Shell. In other words, SharePoint delivers high-quality document
storage and management with a minimum of fuss.

Two reasons exist why the significant role of SharePoint is often hidden within Office 365. First, the evolution of Office
365 has removed some of the reasons why administrators have needed to interact with SharePoint because Microsoft
has developed service-wide components. Take eDiscovery for instance. In the past, SharePoint administrators set up
the eDiscovery Center and created eDiscovery cases there. Those cases could span both Exchange mailboxes and
SharePoint sites, but they could not process information in public folders or group mailboxes or other workloads
running inside Office 365. The SharePoint eDiscovery Center served a purpose in an on-premises context but not in
the cloud where we use the Security & Compliance Center

The second reason is that SharePoint is a backend service for other applications such as Office 365 Groups, Microsoft
Teams and Project Online. Some users understand that they interact with SharePoint when they access Files in Teams;
others will not, especially if they never used SharePoint before or never want to use advanced document management
features like check-in/check-out, metadata or document versions. Delve is a third example. The documents that Delve
shows to users come from SharePoint and OneDrive sites, both of which use SharePoint storage, but Delve replaces
SharePoint Search with a more intelligent way for users to find information.

Other differences exist between both platforms. SharePoint Online is not as customizable, configurable and
extendable as is SharePoint Server. The cloud version does not support features like BLOB storage or self-service
backup and restore. But a lack of customization options (at least compared to what you might expect from an on-
premises server) is the same for all cloud applications and SharePoint is not unique in this respect. In any case,
Microsoft gives tenants different methods to customize and extend their infrastructure that are better suited to a
cloud platform, like Remote Provisioning pattern for SharePoint solutions, Site Scripts and Site Designs, the SPFx,
PowerApps or Flow. It is not a case of less customization. It is just different in the cloud.

Basic SharePoint Online Concepts

Those who have not used the on-premises version of SharePoint are probably unaware of the basic concepts that
underpin the operation of the application. Here is a quick guide of the major SharePoint terms and components.

Sites
A Site is the basic building block for SharePoint Online and is the place where content is kept and managed. Each site
can have its own security, layout, theme, navigation, regional settings, and custom search settings. A site can also
have workflows built-in to ensure that the information held in the site is processed in a specific manner. The latest
form of sites is known as modern SharePoint Online sites and includes “group-enabled” team sites, modern team sites
and communication sites. A “Group-enabled” site is associated with an Office 365 Group, which is responsible for the
management of the site membership. Group-enabled sites are more powerful than traditional SharePoint sites because
they gain access to the other functionality available to an Office 365 group, like the group mailbox and calendar. On
the other hand, group-enabled sites are less configurable, since the number of options available in the site settings
page is much smaller compared to a classic SharePoint Online site. Communication sites are designed to be mobile
friendly and configured to display information in a dynamic way. Their role is to serve up information to users that is
drawn from different sources available to SharePoint.

Updates to SharePoint self-service site creation: In late July 2018, Microsoft began to roll out to targeted release
tenants the following important changes to the site creation experience :

Allow users to create modern team sites even when Office 365 Groups creation is disabled. This change lets
users create a modern team site that is not connected to an Office 365 Group.
Select the default language for both modern team sites and communication sites at the time they are created.
The language selected can be different from the organization’s default language.
Provide new admin controls in the SharePoint Online admin center to enable/disable self-service site creation
and subsite creation.

As shown in Figure 8-1 , sites are the entry point to SharePoint Online home page, the entry point to SharePoint
Online from the Office 365 Portal. SharePoint Online decides what sites appear on a user’s home page based on the
user’s activity within Office 365. The activity is stored as signals in the Microsoft Graph gathered from user’s
interaction with SharePoint Online team sites, Office 365 Group document libraries (including those used by Teams),
and the sites used to hold content for the Office 365 Video channels. Finally, from the SharePoint Online home page
you can create new sites (modern team sites or communication sites) and news items posted to any site the current
user can access.

Figure 8-1: Sites listed on the SharePoint home page

Search
SharePoint Online uses the Search Service to create content indexes from the files kept in sites. The Search box on
the SharePoint Online home page allows users to search specific search terms across different scopes (Sites, Files,
People and News). In the case of the Files and Sites Scopes, the search engine returns all the sites (including
OneDrive For Business site) and files and/or folders to which the users have access where the search term is found.
News and people scopes provide results of any news containing the search term and any Office 365 user whose user
profile includes that term. The way that Search displays results is like the approach taken by Delve, with the
difference being that the results of SharePoint searches can include sites, documents, people and news while the
results from Delve searches provide references to “boards” and “people.” Each site also has a search capability, but in
this case, the scope of the search is limited to the site.

SharePoint Search and Artificial Intelligence (AI): SharePoint Online Search uses Microsoft AI Services to
search photos stored in a site or in OneDrive Of Business. When a picture or a document with a picture is uploaded to
SharePoint Online or OneDrive, Microsoft AI Services detects the new object and extracts metadata and text for use
in searches.

Apps
Sites can include Apps (or Add-ins) and Web Parts to expand the functionality available through the site. A document
library is a very common example of an Add-in used with SharePoint sites to allow for the organization and
management of documents belonging to the site.

Traditionally, web browsers have served as the interface to information held in SharePoint sites. This aspect of the
product is now changing as Microsoft delivers more mobile applications that consume SharePoint contents. Examples
of these mobile applications include Delve, SharePoint, Teams and OneDrive for Business.

Site Collections
To enable easier administration and to ensure that sites that have a common purpose are grouped together, sites are
organized into Site Collections . A site collection is a container including a top-level site and all the sites below it,
but It can also be just a single site. For example, each Office 365 Group (including those used by Microsoft Teams) has
a site collection composed of just one site. By default, SharePoint Online hides the site collection for an Office 365
Group to prevent the use of current SharePoint Online administration tools to manage its content. Another example is
the Office 365 Video Portal, which organizes videos into channels, each of which is a site collection. However,
Microsoft allows tenants to manage those hidden site collections through the new SharePoint Online Administration
Center.

SharePoint email notifications : SharePoint sends email notifications for actions such as sharing. For this
purpose, SharePoint uses its own basic SMTP server and does not use Exchange Online, even if Exchange is available
within a tenant. You cannot alter this processing, nor can you control how the SharePoint SMTP servers work.

SharePoint Online Quotas and Limits

Many of the SharePoint on-premises boundaries and limits are also present in its cloud counterpart, but there are also
some specific limits that only apply to SharePoint Online. For instance, in a SharePoint Online tenant we can create up
to 500,000 site collections and a single site collection can store up to 25 TB of information.

On-premises SharePoint servers use SQL databases to store the data held in site collections. SharePoint Online also
uses SQL, but the management of storage is much simplified in that Microsoft takes care of it. The storage limit for a
tenant is calculated based on the Office 365 licenses owned by the tenant plus any added storage the tenant buys
from Microsoft. Briefly, the storage quota for a tenant is:

1 TB plus 10 GB per licensed user plus purchased storage

Thus, a tenant with 200 licensed users has a SharePoint Online storage base of 3 TB. If this is insufficient, the tenant
can buy an unlimited (in practice) amount of extra storage . The added monthly charge is based on the amount of
added storage. If your storage requirements decrease, you can change the amount of added storage or cut it
altogether.

Increase in SharePoint Online Storage : When Microsoft increased the amount of per-user storage available to
tenants, they also increased the number of on-premises items an organization can index in Office 365 with the cloud
hybrid search solution as detailed in this support article . This increase will not be added to SharePoint Online
standalone plans, Office 365 F1 and Microsoft 365 F1 plans.

Storage quotas for Office 365 Groups

The quota consumed by Office 365 Groups comes from the tenant’s storage allocation. The limit for an individual
group is the same for a regular Site collection and obeys the normal limits for SharePoint Online . SharePoint Online
usually manages storage automatically but it is possible to set the Site Collection Storage Management option to
Manual to be able to assign a set quota with PowerShell using the Set-SPOSite cmdlet. The Site Collection Storage
Manage option can be configured in the “settings” menu in the SharePoint Online Admin Center. The following
example sets a quota of 25 GB (25,600 MB) and a warning limit of 24.42 GB (25,000 MB) for the specified site. Use
the Get-SPOSite cmdlet to confirm that the new setting is in place. In this case, SharePoint Online adjusted the quota
up slightly, but the important thing is that the assigned quota is higher than the current storage usage.

[PS] C:\> Set-SPOSite –Identity https://office365itpros.sharepoint.com/sites/o365itpros

-StorageQuota 25600 -StorageQuotaWarning 25000

[PS] C:\> Get-SPOSite –Identity https://office365itpros.sharepoint.com/sites/o365itpros -Detailed | Format-List

LastContentModifiedDate : 05/02/2016 11:41:10

StorageUsageCurrent : 6465

Url : https://office365itpros.sharepoint.com/sites/o365itpros
StorageQuota : 26624

StorageQuotaWarningLevel : 25000

ResourceQuota : 300

ResourceQuotaWarningLevel : 100

Title : Office 365 for IT Pros

Office 365 Groups quota management in the new SPO Admin Center : The quota for an Office 365 Group site
can also be modified through the Active sites section in the new SPO Admin Center. To change the quota for an
existing Group, just select the Group site in the list of active sites, click on the “Storage” action in the action bar so
the “Change storage limit” panel is shown. Add the new value for the quota storage, click save and you are done.

To discover the amount of storage your group document libraries occupy, go to the Reports section of the Office 365
Admin Center, and select the SharePoint Site usage report. At the bottom of the screen you’ll see a table of
SharePoint site details for activity and usage . Click a heading to sort the table in that order. For instance, click
Storage Used (MB) to sort by the storage used and you will soon discover what sites occupy most space. See
Chapter 21 for more details about the reports available in the Office 365 Admin Center and third-party reporting
options.

Managing SharePoint Online

SharePoint Online can be managed in the same way as other Office 365 core workloads, through a dedicated Admin
Center and PowerShell. This section describes how to manage SharePoint using both approaches.

Accessing the SharePoint Online Administration Center

Like SharePoint on-premises, SharePoint Online has a "Central Administration" portal called the SharePoint Online
Administration Center. To access the SharePoint Admin Center, go to Admin Centers in the Office 365 Admin Center
and select SharePoint, " or type the URL https://[tenant]-admin.sharepoint.com into the browser. Note that the "-
admin" part in the SharePoint Admin Center is mandatory as this tells SharePoint that you want to access the
administrative portal instead of sites.

Although the current SharePoint Admin Center has been part of Office 365 since the first release of the platform in
2011, work has progressed on the new Admin Center (Figure 8-2 ) and this is now Microsoft’s focus for future
administration of SharePoint Online. We will discuss the key elements of the classic Admin Center later in this
chapter.

Figure 8-2: The new SharePoint Admin Center

One of the basic shortcomings of the classic SharePoint Admin Center is its inability to manage modern sites (modern
team sites, communication sites and hub sites) through the Site collection list. Modern SharePoint sites can only be
managed using PowerShell. However, the new Admin Center displays all existing Site collections in a modern list view
that allows the administrator to apply filters, add extra columns such as the external sharing column or find Site
collections linked to an Office 365 Group. Site collection management in the new SharePoint Admin Center also
supports the export of the list of site collections to a CSV file.

The new Admin Center is visually aligned with the modern SharePoint Online experiences. In addition to a clean look
and feel and the enhanced Site collections management, there are some other improvements in the new Admin
Center, such as displaying a summary of the SharePoint activities in the tenant, messages from the Message Center
and the Service Health status. Message Center and Service Health dashboards reflect specific information about
SharePoint Online and eliminate the need to check this information in the Office 365 Admin center. The new
SharePoint Admin Center is an ongoing, project that still misses many features. Over time, the plan is for Microsoft to
add the missing functionality. Table 8-1 details the options available today in the new Admin Center.

Admin Description
Option

Active sites Lists the active sites in the tenant. To see the details of a specific site, select it in the list and click the
“i” setting so the site details panel displays site usage statistics and site properties (Domain, URL,
Template, etc.). The default sites list view can be changed at any point for any of the default site views
or a custom view. For a selected site, the following management actions are provided:

Owners: Depending on the site type, you can change/modify Group owners (Group sites) for the
selected site or change/modify primary and secondary site collection admin (Classic sites and
Communication sites).
Hub sites: You can promote the selected site to become a Hub site (“Convert to hub site” option)
or to join the site to an existing Hub site (“Associate with a hub site” option.
Sharing: This option configures the external sharing setting for the site (Sharing options for SPO
sites are detailed later in this chapter). From the sharing setting panel, it is also possible to access
the global sharing settings page.
Delete: It deletes the selected site by moving it to the site collections recycle bin.
Storage: If manual quota management is enabled in the SPO tenant, you can modify the storage
quota for the site selected.
Email: Sends a custom e-mail to the administrators/owners of the site.

You can create a new SharePoint site using the Create button. Currently it’s possible to create both
modern team sites, classic sites, and communication sites. However, only a few classic site templates
are available (Document Center, Enterprise Wiki, Publishing Portal, Team Site).

Deleted sites Lists all deleted site collections in the tenant. An administrator can select any deleted site collection
and restore it using the “Restore” option.

Access Gives access to different settings that can be used to restrict access to SPO sites:
Control

Unmanaged devices: Settings to control how unmanaged devices can access SPO sites and
information. Those settings are available through Azure AD and need an Enterprise Mobility +
Security subscription.
Idle session sign-out: Disabled by default, this setting is a mechanism to first warn and then sign
out users on unmanaged devices if they have been inactive for a period and they don’t opt to stay
signed in when they sign into SPO or Office 365. Two settings can be configured when this setting
is enabled: the idle session period (1-hour minimum, 24 hours maximum) and the warning notice
(1 minute minimum, 30 minutes maximum).
Network location: When enabled, it shows the IP addresses from which access to SPO is allowed.
Apps that don’t use modern authentication: Enabled by default, this setting blocks access to SPO
from Apps that don’t use modern authentication.

Settings This section has shortcuts to settings that can be applied globally to SPO:

List & libraries: Enables the modern experience in SPO lists and document libraries. By default,
this setting is on.
Notifications: When this setting is on and notifications are also turned on in the end users’ mobile
apps, it allows to send notifications about SharePoint content.
Site storage limits: It defines how the available SharePoint storage is managed. There two
 possibilities to manage storage: Automatic and Manual.
Default admin experience: Enables the default Admin Center for SharePoint Online. If the setting
is on (It’s off by default), the new SharePoint Admin Center will be opened.
Site creation: In this section, configure the managed path and time zone to be used when a new
SharePoint site is created and also if end users are allowed to create sites from the SharePoint
home page.

API Simplifies the process to manage permissions requested by third-party APIs called by SPFx solutions
management and custom scripts. An administrator can easily approve or reject a request from the list using the
Approve or reject panel.

Table 8-1: Administration options in the new Admin Center

Links are also available to launch the classic SharePoint Admin Center, the OneDrive for Business Admin Center, and
to access the SharePoint migration tool.

Admin roles to Manage SharePoint Online

To administer SharePoint Online, several administrator roles can be assigned to users:

The Global Office 365 Administrator role. When provisioning the Office 365 tenant, a few site collections are
automatically created, and the Office 365 Global admin is added as the primary Site Collection administrator for
each site.
The SharePoint Online administrator role. This role must be assigned by the Global Office 365 administrator.
This account has access to the SharePoint Online and OneDrive admin centers and can manage both services. To
assign an account as SharePoint Online administrator, go to the Office 365 Admin Center, go to "Users" "Active
users" and click on the required account. From the new window, click on "Edit" of "Roles", select "Customized
administrator" and then select "SharePoint administrator".
The Site Collection administrator is chosen by a SharePoint Online administrator or an Office 365 one and has
permissions to manage a specific site collection. A single site collection can have several administrators, but only
one primary administrator. The SharePoint Online administrator can assign permissions to the primary site
collection administrator when creating a site collection and can add more administrators for the site collection at
any moment afterwards. Site collection administrators do not have access to the SharePoint Online admin center.
A site collection administrator can be added to a site collection in three different ways: from the site collection
lists in the SharePoint Admin Center, PowerShell - and from the site settings page.

Managing Site Collections

In May 2016, Microsoft presented a new ("Modern") vision for the future of SharePoint , vision that has been rolled
out in several waves of enhancements to SharePoint Online team sites in Office 365. Modern team sites and
communication sites, Hub sites, modern site pages, new SharePoint home page, or PowerApps and Flow integration
are some examples of the new enhancements introduced in the platform

Modern pages are one of the key building blocks of the modern SharePoint. They are fast, easy to author and support
rich multimedia content to make sure that they look great on any device, in a browser, or within the SharePoint mobile
app. Examples of how to use modern Sites and pages are showing status and trip reports, guides and frequently asked
questions. SharePoint Modern pages are built with modern Web Parts that can be customized based on business
needs, supporting the addition of documents, videos, images, site activities, Yammer feeds and increasingly more
items as Microsoft releases new Web Parts. In addition, the user experience of adding content to modern pages has
been changed: Just click the + sign and pick a Web Part from the toolbox to start customizing a modern page. Using
the new SPFx, developers can build custom Web Parts that show up directly in the toolbox. SPFx also supports the
creation of SPFx extensions to customize specific areas in SharePoint modern pages such as the page footer and page
header or modern list and document libraries by placing new actions in the list/document library command bar or the
item/document menu.

Metadata support in modern SharePoint Online Pages : Although modern SharePoint pages have been around
for a while, they didn’t support custom metadata in the way classic publishing pages did. Fortunately, in July 2018,
Microsoft began to roll out the ability to tag modern pages with custom metadata added to the Site Pages document
library by means of custom columns. Once the custom columns are added to a document library, page properties can
directly be updated from the page itself through Page details in the page menu. In addition, the Highlighted Content
and News web parts have been updated to support the use of metadata to organize how pages are displayed in both
web parts.

The integration of Groups and SharePoint team sites means that when a new team site is created, and Office 365
Groups creation is enabled in the tenant, a new Group is created to manage site membership. When a Group is
created, Office 365 provisions the Group with a shared inbox, its own calendar, OneNote notebook, a Planner Plan for
task management and a Modern SharePoint team site. Each Group gets a Modern home page, document library, lists
and business apps. Membership can be managed through a variety of interfaces, including PowerShell and the Office
365 Admin Center.
Because Office 365 Groups expose the breadth of capabilities offered in SharePoint Online plus several other services,
such as Office 365 Video and portal sites, their content storage requirements are higher than for traditional Site
Collections. To address this, Microsoft increased the SharePoint Online Site Collection storage limit from 1 TB to 25
TB. Additionally, managing modern team sites lines up to a great degree with Office 365 Groups administration by
following usage guidelines, naming conventions and classification, meaning that there are some site administration
tasks that team sites rely on that are controlled within the SharePoint Online Administration Center (like storage
quota or the ability to adhere to a custom site provisioning experience) and some others through the Office 365 Admin
Center; this divergence might be addressed in the new SharePoint Admin Center.

Modern Sites and pages can be configured using Web Parts and extended writing custom SPFx code. Existing
customized home pages remain in classic mode.

Microsoft’s Deployment of SharePoint : At the Ignite 2018 conference, Microsoft shared some details of their
internal deployment of SharePoint and said that they had 233,000 users spread over 125 countries accessing 253,000
sites, 74% of which are modern sites. Membership is controlled by 179,000 Office 365 Groups, and the infrastructure
manages over 3 petabytes of data.

Controlling Site Collection Creation

It is possible to allow users to create new Sites Collections or restrict creation to only administrators. Because of the
tight connection that Team Sites now have with Office 365 Groups, this is not a simple matter as you must consider
whether you want users to be able to create sites, group-enabled sites, or both.

The basic control over Site creation is achieved by showing or hiding the Create site command on the SharePoint
home page of the root Site of the Site Collection. This is the button that invokes the Create Site wizard. A SharePoint
tenant setting controls whether the button appears on the page, for all the Site Collections. To access the setting,
open the SharePoint Admin Center and click "Settings". Then, under "Site creation", select either:

Hide the Create site command, or

Show the Create site command.

The Site creation section also allows tenant administrators to decide what kind of Sites they allow users to create.
Two options are available to accommodate the diverse needs that exist within tenants:

Create a new team site or a communication site (the default): This choice allows users who do not have the
permission to create new Office 365 Groups to go ahead and create a modern site or a communication site. Those
who have the necessary permission to create an Office 365 group can create a new group-enabled site or a
communication site.
Create a classic team subsite : If you select this choice, users will create classic subsites under the path
specified in the “Create sites under” field. You can create your own template as the basis for new sites (the Office
365 Patterns and Practices initiative is a better way to create a template for SharePoint Online).

In a similar way, there is also the possibility to authorize users to create sub-Sites in Site Collections or restricting it
to only SharePoint Administrators. The section "Subsite creation" in the "Settings" page controls if the creation button
is hidden or showed.

Creating a Group-Connected Team Site

If the SharePoint settings for a tenant allows the creation of group-connected sites and the user can create Office 365
Groups in the Office 365 tenant, the creation process for a team site also creates an Office 365 Group to hold the
membership for the site and to allow site members to have access to group resources.

Unlike the creation process used by OWA, Outlook, PowerShell, or an Admin console, SharePoint creates the new
team site at once rather than waiting for the first user to access the site. In effect, this is opposite to what happens
when OWA or another email client creates a new group, where Exchange Online creates the group mailbox first to
allow conversations to occur and notifications to go to new group members. After SharePoint creates the new group
object in AAD, a process of forward directory synchronization makes the presence of the new group known to
Exchange Online and forces the creation of the group mailbox. While this process proceeds, the user can build out the
contents of the SharePoint team site with new lists, document libraries, a customized home page, and so on. Office
365 uses the same creation process to set up a new team site and group when a user invokes this choice from
OneDrive for Business.

If there are existing team sites, it is possible to connect them to Office 365 Groups. When this happens, the existing
content, hierarchy, and permissions for a site stay intact and Office 365 connects the site to a new Office 365 Group
with the group membership populated based on the existing site membership. The owner can adjust the group
membership after creation. To connect an existing team site to Office 365 Groups, select "Connect to new Office 365
Group" from the Office 365 settings menu (cogwheel) and follow the prompts.

Connect existing SharePoint Team Sites to new Office 365 Groups : In May 2018 Microsoft began to provide
the capability to connect existing SharePoint Online Team Sites to new Office 365 Groups . Existing SharePoint sites
can be connected to a new group (or “groupified”) using the Set-SPOSiteOffice365Group cmdlet, CSOM API or using
the Connect to new Office 365 Group feature available in site settings options. An Office 365 Administrator or a
SharePoint Online one can disable this option by selecting the option Prevent site collection administrators from
connecting sites to new Office 365 groups in the Settings page in the SharePoint Admin Center. Microsoft has
also given detailed information about what SharePoint sites can be connected to a new Office 365 Group, a scanner
tool to analyze if existing SharePoint sites are suitable to be connected to Groups, and PowerShell scripts to enable
modern features in the Sites and configure Groups membership.

Settings and Permissions for Group-enabled Sites

When a new group-enabled team site is set up, SharePoint creates a team site to host the document library and
notebook. The gear (options) menu for a site reveals the Site Information panel to customize the details published to
users about the site as well as its classification, or label defining the importance of the information held in the site.
Also, the owner can access the Site Permissions panel and set the desired level of access for different member types.

The default configuration for a group-enabled site is that group owners have SharePoint full control permissions over
the site, while group members have SharePoint Edit permissions. If the site is public, every account in the tenant
excluding external users has SharePoint Edit permissions. Amend the permissions for the site to restrict access as
needed, but be aware that because you are working with Office 365 Groups rather than individual user accounts (as is
the case for on-premises SharePoint or classic sites), some care is needed to ensure that you do not interfere with the
ability of group members to access the resources available to the group, or for group owners to manage the group. It
can even be possible to select an individual user account and give them full control for the site. This permission is a
SharePoint permission over the team site and does not make the user an owner of the group.

One of the permissions that can be assigned for a site is “Share Site Only” access, which means that the owner can
give someone access only to the SharePoint resources instead of all the resources belonging to the group. Also, is it
possible to assign a user read-only access to the site, which removes their ability to edit or remove data from the site
(they might still be able to share files from the site). Assigning read-only access to users is a good approach to take for
sites holding information (like HR documents) and it is necessary to make the site content generally available.

Apart from the ability to set access to site control through these settings, it is unwise to try and use traditional
SharePoint access control over team sites used with groups because these controls expect to deal with individual
users rather than when an identity is shared by group members.

Hub Sites
Since the debut of SharePoint Online Communication sites in 2017, tenants have an easier way to work with
SharePoint sites via the modern user experience. Communication sites work best to present news and visually-rich
content, making them the best choice for sites around a single topic. Although Communication sites are easily edited,
the list of Web Parts that can be used is growing, and the pages are mobile-friendly, they are still not the best solution
to build real-world Intranets. Hub Sites provide the glue to group modern sites with a common purpose and that share
elements such as branding or navigation.

Hub sites integrate multiple Communication, Modern Team sites and even classic sites together to build a real
Intranet. For example, it is possible to have several enterprise departments, each with its Team site for collaboration,
together with multiple Communication sites with news and updates, interconnected to become an information
gateway, without having to create the sites as a single hierarchical collection.

Hub Sites Limits : The number of Hub Sites that can be created in a SharePoint Online tenant is limited to 50. This
limit, as announced at Microsoft Ignite 2018, will be increased to 100 soon. Microsoft also expects that in the future,
a tenant will be able to create more than 100 Hub Sites. Classic sites can also be added to a Hub site through
PowerShell or the “Associate with a hub site” option available in the new SPO Admin Center, but the Hub site
features will not be visible in any classic experience in the site.

The Hub site collection can aggregate news, events and highlighted contents from all its associated site collections
and fulfil some other valuable functions: it adds a consistent branding across all sites and a common menu, making it
easier to navigate between the sites. The user can search within the complete Hub site components and can use the
digest option that lets them organize selected news stories to go out as an email summary, with images and links.

There are also some limitations for the Hub site collections: there’s no workflow around the publishing process,
anyone who has edit permissions for a site can change any element of that site, and any new story pushes older ones
down the stack, making it impossible to create a "Most important" news that stays always at the top. Additionally, Hub
sites cannot be nested, limiting their potential as well, and they cannot deal with multiple languages. It is foreseeable
that Hub sites will mature in the future because they are ongoing work.

A Hub site is just a modern team site, or a communication site registered as a Hub using PowerShell, or a site
converted to be a hub site using the “Convert to a hub site” option in the new SharePoint Admin Center. Registering a
hub site with PowerShell is done using PowerShell cmdlets from the SharePoint Online Management Shell or by using
specific PnP cmdlets (We will talk about how to manage SharePoint Online with PowerShell later). To use the following
example, you need to use the latest version of the PnP Cmdlets, and open first a connection with the tenant. Then, you
can create a Teams site collection, elevate it to a Hub site, and associate other site collections to the Hub.

[PS] C:\> Connect-SPOService -Url https://<TenantName>-admin.sharepoint.com

[PS] C:\> Connect-PnPOnline -SPOManagementShell # Will ask for the site URL

[PS] C:\> New-PnPSite -Type TeamSite -title "New SiteColl" -alias "NewSite" -Description "New SiteColl" # Create a new Site
Collection

[PS] C:\> Register-SPOHubSite -Site https://<TenantName>.sharepoint.com/sites/NewSite # Ask for the principal(s) - Register the
SiteColl as a Hub site

[PS] C:\> New-PnPSite -Type TeamSite -title "Sub-SiteColl" -alias "SubSite" -Description "Sub-SiteColl" # Create a sub-
sitecollection

[PS] C:\> Add-SPOHubSiteAssociation -Site https://<TenantName>.sharepoint.com/sites/SubSite -HubSite

https://<TenantName>.sharepoint.com/sites/NewSite # Associte the subsite to the Hub site

Managing SharePoint Online Services and Settings

As with SharePoint on-premises, you can manage some of the core services available in the platform as well as key
settings. This section provides an overview of the services and settings that can be managed by means of the
SharePoint Online Admin Center.

User Profiles
Identity management in Office 365 is centralized in Azure Active Directory, but each core workload also has its own
directory (SharePoint Directory Store or SPODS in the case of SharePoint Online) that is synchronized bidirectionally
with Azure AD. While SharePoint Online does not allow us to manage Azure Active Directory users, it has the User
Profiles service as a mechanism to modify existing user properties or add new ones. Behind the scenes, a SharePoint
Timer Job ( Active Directory Import Job ) is responsible for all the actions required to import existing users into the
User Profiles service.

In the User profiles service page, Manage User Properties and Manage User Profiles links are the most commonly
used by a SharePoint Online Admin. The first link takes us to a page that lists all the user profile properties and from
where we can create new properties. The second link redirects to a central location from which we can look for a
specific user profile to modify some of his/her properties and their visibility (only for some of the properties), delete
the user profile, grant/control access to the user’s OneDrive or add additional OneDrive For Business administrators
to users’ OneDrive . Depending on the Office 365 subscription in use, some of the properties needed to take full
advantage of SharePoint Online features can be blank, so they must be manually or automatically filled (via
PowerShell). An example of such properties is the Work email property that is empty in standalone SharePoint Online
plans used for sharing functionality and alerts.

User profiles and Delve : User profile properties can be modified not only in the User Profiles section in the
SharePoint Online Admin Center, but also in the user profile page in Delve. The first approach implies that only an
Office 365 Admin or a SharePoint Online one can update user profile properties for any user in the Office 365.
However, the user profile page in Delve allows each individual user in an Office 365 tenant to update some of his/her
user profile properties. Delve does not allow to update all the user profile properties that can be modified in the User
profiles section in the SharePoint Online Admin Center.

Term Store
The Term Store in the SharePoint Online Admin Center provides a central location to create taxonomies ( Figure 8-3 )
that can be used in three different ways in a SharePoint Online tenant:

Create manage metadata fields that allow to classify a document or a list items by means of a term that is part of
a specific taxonomy.
Configure the global navigation of an Intranet solution deployed in the tenant.
Improve end user experience when using search capabilities by means of the search dictionaries that contain
specific Term sets to provide features such us including or excluding words for query spelling correction.

A custom taxonomy consists of a Group, at least a term set in the Group and terms in a term set. The maximum
number of levels of nested terms in a custom taxonomy is seven. The Term Store also contains enterprise keywords
and hashtags as a mechanism to freely tag contents with existing Keywords or new ones created by the end user.
Figure 8-3: Global SharePoint Online Term Store

Records Management
Despite its name, the Records management section in the SharePoint Online Admin Center simply allows
administrators to create new Send To connections that can be used by a Content Organizer configured on a
SharePoint site to route documents to a specified location (a document repository or a records center). It is also
possible to configure Send To connections to be used by end users, so they can manually copy, move or move and
leave a link when a document is selected in a document library.

Search
Search is one of the core features in SharePoint Online that can be customized by an Office 365 Administrator or a
SharePoint Online one to ensure that end users have a great experience when searching documents by content and
metadata through any site in the tenant. The search administration in SharePoint Online includes the following items:

Search Schema : This option provides access to the list of managed properties and crawled properties in the
tenant. An Administrator can update exiting managed properties or create new ones. Each managed property can
be mapped to one or more crawled properties. A crawled property is just content and metadata that is extracted
during a crawl from an item stored in SharePoint, such as a document, a SharePoint page or list item. A good
overview of crawled and managed properties can be found in this Microsoft article .
Search Dictionaries : Allows the creation of a list of terms in the Term store to include or exclude company
names extracted from the contents indexed by the search engine or to include or exclude words for query
spelling correction.
Authoritative Pages: This option helps to identify high-quality pages to improve search results relevance. There
are three different levels of authoritative pages that can be configured to identify more relevant contents in the
search index. We can also mark any not relevant pages in the Authoritative pages page.
Query Suggestions : Using this option we can upload query suggestions to the search engine so when the user
enters some search criteria in the search box, search suggestions for that search criteria are shown.
Result Sources : The ability to limit searches to a subset of search results is achieved through Result Sources.
From the Result sources page, a SharePoint Online administrator can review existing Result sources or create
new ones. Result sources managed at this level can be used for all Site collections and sites in a SharePoint
Online tenant. A Site collection administrator or a site owner can manage result sources for a site collection or a
site, respectively.
Query Rules : Search results can be improved through Query rules that give a mechanism to specify conditions
and actions to promote the relevance of the search results that meet those conditions. As an example, we could
create a query rule that boost technology news in our Intranet that are tagged with a specific category so when
someone searches for that news category, all the news in that category are displayed on the top of search results.
Query Client Types : Gives a list of client types that can send a search query to the search engine. Included
client types are prioritized by tiers and the top tier has the highest priority in regards of query processing in
scenarios where query throttling is enabled. It is possible to add new query clients to the list.
Remove Search Results : Lists the URLs that are removed from search results.
Usage Reports : To evaluate how users are performing searches in SharePoint Online, a set of predefined
 reports is provided. Those reports provide insights about the number of queries performed by users, top search
queries, queries with no results, abandoned queries or query rules usage.
Search Center Settings : This page allows to indicate the URL of the Global Search Center in the SharePoint
Online tenant and the search results loading strategy to be used.
Export Search and Import Search Configuration : Those options provide a simple mechanism to export to a
text file any customization done in search configuration (query rules, result sources, result types, ranking models
and site search settings) or to import a search configuration file.
Craw Log Permissions: This page lists the users with read access to crawl log information in the tenant.

Changes done in the search administration page are applied to the whole tenant, but it’s also possible to customize
search on the site collection level and on the site level.

Secure Store
Secure Score service is designed to manage and store securely the credentials needed to connect to an external data
source. Credentials are just one of the settings of a Secure Store Target Application that also defines the mapping
between them and a group of users in SharePoint Online, who can use the credentials to access the external data
source. A Secure Store Target Application can be used by services such as Business Connectivity Services (BCS) to
connect to an external system and integrate data in SharePoint Online sites in the form of SharePoint external lists,
external data columns, default BCS Web Parts or a custom solution.

Apps
SharePoint Online can be extended by means of Apps or Add-ins . Two main types of Apps can be added to a
SharePoint site:

SharePoint Add-ins, defined by Microsoft as self-contained extensions that may include logic and data deployed in
the cloud, SharePoint components (such as Content types, Lists, Document libraries, etc.) and client-side scripts.
Web Parts and Extensions created with the latest SharePoint development model: the SharePoint Framework.
SPFx Web Parts can be added to both classic and modern pages. SPFx extensions can only be used in the modern
SharePoint experience.

SharePoint Add-ins can only be installed from the Office Store or the global App Catalog. A SharePoint Online tenant
can have only a global App Catalog, a special type of Site collection that needs to be created by a SharePoint Online
Administrator. SPFx Web Parts and Extensions can also be distributed from the global App Catalog or from a Site
collection catalog . To create a catalog in a specific site collection, connect to SharePoint Online with PowerShell and
run the following command:

[PS] C:\> Add-SPOSiteCollectionAppCatalog -Site $sSiteCollectionUrl

Web Parts and Extensions deployed to a Site collection catalog can only be installed in the root site and any site
created in the collection, and not in other Site collections.

Sharing
As the application name implies, sharing documents, folders, and sites with internal and external users is an
important SharePoint feature. Apart from email, SharePoint was the first major application to support sharing with
people outside an Office 365 tenant. Since the introduction of guest user access for Office 365 Groups, Teams, and
Planner (all of which make use of SharePoint sharing), Microsoft’s terminology can be confusing because of the way
SharePoint documentation refers to “external users.” For SharePoint, an external user is someone outside the tenant.
That user might have a guest user account in the tenant, created by an administrator or because they were invited to
an Office 365 Group or a Team. Previous SharePoint sharing mechanisms also created guest user accounts for
external users, a need reduced by the introduction of one-time time-sensitive access codes in January 2018. On the
other hand, an external user can also be an ad-hoc recipient of a sharing invitation, in which case they authenticate
with an access code, or they can receive an anonymous link, which allows anyone with the link to access content.
Generally, the rule is:

Use Office 365 Groups and Teams to control ongoing access for external users to content in modern SharePoint
team sites. External people who join Office 365 Groups and Teams automatically get a guest user account as part
of the invitation process (see Chapter 12) and use those accounts to access the document library belonging to the
team or group they join.
Use individual guest user accounts to control access for external users for a sustained period to content in
traditional SharePoint sites. You can create these accounts in the Users section of the Azure Active Directory
blade of the Azure portal. SharePoint creates guest accounts when someone shares a site with an external user.
Use ad-hoc codes for one-time access for external users.

Updates to secure external sharing: In June 2018, Microsoft announced that they had updated the external
sharing experience using one-time passcodes , with the effect that any recipient with an Office 365 account will be
added to the sharing tenant as a guest user once he/she enters the one time passcode sent for verification. Therefore,
the next time the user accesses any shared resource, he/she will authenticate using his/her Office 365 account.
Additionally, at Ignite 2018, Microsoft disclosed plans to integrate the one-time passcodes sign-in experience with the
Azure AD B2B collaboration platform meaning that external users will exist in Azure Active Directory as guest
accounts and can be managed like any other user in the corporate directory.
SharePoint Online supports up to 50,000 external shares per securable object (a site, a list / document library, or a(n)
item / document), which should be enough for even the most dedicated sharer. For performance reasons, Microsoft
recommends keeping the number of shares on an object to 5,000 or less. Somewhat confusingly, the settings that
control how users can share SharePoint content are found in three administrative consoles. The consoles are:

Office 365 Admin Center : Select Settings, then Services & Add-ins, and then Sites.
SharePoint Admin Center : Select Sharing.
OneDrive Admin Center : Select Sharing (we will discuss how to use this console later in this chapter).

The same basic settings for sharing controls are made available through any of these consoles and the same values
appear in each. However, you will find some differences in presentation and emphasis. For example, the OneDrive
Admin Center allows you to adjust the sharing behavior for user personal sites to be less permissive than general-
purpose SharePoint sites. On the other hand, the Office 365 Admin Center presents a somewhat more simplified view
of sharing controls than either of the other consoles. And of course, you can use PowerShell to adjust settings too.

Figure 8-4: External sharing controls for SharePoint Online

The Sharing section of the SharePoint Admin Center (Figure 8-4 ) exposes the settings to control how sharing occurs
within a tenant. SharePoint Online and OneDrive for Business support four sharing options, shown here in order from
restrictive to most permissive:

No external sharing . In other words, users can only share with other users within the same tenant.
Share with known external (guest) users : Users can share with external people, but only if the target users
have a guest account in the tenant directory. It is a good idea to create guest accounts when users need to share
information with known external people over a sustained period. Guest users use their credentials to access
content shared with them.
Share with authenticated external users : Users can share with external users who have an account in
another Office 365 tenant or a Microsoft Services Account (MSA). These users must authenticate themselves by
signing into their account before they can access content shared with them. When someone shares a file with an
authenticated external user, SharePoint sends email containing a link to the content. If that external user does
not exist in the tenant directory, they can use a one-time code issued by SharePoint to authenticate their access.
Share with anonymous users : This is the loosest permission as it allows users to share with anyone with an
email address. SharePoint sends an email with a link to allow the recipient access to the content, but anyone who
subsequently has access to the link can use it to access the associated content. If you allow anonymous sharing,
you can limit the lifetime of a link (to say, 7 days) and restrict access to view-only rather than view and edit.

If you do not change the default sharing setting, users can use anonymous sharing. Changing the sharing setting for a
tenant is a notoriously slow process and can take up to 24 hours before a change is effective. In addition to controlling
the kind of sharing you want within a tenant, you can limit the set of domains that users can share with (for instance,
the domains belonging to partners). You can enter a maximum of 120 domain names separated by spaces.

The sharing settings also allow you to control the type of link created when a user shares a document with someone
as well as the default permission (edit or view-only). Because users usually accept the default link type, you should set
the default permission to view-only. We can also restrict users so that they can only share files with authenticated
external users who belong to two specific domains. The sharing settings apply to all the SharePoint Online site
collections and OneDrive for Business sites in a tenant. You can also customize sharing for a site collection , but you
cannot make sharing more permissive at a collection level than it is for the tenant.

Links created for SharePoint and OneDrive For Business users can block recipients from downloading copies when an
Office document is shared anonymously or to all the organization. In this case, recipients can use Office Online to
view the content. To block download, a user edits the sharing link settings and toggles the “block download” setting
(Figure 8-5 ).

Figure 8-5: Block download feature for Office documents in SharePoint and OneDrive For Business

Customized Sharing Defaults per SharePoint Site Collection: Default link type and default link permission can
also be configured at the site collection level overriding the tenant level setting. This feature was made available at
the end of June 2018 and can be set through the classic SharePoint Admin Center or PowerShell.

Sharing from Group Document Libraries

Much of the sharing activity within Office 365 is now done by guests who are members of Office 365 Groups or Teams.
Guests who join the membership of a group have access to all the files in the document libraries belonging to the
group. Group members can also share individual documents with external users, including people outside your
organization who are not in your directory. Take the example in Figure 8-6 where a group member shares a document
with someone outside the tenant. Because the tenant allows limited external sharing, the user can email a sharing link
to invite an external person to access the file. In this instance, the tenant sharing capability setting does not allow
anonymous guest sharing, so the user can only share files with tenant users, guest users who have accounts in the
tenant directory, and users who can authenticate themselves with one-time codes.
Figure 8-6: The sharing capability for the site controls what users can do

When an Office 365 group or a team is created, Office 365 provisions a site collection to hold the document library
and other resources. The tenant sharing settings control what sharing group users can do for files in the document
library, but a group owner or tenant administrator can make the sharing more restrictive by changing the
SharingCapability setting for the group’s site collection. The values that you can set are equivalent to the four settings
available for the SharePoint tenant as explained above. Again, they are from most restrictive to most permissive are
listed below.

Disabled : External sharing is not possible.

ExistingExternalUserSharingOnly : Sharing is only possible with external people who already have a guest
account in the tenant’s Azure Active Directory.
ExternalUserSharingOnly : Users can share documents freely with external people with guest accounts and
can invite authenticated external users to share documents using one-off time-limited codes. This is the default
setting applied to site collections created for new Office 365 Groups, if a tenant supports guest access.
ExternalUserAndGuestSharing : Users share documents freely with guest users and authenticated external
users and can also create links that can be used by anyone who can access the link (also called anonymous or
guest link sharing). Be careful about using this sharing capability because it means that any group member can
share documents from the document library with anyone else by emailing a sharing link to that person. While
acceptable for libraries that do not hold confidential material, this setting should never be present for libraries
that hold anything other than documents intended for public access.

The tenant sharing setting prevails over the setting for a site collection. If you want to use a setting like
ExternalUserAndGuestSharing for the site belonging to a group, you must first make sure that the organization allows
anonymous sharing. To check the tenant setting, use the SharePoint Admin Center or connect to SharePoint Online
with PowerShell and run this command:

[PS] C:\> Get-SPOTenant | Select SharingCapability

SharingCapability : ExistingExternalUserSharingOnly

To check the sharing capability for an Office 365 Group, run the command (A connection to both SharePoint Online
and Exchange Online is required):

[PS] C:\> Get-SPOSite -Identity (Get-UnifiedGroup -Identity GroupName).SharePointSiteUrl | Select SharingCapability

SharingCapability : ExistingExternalUserSharingOnly
In this instance, the sharing capability for the site matches that of the tenant and is appropriate to support guest
access to the site’s document library through Office 365 Groups or Teams.

New Manage access experience in SharePoint and OneDrive For Business: In October 2018, Microsoft started
to roll out a new Manage Access experience to make easier to check how a file or a folder is shared with specific
people, add and remove individual users from links defined for specific people, and to modify the permission applied
on each link.

Sharing Capability for Newly-Created Office 365 Groups

The sharing capability for new Groups sites depends on whether the tenant allows Office 365 Groups to have guest
members. Until August 2017, the default setting was ExistingExternalUserSharingOnly . Based on customer feedback
and to make sharing work in a consistent manner for sites when tenants allow guests to join groups, Microsoft
decided to make ExternalUserSharingOnly the default for new Groups sites. If a tenant blocks external access, Groups
sets the sharing capability for new sites to ExistingExternalUserSharingOnly . In these cases, it is likely that tenant-
wide sharing controls are in place that override the sharing capability set for individual site collections.

To update older Groups sites, we can use the Set-SPOSite cmdlet to update their SharingCapability to
ExternalUserSharingOnly. After you connect to SharePoint Online with PowerShell, this code looks for sites with
sharing capability set to ExistingExternalUserSharingOnly and then loop through the set of objects to update each
site:

[PS] C:\> $Sites = (Get-SPOSite -Template GROUP#0 -IncludePersonalSite:$False | ?

{$_.SharingCapability -eq "ExistingExternalUserSharingOnly"})

If ($Sites.Count -gt 0)

Write-Host "Updating" $Sites.Count "sites..."

ForEach ($S in $Sites)

Write-Host "Updating sharing capability for" $S.Url

Set-SPOSite -Identity $S.Url -SharingCapability ExternalUserSharingOnly -Nowait

ElseIf ($Sites.Count -eq 0)

{ Write-Host "No Sites to upgrade"}

As an alternative to PowerShell, the SharingCapability for older Groups sites can also be changed through the
“Sharing” option available when a Group site is selected in the new SharePoint Admin Center.

A change to a site’s sharing settings is effective at once. However, it can take ten to twenty minutes before users pick
up the effect of the change and are able to share with external users. Remember that the sharing mechanism relies on
SharePoint Online and that the sharing restrictions implemented for the tenant might override the settings for an
individual site. For instance, the tenant might restrict sharing to a certain set of domains.

Tracking Document Sharing Through Audit Records : Each time someone shares a document or folder with
someone else, SharePoint records the event in an Office 365 audit record. You can search the Office 365 audit log to
retrieve and analyze these records to understand what sharing occurs within a tenant. See Chapter 21 for details.

SharePoint Settings
This section holds other global SharePoint Online settings that are not included in the other sections of the SharePoint
Admin Center. To mention some of them, from here we can change how site collection storage is managed (Automatic
vs Manual), user experience to be used for OneDrive For Business and for lists and document libraries (New
experience vs classic one) or show or suppress the option to create new sites from the SharePoint Online home page.

Hybrid
SharePoint Online and SharePoint On-Premises can be integrated and connected in a hybrid deployment. Assuming
the pre-requisites required for a hybrid deployment are met, the list of hybrid features currently available is the
following:

OneDrive redirection : If enabled, on-premises users are redirected to OneDrive For Business in Office 365.
Hybrid search : provides the ability to combine on-premises and cloud search results in the same search index
in Office 365 so it’s possible to search content from both sources in Office 365 or in the on-premises farm. On the
other hand, hybrid federated search allows to search content from on-premises and Office 365 index in a single
search center.
Hybrid App Launcher : This feature allows to add access to Office 365 services such as Delve or Office 365
Video in the on-premises App launcher.
Business to Business (B2B) Extranets : By using this feature, it is possible to create extranet in SharePoint
Online to collaborate with partners, so they don’t have access to any on-premises data.
Hybrid Auditing : Through this feature (still in preview), it’s possible to upload diagnostic and usage logs to
Office 365 so an Office 365 Administrator can also generate on-premises reports in the Compliance and security
center.
Hybrid Taxonomy : Delivers a mechanism to replicate taxonomies created in SharePoint Online to SharePoint
on-premises. In a nutshell, hybrid taxonomy allows an organization to have a shared taxonomy between
SharePoint Online and SharePoint on-premises.
Hybrid Self-Service site creation : If enabled, any user that creates a new site is redirected to the self-service
site creation page in SharePoint Online.

Access Control
Access to SharePoint Online and OneDrive For Business can be controlled in two different ways:

Creating conditional access policies, which require an Azure AD Premium Subscription . Among other settings,
these policies can exert control over access from devices that are not compliant or joined to a domain
(unmanaged devices). By default, SharePoint Online and OneDrive contents can be fully accessed from
unmanaged devices but it’s possible to configure limited access or even block access through the conditional
access policies created in Azure AD. You can also apply conditional access policies at the Site collection level as
explained in this blog post .
Based on a network location, meaning that only devices connecting from specific IP address locations can access
SharePoint and OneDrive contents.

Forcing timeouts for idle SharePoint Online and OneDrive For Business sessions : For added protection, you
can create a policy to force SharePoint Online and OneDrive For Business to automatically sign a user out of a
session after a period of inactivity. When the policy is enforced, any user accessing SharePoint and OneDrive is first
warned and then subsequently signed out after a period of inactivity. To configure session timeouts , a SharePoint
Admin can run the Set-SPOBrowserIdleSignOut cmdlet. For example, this command enables session timeout and sets
a warning period of 45 minutes (2700 seconds) and a forced sign-out after an hour (3600 seconds). The timeout
values apply to both SharePoint Online and OneDrive for Business sessions.

[PS] C:\> Set-SPOBrowserIdleSignOut -Enabled $True -WarnAfter (New-TimeSpan -Seconds 2700)

-SignOutAfter (New-TimeSpan -Seconds 3600)

Data Migration
This section simply gives access to the SharePoint Migration Tool download page.

Some legacy configurations: InfoPath Forms Services and Business Connectivity Services
InfoPath Forms Services and BCS Services are two legacy services still supported by SharePoint Online that are
managed through the SharePoint Online Administration Center. InfoPath enables tenants to easily design electronic
forms that can be deployed to any SharePoint site in the tenant using InfoPath Designer 2013 as an authoring tool.
The later provides a simple mechanism to connect a SharePoint site to external business data sources such as SQL
Azure Databases or any other one exposed by a Windows Communication Foundation (WCF) service.

While it’s true that InfoPath is a technology used mainly on-premises, it’s also true that it can also be used in
SharePoint Online by simply enabling the checks available in the User Browser-enabled Form Templates section in
the InfoPath configuration section of the SharePoint Online Admin Center. Microsoft has committed to support
InfoPath Forms Services 2013 and InfoPath 2013 clients until 2026 and has positioned PowerApps as its natural
replacement. The general recommendation about InfoPath usage in SharePoint Online is to avoid the creation of new
InfoPath Forms and carefully plan the migration of existing ones to PowerApps or custom solutions built using any of
the frameworks and platforms provided by Microsoft (including SPFx).

BCS supports the integration of data stored in external sources in SharePoint sites through of SharePoint external
lists, BCS Web Parts or external data columns can be added to existing lists and/or document libraries. To make this
integration possible, BCS settings in the SharePoint Online Admin Center include these options:

Create data connections to both online and on-premises services by giving the address of the service and the
authentication mechanism.
Manage BDC Models and External Content Types, including operations such as import new models, export
existing ones, configure External Content Types actions, etc. A BDC Model is just an XML file containing all the
 information required by SharePoint Online to integrate an external data source in a site: data source connection
details including the authentication mechanism, external content types (entities), operation with entities among
others. An External Content Type is a collection of metadata that models a data entity present in the data source
to be integrated.

While Microsoft has not made an official statement about the future of BCS in SharePoint, they have not made any
kind of investment in BCS since the release of SharePoint 2013. In SharePoint Online, the integration of external data
in SharePoint sites can be achieved by using a combination of modern cloud technologies and platforms such as
PowerApps, Flow, Azure Logic Apps or cloud development patterns.

Using PowerShell to manage SharePoint Online

Any Office 365 Global or SharePoint admin can use the SharePoint Online Management Shell to manage users, sites,
and site collections. The SharePoint Online Management Shell is a Windows PowerShell module that lets run
command-line operations and PowerShell scripts. It allows to perform batch (script) operations and is the only way to
achieve some management tasks in SharePoint and OneDrive.

There are some things to take in consideration before starting to use PowerShell for SharePoint Online. Because the
module is designed to run administrative tasks, it is imperative to run PowerShell as an administrator: right-click the
PowerShell icon and select Run as Administrator. The account used should be a Global or Office 365 administrator,
and you should use PowerShell 3.0 or later to be able to automatically import new modules and use the cmdlets
included. To check the version of PowerShell version installed on your workstation, check the variable
$PSVersionTable.PSVersion . For example:

[PS] C:\> PS C:\temp> $PSVersionTable.PSVersion

Major Minor Build Revision

----- ----- ----- --------

5 1 17134 407

Also, you might want to run remote signed scripts that you download from Microsoft or other trusted sites; to see the
current execution policy, type Get-ExecutionPolicy in the command line: if you see that the policy setting is Restricted
, which means that only scripts created locally can be run. To change the setting to allow the execution of remote
signed scripts, run

[PS] C:\> Set-ExecutionPolicy RemoteSigned

You can also change the execution policy to Unrestricted, which allows you to run any script no matter where it comes
from. This can be dangerous, as you might then run a script with some malicious code with unpredictable
consequences.

To begin managing SharePoint Online with the standard PowerShell cmdlets, download and install the latest
SharePoint Online Management Shell ( https://www.microsoft.com/en-us/download/details.aspx?id=35588 ). If a
previous version is installed, uninstall it first and then install the latest version.

To connect to SharePoint Online use the cmdlet Connect-SPOService as explained below (change "TenantName" to
reflect your value):

[PS] C:\> $UserCredential = Get-Credential

[PS] C:\> $AdminCenterUrl="https://TennatName-admin.sharepoint.com"

[PS] C:\> Connect-SPOService -URL $AdminCenterUrl -Credential $UserCredential

To check that you have a valid connection, run the Get-SPOSite cmdlet to return a list of all the Site Collections
available in the tenant. The Limit parameter specifies that all sites are to be returned.

[PS] C:\> Get-SPOSite -Limit All

To see all the cmdlets provided in the SharePoint Online Management Shell, run

[PS] C:\> Get-Command -Module "Microsoft.Online.Sharepoint.PowerShell"

The November 2018 release of the SharePoint Online Management Shell includes 151 cmdlets.

SharePoint Online Management Shell updates : Microsoft updates the SharePoint Online Management Shell
almost every month to keep it aligned with the release cadence of the SharePoint Client-Side Object Model (CSOM)
API libraries. Since August 2018, the latest version of the SharePoint Online Management Shell is available in the
PowerShell Gallery , so to fetch the latest update, all you need to do is to run the command:
[PS] C:\> Install-Module -Name Microsoft.Online.SharePoint.PowerShell

To update the SharePoint Online Management Shell from the PowerShell Gallery, run the command:

[PS] C:\> UpdateModule -Name Microsoft.Online.SharePoint.PowerShell.

The functionality available through the SharePoint Online PowerShell module is limited and restricted to the most
basic administration tasks of a SharePoint Online administrator, such as managing Site Collections, site groups, and
users. To get extra functionality, use the cmdlets available in the SharePoint PnP PowerShell cmdlets project in
GitHub , part of the Patterns & Practices community initiative. To go further and be able to access all the aspects of
SharePoint, you will need to use the CSOM API in your PowerShell scripts.

In addition to managing SharePoint, PowerShell can be used to work with Office 365 Groups, modern sites, and
modern pages. For example, if you want to include a group document library in an eDiscovery case or content search
(in the Security and Compliance Center), you need to specify the URL to the site. At first sight, the URL displayed in a
browser when accessing a group’s SharePoint site seems complex, something like:

https:/office365itpros.sharepoint.com/sites/O365ExchPro/Shared%20Documents/Forms/DocViewModified.aspx

Stripping off the commands, you end up with a URL like the one shown below where the site alias is the last part of
the name:

https://office365itpros.sharepoint.com/sites/O365ITPros

This is the URL used to find the SharePoint Online site for the Group. If you use PowerShell to examine the properties
of a Group, you will see three SharePoint Online URLs returned for the site, the document library, and the shared
OneNote notebook. The value returned in SharePointSiteUr l property is the one needed when you wish to add a site
to content searches, found using the Get-UnifiedGroup PowerShell cmdlet:

[PS] C:\> [PS] C:\> Get-UnifiedGroup –Identity "Office 365 for IT Pros" | Format-List Share*Url

SharePointSiteUrl : https://Office365ITPros.sharepoint.com/sites/O365ITPros

SharePointDocumentsUrl : https://Office365ITPros.sharepoint.com/sites/O365ITPros/Shared Documents

SharePointNotebookUrl : https://Office365ITPros.sharepoint.com/sites/O365ITPros/SiteAssets/Office 365 for IT Pros Notebook

For Modern Pages, SharePoint Online displays usage information (the number of views) and allows users to like and
enter comments for pages. It is possible to disable this feature via PowerShell. At the tenant level it can be done by
running the following command:

[PS] C:\> Set-SPOTenant -SocialBarOnSitePagesDisabled $True

Alternatively, you can disable the feature for specific sites by running the Set-SPOSite cmdlet and passing the URL for
the site. For example:

[PS] C:\>Set-SPOSite -Identity https://Office365itpros.sharepoint.com/Projects/ -SocialBarOnSitePagesDisabled $True

PowerShell and SharePoint Sharing

As discussed earlier, you can limit the ability of users to share content with others in different ways. The SharePoint
PowerShell module has several cmdlets that work with sharing settings. The most basic example is how to limit the
type of sharing for a tenant using the Set-SPOTenant cmdlet. In this example, we allow sharing with authenticated
external users. To disable external sharing, set the value to Disabled , or use ExternalUserAndGuestSharing if you
want to allow the generation of anonymous guest links (or to allow Excel Surveys to be used in team sites).

[PS] C:\> Set-SPOTenant –SharingCapability ExternalUserSharingOnly

In the next example, we create a list of external sites that we allow users to share content with and add an extra level
of security for sharing links by setting RequireAcceptingAccountMatchInvitedAccount to $True to ensure that only the
account (email address) that receives a sharing invitation can redeem it to access the shared content. If you leave this
setting at the default ($False ), then anyone who has a link can use it to access the information.

[PS] C:\> Set-SPOTenant –SharingAllowedDomainList "locklan.com.au Microsoft.com"

–SharingDomainRestrictionMode AllowList –RequireAcceptingAccountMatchInvitedAccount $True

To revert to the default configuration, reverse the settings:

[PS] C:\> Set-SPOTenant –SharingAllowedDomainList $Null –SharingDomainRestrictionMode None

–RequireAcceptingAccountMatchInvitedAccount $False

SharePoint Online supports anonymous guest links that site owners (or users with full control permission for a site)
can send to external people to allow them to access content. The trouble with guest links is that anyone who has the
link can use it, so if you decide to allow these links, it is best to force the links to expire after a period (expressed in
days). As shown below, a tenant administrator can run the Set-SPOTenant cmdlet to assign a default expiry period in
the RequireAnonymousLinksExpireInDays setting. Users can send guest links that expire sooner than the default, but
they cannot exceed it. This setting applies to links sent from both SharePoint Online and OneDrive for Business.

[PS] C:\> Set-SPOTenant –RequireAnonymousLinksExpireInDays 10

Sharing with Office 365 Groups or Teams

Every Office 365 Group has a SharePoint site with a document library. This applies whether the group supports
conversations through Outlook, Yammer, or Teams. SharePoint sets the sharing capability for the site collections
belonging to new groups to ExternalUserSharingOnly, which is a level of access enough to allow guest accounts to
work with content in the document libraries. See Chapter 11 for more information on this point.

Because on-premises versions of SharePoint Online and OneDrive for Business are also available, other sites might
have a different sharing model to that used by Office 365 Groups. If you want to align the two sharing models, you can
restrict sharing to the external users who already have guest accounts in your tenant directory. When this happens,
administrators must create guest accounts in the directory before users can invite the holders of those accounts to
access their documents. You should also allow SharePoint users to search for guest accounts in the people picker used
to select with whom to share a document. By default, guests do not show up in the people picker, so to enable them to
appear, you need to run the following cmdlet:

[PS] C:\> Set-SPOTenant -ShowPeoplePickerSuggestionsForGuestUsers $True

Not all team sites are linked to Office 365 Groups as some are still of the classic variety to serve specific purposes. To
list all the team-enabled sites in a tenant, use this cmdlet, which includes the template for group-enabled sites to
isolate the set we want to see:

[PS] C:\> Get-SPOSite -Template "GROUP#0" -IncludePersonalSite:$False

Taking this a little further, here is how to loop down through the set of group-enabled team sites and report some
information about the sharing activity for each site.

$SiteNumber = 0

ForEach ($Site in Get-SPOSite -Template GROUP#0 -IncludePersonalSite:$False)

$SiteNumber++

Write-Host "Site number: " $SiteNumber " " $Site.Url

Write-Host "Owned by: " $Site.Owner

Write-Host "Sharing: " $Site.SharingCapability

Write-Host "--------------------------------------------------------------"

Get-SPOExternalUser -SiteUrl $Site.Url

Guest Accounts No Longer Needed For One-Off Sharing : In late 2017, Microsoft introduced a new sharing
model for SharePoint Online and OneDrive for Business based on using security codes to validate recipients of
sharing invitations. Security codes are one-time eight-digit numbers sent to an email address contained in a sharing
invitation to allow them to open content. Codes are good for 15 minutes. Using security codes removes the need to
create guest accounts in the tenant directory, which is a good thing if you only share a single document with a user.
You can still restrict external sharing on a tenant-wide or site-specific basis so that users can share files only with
guest accounts, which remain the best solution when people need to share content with external users on an ongoing
basis. In this case, security codes are not used as guest credentials are used to access content.

Tracking Shared Files

The Set-SPOTenant cmdlet also controls a setting often used by administrators to keep track of files shared with
external users. The example shown below sets the BccExternalSharingInvitations setting to $True to force SharePoint
Online to generate a BCC message to a nominated list of email addresses (in a comma-separated list with no spaces)
each time a user shares a file stored in a SharePoint Online or OneDrive for Business document library with an
external person. The users specified in the BccExternalSharingInvitationsList parameter will receive a message
saying that the user wishes to share a file with the addressee of the message.

[PS] C:\> Set-SPOTenant –BccExternalSharingInvitations $True –BccExternalSharingInvitationsList

administrator@Office365ITPros.com

OneDrive for Business Notifications : SharePoint Online controls many settings for OneDrive for Business, among
which are notifications for events connected with sharing files. The Sharing settings available in the SharePoint
Admin Center allows you to opt for users to receive notifications via email when:

Users invite external users to access shared files.
External users accept invitations and open shared files.
Users create or update an anonymous link used to share files.

Clearly, there is lots more that you could do to manage SharePoint Online with PowerShell and these examples just
scratch the surface. Many script examples on the Internet, including in the TechNet script gallery, are available to
give guidance and help in getting to know the cmdlets and how they work.

SharePoint Online Extensibility Options

One of the traditional strongest characteristics of SharePoint has been always its natural extensibility: an organization
can change the system to meet business requirements in several ways. Unfortunately, the introduction SharePoint
Online initially featured a strong reduction of the extensibility options when compared to SharePoint on-premises.
This was due to the intrinsic multitenant design of the platform and because Microsoft wanted to reduce the fragility
of custom development.

SharePoint Online inherited from its on-premises counterpart the Add-ins model as a recommended customization
mechanism. This model supports the creation of integrated applications in the platform while running totally outside
the main structure. Furthermore, it was (and still it is) impossible to install and run server-side code in SharePoint
Online, something that removes the possibility to create traditional SharePoint artifacts such as Web Parts, Timer
Jobs, Event Handlers, etc. Slowly, as the SharePoint Online platform matures and evolves, substitutes for older
integration mechanisms have been added to SharePoint Online through the natural integration with other platforms
(Azure), the work done by the SharePoint Team with SPFx and specific SharePoint community initiatives such as the
PnP project.

SharePoint Add-ins and the App Catalog

SharePoint Add-ins are self-contained pieces of functionality that extend the capabilities of SharePoint to solve
business problems. Add-ins don't have custom code that runs within SharePoint Online: all custom logic moves to
servers outside the SharePoint platform. Keeping custom code out of SharePoint guarantees that the Add-in can't
harm SharePoint or reduce the performance of the system.

The App Catalog makes internally developed custom Apps available for users to install when they look through apps
under the "From Your Organization" filter on the Site Contents page. Site owners can add these apps to customize
sites with specific functionality or to display information. After an App Catalog site has been created, it is possible to
upload any custom App that the organization has developed by copying the manifest document to a document library
and setting some properties. The App Catalog site also supports the management of app requests from users,
checking how Add-ins are used, and the maintenance of license information.

The App Catalog is not created by default in a new tenant but can be added from the SharePoint Admin Center. Select
the apps link in the left pane, and then select App Catalog. This creates a Global App Catalog, meaning that every
Add-in uploaded to the App Catalog will be available in all Site Collections in the tenant. But tenant administrators
can choose to enable "Site Collection App Catalogs" on specific Site Collections so that solutions deployed to the Site
Collection App Catalogs can only be installed in that specific Site Collection.

Site Collection App Catalogs are configured and managed using the SharePoint Online Management Shell and the
Global App Catalog must exist already. Use the Add-SPOSiteCollectionAppCatalog cmdlet to create a Site Collection
App Catalog, indicating the Site Collection where the app catalog should be created in the -Site parameter. An "Apps
for SharePoint" document library will be added to the Site Collection to deploy SharePoint Add-ins and SPFx solutions.
To disable the Catalog, use the Remove-SPOSiteCollectionAppCatalog cmdlet indicating the Site Collection in the -Site
parameter; this prevents new components from being added and any previously installed components from executing
their code. At present, it is not possible to list all the collections in the tenant that have the Site Collection App
Catalog enabled. In addition, be aware that although solutions installed in Site Collection App Catalogs can only be
used in these specific Site Collections, they can theoretically use resources from other sites in the tenant.

SharePoint Framework (SPFx)

SPFx is a Page and Web Part development model that empowers client-side development, integration with SharePoint
Online and on-premises and integration with the Microsoft Graph. It is based on open source tooling and JavaScript
technologies. SPFx is used in SharePoint as the extensibility paradigm for the client-side development framework to
allow developers to implement new functionality in SharePoint.
SPFx is used extensively by Microsoft to build the modern user experiences seen in SharePoint Online, but external
developers can use the same technology, tools and techniques to build more productive experiences and apps that are
responsive and mobile-ready.

From the infrastructure point of view, the files required for SPFx are just JavaScript archives plus the accompanying
style sheets, graphics and other required files that should be deployed to an accessible place for the client computers.
This can be implemented using the SharePoint Content Delivery Network (CDN), a SharePoint document library, the
Azure CDN, or any other available private or public CDN. Additionally, a manifest file must be deployed to the
SharePoint Catalog, so that SharePoint is aware of the existence of the components. Normally a distribution package
is created by the developers and delivered to the administrators that install them in the SharePoint Catalog, in a
similar way as for SharePoint Add-ins.

SharePoint Content Delivery Network (CDN)

Static assets in Office 365 can be hosted in the SharePoint Content Delivery Network (CDN) to offer better
performance for SharePoint Online. Static assets are files that are not altered very often, like images, video and audio,
style sheets, fonts, and JavaScript files. The CDN is therefore an ideal place to locate SPFx and Add-in resources. The
CDN works as a geographically distributed caching proxy, by caching static assets closer to the browsers demanding
them.

The SharePoint CDN is part of any SharePoint Online subscription and you don't have to pay extra for it. SharePoint
CDN provides support for private and public access and allows tenants to host static assets in multiple locations. To
enable the SharePoint CDN in a tenant use the Set-SPOTenantCdnEnabled cmdlet with the -CdnType parameter
(which can be "Public", "Private" or "Both") and the -Enable ("$true" or "$false") parameter. There are other cmdlets
to work with the SharePoint CDN, such as Get-SPOTenantCdnEnabled to enumerate the configured CDNs, Add/Get-
SPOTenantCdnOrigins to manage the origin of the CDN (the origin is a URL that points to a SharePoint document
library or folder that contains the assets that are wanted to be hosted by the CDN), and Set-SPOTenantCdnPolicy to
include extra file types in the CDN.

SharePoint Patterns and Practices (PnP)

The SharePoint Patterns and Practices (PnP) community initiative comprises several components aimed at enhancing
the functionality of SharePoint on-premises and Online in several ways, filling the development and infrastructure
gaps that have not been addressed by Microsoft. The components include:

PnP PowerShell cmdlets ( http://aka.ms/officedevpnppowershell ).

A complete PnP Provisioning Engine ( https://github.com/OfficeDev/PnP-provisioning-schema ).
PnP tools ( https://github.com/OfficeDev/PnP-Tools ).
PnP guidance ( http://aka.ms/OfficeDevPnPGuidance ).
Ready to use solutions such as the PnP Starter Kit ( https://github.com/SharePoint/sp-starter-kit ).

Using PnP reduces the development effort needed to work with SharePoint. For example, to create a new list or
document library using PowerShell requires the use of the SharePoint CSOM API (create a context, recall the web and
site objects, create a new List object, execute the operation, etc.), as well as some knowledge about programming and
the internal structure of the service. With PnP PowerShell cmdlets, you only need to connect to a SharePoint Online
tenant and use New-PnPList as follows:

[PS] C:\> Connect-PnPOnline -Url "https://tenantname.sharepoint.com/sites/namesitecollection"

[PS] C:\> New-PnPList -Title "PnPList" -Template GenericList -Url "Lists/PnPList"

Connect-PnPOnline cmdlet needs user credentials to make the initial connection to the tenant. Then a new list called
“PnPList” is created using New-PnPList cmdlet and the parameter -Template that specifies the list template to be
used. Be aware that the connection is only valid for the given URL endpoint; if you want to use other site collection,
you need to make a new connection.

In a similar way, there are several specific cmdlets that don't exist in the default cmdlets provided by Microsoft, and
that are tough to create using a mixture of PowerShell and SharePoint CSOM API. For example, to measure the
performance of an endpoint, getting statistics on response time by sending probe requests, we can use Measure-
PnPResponseTime cmdlet:

[PS] C:\> Connect-PnPOnline -Url "https://Office365TenantName.sharepoint.com/sites/namesitecollection"

Measure-PnPResponseTime -Count 100 -Timeout 20

The parameters indicate that the measure will be made using 100 requests with 20 ms between each request. The
output is like this:

Mode : RoundTrip

Average : 630,84

Max : 13562

Min : 243

StandardDeviation : 1314,61
TruncatedAverage : 482,725

Histogram : {[2712, 99], [5424, 0], [8137, 0], [10849, 0]...}

Count : 100

Provisioning is probably the most commonly used and most powerful PnP characteristic. The recommended way to
provision in SharePoint Online is to create a new "standard" object in SharePoint, and then apply all the
customizations using scripting (thus not creating new templates as traditionally used). The PnP provisioning engine is
powerful enough to create and modify almost any element in a Site Collection: creation of subsites, Lists, Libraries,
Custom Fields, Content Types, views, attach users, etc. There are several cmdlets and ways to provision them using
PnP:

Make a template (just a xml file) that contains all the elements to be created, deleted or modified, and run the
Apply-PnPProvisioningTemplate cmdlet pointing to the xml file and URL endpoint. The creation of fully
customized SharePoint objects using this way is possible (and recommended).
Create manually all the customization required in a dummy Site Collection and generate the xml template using
the Get-PnPProvisioningTemplate cmdlet. Then, the template can be used to provision as many clones as
necessary using Apply-PnPProvisioningTemplate.
PnP Provisioning Engine and XML templates can easily be used in .NET code, so the same process described to
get and apply a provisioning template with PowerShell can be done programmatically.
There are also specific PnP cmdlets to manage some provisioning objects. For example, to create a new view for
the early created List, use:

[PS] C:\> Connect-PnPOnline -Url "https://tenantname.sharepoint.com/sites/namesitecollection"

[PS] C:\> $myList = Get-PnPList -Identity "Lists/PnPList"

[PS] C:\> Add-PnPView -List $myList -Title "myView" -SetAsDefault -Fields ID,Title,Created

Site Scripts and Site designs

Site Scripts and Site designs is a extensibility mechanism for SharePoint Online that allows administrators to create,
deploy and apply templates to modern and classic sites based on a specific site template . Site Scripts and site
Designs are supported for the following site types:

Modern Team Sites linked to an Office 365 Group.

Modern Team Sites not linked to an Office 365 Group (STS#3).
Communication sites.
Classic Team Sites (STS#0).
Classic Publishing Sites (BLANKINTRANET#0).

Site Designs and Hub Sites : Starting in August 2018, Microsoft released an update to SharePoint Online to allow
site designs applied to the Hub Site to apply to the associated sites too. This option is an optional setting for the hub
site and requires the site design and site script to already be published in the tenant.

Site designs are always based on an out-of-the-box template, and are deployed through PowerShell as follows:

[PS] C:\> Add-SPOSiteDesign -Title "HR Site" -WebTemplate "64" -SiteScripts "<ID>" -Description "HR Department Site"

Site designs always reference one or more site scripts (by their identifiers) to apply the customizations defined by the
script. WebTemplate 64 is the identifier for the Modern Team sites while 68 is for Communication sites.

A site script is a JSON archive defining the customizations to be added to a new site (where the site design is the
actual template to define the site). Site scripts are uploaded to a gallery at the tenant level, making them available for
all sites. Site scripts are deployed using a PowerShell cmdlet, which returns the ID to be used by Add-SPOSiteDesign :

[PS] C:\> Add-SPOSiteScript -Title "Create HR lists" -Content $mySiteScript -Description "Creates lists for HR site"

In the above example, the $mySiteScript variable contains the JSON string with the actions to implement. Currently,
the actions are: set regional settings, add users to SharePoint Groups, manage guest access, creating a new list, apply
a theme, set a site logo, add / remove navigation, create and add site columns, content types, fields and views, install
Add-ins or SPFx solutions, register a SPFx extension, trigger a Microsoft flow, join a Hub site, or invoke cmdlets to
apply a site design to an existing site.

Site Design JSON Schema : Microsoft updates the JSON Schema regularly. The schema defines all the actions and
sub-actions that can be defined in a site design. To simplify the creation of a Site Design, there is a free online tool to
visually define the JSON structure for a specific Site Design.

A site design can also be applied to an existing site using the Invoke-SPOSiteDesign or Add-SPOSiteDesignTask
cmdlets. Both cmdlets can add a site design to any target site collection but each has their own approach.
 Invoke-SPOSiteDesign applies the site design to the site immediately.
Add-SPOSiteDesignTask puts the site design invocation into a schedule to run and it also allows to overcome limit
on the number of actions that can be defined on a site design.

One of the latest additions to Site Scripts and Site Designs disclosed at Microsoft Ignite 2018 conference is the ability
to generate lists and document library site scripts from existing configurations so existing lists and document libraries
can be easily ported to another site. The cmdlet Get-SPOSiteScriptFromList allows to autogenerate a Site Script from
a list.

Discover which site designs have been applied to a site : The Get-SPOSiteDesignRun cmdlet shows all the site
designs that have been applied to an existing site. The Get-SPOSiteDesignRunStatus cmdlet returns the result of
each action from every site script in a site design.

Configuring Site Regional Settings with Site Scripts

Since April 2018 , Site Scripts can set default regional settings for new sites. This is important if you want to ensure
that sites display the correct document creation and modification times in local format when accessed through a
browser. By default, SharePoint Online sets the default time zone to Pacific Daylight Time (UTC -8), which is fine for
people in Redmond but not so good for users elsewhere in the world. You can always update these settings by
accessing the Regional Settings of a site. However, given that Teams and Office 365 Groups create many of the new
SharePoint Online sites and their owners are usually people with little knowledge of SharePoint internals, it is best if
the settings are correct from the start.

An example of how to configure regional settings is available in GitHub. The steps are straightforward. First, create a
JSON-formatted input file holding the settings that you want to use. In this example, I select GMT as the default time
zone, that we use the 24-hour format instead of 12-hour format, and that Ireland is the default country (locale id or
LCID). Examples of other locale ids are 3082 (Spain), 5129 (New Zealand), and 1035 (Finland). A full list of local ids is
available online.

"$schema": "schema.json",

"actions": [

"verb": "setRegionalSettings",

"timeZone": 2, /* Greenwich Mean Time */

"locale": 6153, /* Ireland */

"sortOrder": 25, /* Default */

"hourFormat": "24"

],

"bindata": { },

"version": 1

Save the JSON data to a file. Then connect to SharePoint Online with PowerShell and execute the following command
to retrieve the settings from the clipboard and load them into a variable ($SiteScript ). Make sure you install the latest
version of the SharePoint Online PowerShell module before trying to use the site design cmdlets .

[PS] $SiteScript = (Get-Content "C:\Temp\RegionalSettings.json" -Raw | Add-SPOSiteScript -Title "Set Ireland Regional Values" -
Description "Sets locale, time zone, and hour setting for Ireland")

Now run the Add-SPOSiteDesign cmdlet to load the settings as default regional values.

[PS] C:\> Add-SPOSiteDesign -Title "Set Ireland Regional Values" -WebTemplate "64" -SiteScripts $SiteScript -Description "Applies
Ireland regional settings" -IsDefault

To test that the change is effective, create a new team or group and examine the regional settings for the site that
Office 365 provisions. Open the site with SharePoint, select Site Contents, then Site Settings, and then Regional
settings. The locale and time zone settings should be the values set in the script.
If you make a mistake, you can update the site design with the Set-SPOSiteDesign cmdlet or remove the site design
and start over. Run the Get-SPOSiteDesign cmdlet to return a list of site designs in the tenant and note the identifier
(Id) for the design you want to remove. Then run the Remove-SPOSiteDesign cmdlet to remove it.

[PS] C:\> Get-SPOSiteDesign

Id : f466a1da-e2a2-4107-b558-35954ad199de

Title : Default

WebTemplate : 64

SiteScriptIds : {9e287dd9-b2b1-4ceb-8649-998f43b24c1d}

Description : Applies Ireland regional settings

PreviewImageUrl :

PreviewImageAltText :

IsDefault : True

Version : 1

[PS] C:\> Remove-SPOSiteDesign -Identity f466a1da-e2a2-4107-b558-35954ad199de

Setting default regional settings for new sites with a site script does not update regional settings for existing sites. If
you want those sites to use the correct time zone and locale, you can either update the settings manually or use a
script. Here’s an example that looks for Office 365 Groups that are provisioned for SharePoint and then applies a site
design (identified with a GUID) to each site.

[PS] C:\> $SitesUpdated = 0

$DesignID = "11c2efae-d0eb-4eb9-af71-4f4f5c5c6db8"

$Groups = (Get-UnifiedGroup | ? {$_.SharePointSiteUrl -ne $Null} | Select SharePointSiteUrl, DisplayName, Alias)

ForEach ($G in $Groups) {

Try {

Write-Host "Processing" $G.SharePointSiteUrl "for group" $G.DisplayName

Invoke-SPOSiteDesign -Identity $DesignID -WebUrl $G.SharePointSiteURL -ErrorAction Stop

$SitesUpdated++

Set-UnifiedGroup -Identity $G.Alias -CustomAttribute13 "Site Design Updated" }

Catch {

Write-Host "Problem Processing" $G.SharePointSiteURL}

Write-Host $SitesUpdated "sites updated successfully. You need to check the following and update them manually"

Get-UnifiedGroup -Filter {CustomAttribute13 -eq $Null} | Sort DisplayName | Format-Table DisplayName, SharePointSiteURL

The list of groups generated at the end of the script includes all groups that we cannot update. In some cases, this is
because SharePoint is not fully provisioned for the group (a more common problem for older groups) and no-one has
ever tried to access the document library. If the site does not exist, we cannot update its regional settings. In other
instances, the account used to run the script might not have permissions to update site settings. These sites can be
updated manually as time allows.

JSON Column and List View Formatting

Document library and list columns can be customized with JSON code added to column configurations through the
“Format this column” setting available in modern lists and document libraries. This option allows site designers to
change the column look and feel in different ways, as described in this How-To article . In a similar manner, Microsoft
has provided the ability to customize list views with JSON code .
Figure 8-7: Sample of modern list customized using JSON Column formatting

Conditional Formatting in lists and document library columns: This is a feature announced at Ignite 2018 to
allow developers to add conditional color coding to SharePoint columns without using JSON scripts.

Fast Wins with SharePoint Online

It would take a complete book to cover SharePoint Online in any depth, so we will not try to achieve that feat here.
Often, the most important question for companies moving to Office 365 to answer is how they can extract the greatest
advantage from SharePoint Online as quickly as possible. The answer to the question will differ depending on the
background of the company.

Tenants that have never used SharePoint before can begin the process by expanding out from Exchange Online into
areas like Office 365 Groups, Teams, OneDrive for Business, and Office 365 Video, all of which use SharePoint Online.
Some simple goals can be achieved, including but not limited to:

Better team sharing and collaboration through Office 365 Groups or Teams.
Replacement of Windows roaming folders (personal shares) through OneDrive for Business.
Replacement of Windows network file shares through SharePoint Online document libraries.
Implementation of a company video portal based on Office 365 Video.

Tenants that use SharePoint on-premises for document sharing and collaboration can exploit the features described
above and begin to move existing workload from on-premises document libraries to SharePoint Online. This process
requires careful planning to ensure that workload moves smoothly, and users maintain access to information they
need. The tactics used depend on whether the end goal is a hybrid deployment or to move all SharePoint activity to
Office 365. Although the SharePoint migration API and the SharePoint Migration Tool can be used to move content
over to SharePoint Online, the process is manual and performed on a “build it yourself” basis. Microsoft does not offer
automated tools to move content to SharePoint Online. The Office 365 Import Service is the closest to a Microsoft
migration tool as it can process import packages created from SharePoint on-premises and ingest them into
SharePoint Online. However, preparing and transmitting the packages from SharePoint on-premises takes some
practice and preparatory work and it only moves a subset of data (metadata is not included). If you are looking for
automation and the ability to migrate everything from on-premises SharePoint, you need to review the third-party
tools that are available and decide which is most cost-effective.

Tenants that have invested heavily in SharePoint-based applications need to do a lot more planning to move workload
to SharePoint Online. In some cases, it will be easy to apply the same customizations and code changes to SharePoint
Online that run in the on-premises deployment. In others, the restrictions found in all cloud-based services might
mean that an existing application needs some work before it can be moved. This process might be simple if the
application comes from a third-party developer who has already upgraded the code to work with SharePoint Online.
On the other hand, updating some home-grown code for SharePoint Online can often be a challenge, especially if the
original developer is not available. Applications that make extensive use of Active Directory are another area to focus
on as the mapping between the on-premises directory and Azure Active Directory is not always one-to-one.

Advanced Threat Protection for SharePoint Online

Advanced Threat Protection (ATP) can be enabled on E5 tenants or E3 tenants with the ATP add-on to check files
stored in document libraries are not infected with malware. To turn on ATP, we only need to go to the Security and
Compliance Center, open Threat Protection , then Policy , select Safe attachments , and set the checkbox.

When enabled, ATP works across all SharePoint Online and OneDrive for Business sites in a tenant. ATP uses a
mixture of technologies, including advanced threat heuristics generated from intelligence gathered by Microsoft
about malware exploits to find suspect files. Instead of scanning all files, which would consume enormous server
resources, ATP focuses on blocking the spread of malware. When a user shares a document, ATP checks whether
sharing should go ahead. If all is well, sharing proceeds as normal. If not, SharePoint locks the file and disables the
ability to download, open, or share it. The user can remove the file to solve the problem.

SharePoint also records a “detected malware in file” audit event. As explained in Chapter 21, activity alerts or alert
policies can be created to notify administrators when ATP detects problems in files. Data about blocked files is also
available in the Threat Protection status report in the Security and Compliance Center.

OneDrive For Business

Introduction to OneDrive For Business
OneDrive for Business (or OD4B as it is known in the Office 365 ecosystem) uses SharePoint Online storage to provide
Office 365 users with a personal sharing space. In other words, it is a cloud-based file sharing service that behaves
like a personal version of a SharePoint document library. A major advantage from a tenant perspective is that Office
365 indexes all the information stored in OneDrive, meaning that it is discoverable and available for compliance
purposes.

Referring to anything as a personal storage space is a bit of a mouthful. OneDrive for Business provides personal
storage for Office 365 users to hold anything they care to store there, just like they would do with a personal network
share on an old-fashioned file server. Indeed, the prime purpose of OneDrive for Business is to help companies to
wean themselves off file servers by moving the content held on the servers into the cloud. Users who have personal
files stored in network shares can move their information to OneDrive for Business while Office 365 Groups or
SharePoint Online team sites are good homes for organizational or shared information.

Access on Multiple Devices

The big advantage of using OneDrive for Business from a user’s perspective is that once they store files on OneDrive,
the files are available from any device. When network connectivity was not as pervasive and capable as it is now, it
would not have been possible to contemplate such a movement, but it is now. Although you need to have network
access to browse OneDrive for Business sites, the OneDrive sync client allows users to work with files copied to a
local cache. Some restrictions exist in terms of the types of files and the names of files that OneDrive for Business
supports. It is sensible to check on the current limits and restrictions before you start off a project to help users move
their information from file servers. The limits for the current OneDrive sync client are much higher than in earlier
versions, so you should confirm whether any limits that stopped a project moving forward still exist. It is also sensible
to implement whatever controls are necessary for client synchronization (discussed in the next section) at the start of
a migration project instead of trying to retrofit some restrictions afterwards.

The ability of OWA, Outlook and Teams to access files held in OneDrive for Business gives further encouragement to
use the service. For example, users can upload files to OneDrive for Business and then send a link to the attachment
(a “cloudy” attachment). Users can also save attachments from Outlook or OWA direct to OneDrive for Business or to
the document libraries of Office 365 Groups to which they belong. The intention behind making these features
available is to help users change their habit of storing email attachments on their local PC and refocus towards cloud-
based storage instead. Whether it is possible to break the “always send full copies of attachments by email” habit is
debatable. What is for sure is that breaking the habit needs people to change their personal workflow, and that is
always hard.

Another advantage gained by keeping files in OneDrive for Business is that users can grant others the right to edit the
document in place, which might encourage better collaboration. By using cloudy attachments, recipients get to see
the latest version of the information they receive because access is always to the version held in OneDrive for
Business rather than a personal copy, thus avoiding the problem that someone might update a copy received in email
by the time you read an attachment.

OneDrive For Business Contents and Limits

The first time a user accesses OneDrive for Business, SharePoint Online provisions the site. Once the site is ready, the
user can upload files to the site and share those files (or complete folders). Users can attach files from OneDrive to
messages as easily as local files. To ensure that users have enough space to move their personal information from old
file servers, Office 365 users receive at least 1 TB of online storage, which comes from an allocation that is separate
to and not counted against the pooled storage used for SharePoint Online. Tenants that have five or more licenses for
E3 or higher enterprise plans (plus government and academic equivalents) can take advantage of “unlimited storage”.

OneDrive assigns a default storage quota of 1 TB to each user for their personal site, which should be enough to move
personal files off file shares and even off local PC hard drives. If you want to adjust the OneDrive for Business storage
for an account, you can either send a support ticket to Microsoft or ask an administrator to run the Set-SPOSite
cmdlet to increase the quota. Here is how to increase the storage quota from the default 1 TB to 5 TB for the
Tony.Redmond@Office365ITPros.com account.
[PS] C:\> Set-SPOSite –Identity

https://office365itpros-my.sharepoint.com/personal/tony_redmond_office365itpros_com -StorageQuota 5242880

[PS] C:\> Get-SPOSite –Identity

https://office365itpros-my.sharepoint.com/personal/tony_redmond_office365itpros_com | Format-List

Later, if the account approaches the 5 TB storage limit, you can increase the quota to 10 TB (set the value to
10485760). You can keep on increasing the quota in 5 TB chunks as the need arises. An attempt to increase the quota
will only succeed if the used quota is approaching a chunk boundary. In other words, you cannot increase the quota
assigned to an account from 3 TB or 15 TB but must first increase the quota to 10 TB and then wait until the storage
used passes 12 TB or thereabouts before upping the quota to 15 TB.

To check the quota assigned to each user and how much of the quota they use, you can run the following PowerShell
snippet.

[PS] C:\> $Root = "https://office365itpros-my.sharepoint.com/personal/"

[PS] C:\> $O365Users = Get-MsolUser -All | ? {$_.IsLicensed -eq $True}

[PS] C:\> $OneDrive = $O365Users | ForEach {Get-SPOSite ($($Root)+$($_.UserPrincipalName.Replace(".","_"))).Replace("@","_") | Select

Owner, StorageUsageCurrent, StorageQuota}

[PS] C:\> $OneDrive

Owner StorageUsageCurrent StorageQuota

----- ------------------- ------------

james.gangley@office365itpros.com 1 1048576

brian.weakliam@office365itpros.com 1 1048576

tony.redmond@office365itpros.com 3727 5242880

No error checking is in this simple code, so you will see an error if an attempt is made to check a user’s OneDrive site
that Office 365 has not yet provisioned because the user has never used OneDrive. To fix that problem, you can
request OneDrive for Business to provision a site for the user by running the Request-SPOPersonalSite cmdlet. The
input to the cmdlet is a comma-separated list of email addresses for the users for which OneDrive is to create sites.
You can specify up to 200 email addresses in the list.

[PS] C:\> Request-SPOPersonalSite -UserEmails "Sanjay.Ramaswamy@office365itpros.com, David.Pelton@office365itpros.com” -NoWait

OneDrive uses a timer job to create the sites. The actual time when the sites will exist depends on system load.

Information Views in OneDrive For Business

OneDrive For Business includes several views to give the user the ability to see their information as well as the
information they have shared with other users and the information other users have shared with them. Table 8-2
details the current information views included in OneDrive For Business.

Information Information View Details

View

Files Shows the files and folders stored in the user ODFB.

Recent Shows a list of recent files the user has worked on, including the date and time of the last access.

Shared Two additional information views are available. “Shared with me” shows the files shared with the user
including the person sharing the file and the last file activity. “Shared by me” shows the files the user
has shared with other users and the last file activity.
Table 8-2: Information Views in OneDrive For Business

Sharing in OneDrive For Business

Sharing is one of the key capabilities provided by OneDrive For Business driven by the same features previously
described for SharePoint sites. However, Microsoft trends to release new sharing capabilities in OneDrive first and
after in SharePoint Online. Indeed, Table 8-3 lists some of the Sharing capabilities that appeared initially in OneDrive
For Business and then in SharePoint Online.

Sharing Sharing Feature details

Feature

Customer Organization’s logo is placed in any sharing e-mail sent when a file is shared from ODFB and/or
branding in SPO if the organization’s branding is specified in Azure Active Directory
ODFB and SPO
sharing e-mail

ODFB Link When a file or folder shared from ODFB is opened by the recipient, an e-mail is sent to the sharing
open receipt user it to notify them that the file or folder has been opened. The notification e-mail also includes a
link to file or folder permissions management to review how the file or folder was shared and even
remove the sharing link.

Table 8-3: Sharing capabilities released by Microsoft first in OneDrive For Business

OneDrive Client and Synchronization

OneDrive Sync Client and Files on Demand
The most frequent problem reported by users is when synchronization of updates with document libraries fails. In
fact, two OneDrive sync clients exist:

The older client (Groove.exe) had a bad reputation because it often experienced problems when synchronizing
files from a PC to a SharePoint or OneDrive for Business site. Microsoft has deprecated this client and if possible,
you should no longer use this client in OneDrive deployments unless forced to because a document library uses a
feature not yet supported by the new client. See this support article for more information about current sync
restrictions and limits .
The new OneDrive for Business sync client (OneDrive.exe) shares a common code base with the OneDrive
consumer client and is much more robust and reliable. Performance is better, the client can synchronize libraries
with more than 20,000 files and can handle large files of up to 15 GB and integrates with Windows Explorer. The
new client also can pause synchronization in low-bandwidth environments (such as when connected to Wi-Fi on
an airplane). This client supports OneDrive for Business and SharePoint sites, including the document libraries
used by Office 365 Groups and Microsoft Teams (Files).

Restrictions and Limitations in OneDrive For Business Synchronization : OneDrive for Business can sync both
OneDrive for Business libraries as well as SharePoint Online document libraries from team sites, but there are some
documented restrictions and limitations that must be considered when synchronizing files to a local workstation.

The Settings section of the SharePoint Admin Center includes the OneDrive Sync Button setting to control the sync
client for the tenant. You can hide the button completely if you not want users to be able to synchronize OneDrive and
SharePoint Document libraries to their workstations.

Files On-Demand is a feature that allows users to control local storage for files held in OneDrive for Business or
SharePoint Online sites. The feature is managed through the Settings tab of the OneDrive sync client, where users
can set the checkbox to enable Files on Demand for their account. Afterwards, they can select how to store remote
files locally for the folders from OneDrive and any synchronized SharePoint site that they want to see on the device.
Synchronized files can be in the following states:

Online-only files are in OneDrive or SharePoint sites but appear in File Explorer to let the user know that they
exist. These files occupy no disk space on the local computer.
If users open online-only files, File Explorer downloads them from the host site and makes the files locally
available . A locally available copy can be opened at any time, even if a network connection to Office 365 is
unavailable. When a network connection becomes available, the OneDrive sync client synchronizes any changes
made on the workstation with the host site.
Users can also decide to mark files as Always keep on this device . This means that the OneDrive sync client
copies these files from the host site to make them available locally even if the user never opens them.
File Explorer uses different icons to show users the status of files. Online-only files have a cloud icon, while locally-
available copies have a white circle with a green checkmark and files always present on the device have a green circle
with a white checkmark (Figure 8-8 ).

Figure 8-8: Files On-Demand icons

When the OneDrive sync client is processing a file, the icon changes to two curved arrows in a circle. To change the
synchronization status of a file, select it (or several files at the one time), and use the right-click menu. You can see
the icon showing the synchronization state of the files in the Status column. Files on Demand settings are unique to a
device. If you use several devices, you must configure the settings on each device.

Files On-Demand availability: OneDrive Files On-Demand requires Windows 10 Fall Creators Update (version
16299.15 or later) and OneDrive build 17.3.7064.1005 or later. A preliminary version of Files On-Demand is also
available on Mac OS Mojave 10.4 for Office 365 Insiders and can be configured using the instructions in this support
article .

From October 2018 onwards, Files On-Demand will be also part of the Windows Storage Sense feature so when disk
space runs low, it is automatically free up by moving back to a cloud-only state oldest files (not marked as “always
keep on device”) on your computer.

Controlling Client Synchronization

A common issue often raised by management and those responsible for the protection of intellectual property is the
potential undesirability of allowing users to synchronize documents to their PC, possibly including a PC at home. The
same synchronization issue occurs for SharePoint Online document libraries too. It is true that users do store
confidential information on home PCs (the blurring of home and work life often encourages this practice) and it's also
true that various solutions can stop it (such as restricting synchronization to domain-joined PCs ). However, users
have been swapping data around between PCs ever since floppy diskettes and it’s likely that users will find new
methods of moving information to desired locations as quickly as IT closes off loopholes.

Automatic Updates : Microsoft releases updates to the new OneDrive client independently of other Office 365
updates. The updates are automatically downloaded and installed on client computers unless a group policy is
implemented to control OneDrive updates . Like many current Microsoft products, OneDrive updates are released in
“rings”. You can configure a PC to receive updates faster by configuring a system registry DWORD value called
EnableTeamTier_Internal at HKEY_CURRENT_USER\Software\Microsoft\OneDrive. Set the value to 1 if you want
to receive early updates (fast ring) or 0 (zero) to receive updates as normal (slow ring).

Four approaches can help control the content held in OneDrive for Business sites. First, you could deploy Azure
Information Protection (see Chapter 24) to protect confidential information so that files can only be opened by their
intended recipients. Second, Data Loss Prevention (Chapter 22) policies can prevent inadvertent disclosure of
sensitive data by sharing that information outside the company. Third, as mentioned above, you can configure
SharePoint Online so that the OneDrive sync client will only copy files to workstations that belong to specific on-
premises Active Directory domains such as those used to authenticate user logons. The intention here is to stop
corporate data ending up on PCs not managed by the organization, such as those running at home.

Protecting OneDrive for Business: A Microsoft script helps administrators to configure rights management for
users’ OneDrive for Business sites. The script can also be used to reverse the process if necessary (some problems
have been reported with the sync client when protection is enabled). To use the script to remove protection from
sites, change the line $list.IrmEnabled = $true to be $list.IrmEnabled = $false .

As shown below, the synchronization block is set by running the Set-SPOTenantSyncClientRestriction cmdlet to enable
checking against a set of known domains, each of which is identified by a GUID. You can use the Get-ADForest
PowerShell cmdlet to return the set of domains in an Active Directory forest and then Get-ADDomain cmdlet to find
the ObjectGUID for each domain (see this page for information how to determine the ObjectGUID for a domain).

[PS] C:\> Set-SPOTenantSyncClientRestriction -Enable

-DomainGuids 'Domain-GUID1; Domain-GUID2; Domain-GUID3'

When the block is in place, users see that OneDrive for Business libraries cannot be taken offline if they try to
synchronize them to an unmanaged PC. Administrators can use Office 365 Auditing (Chapter 21) to run a report to
highlight computers that OneDrive blocks from syncing files to discover whether users have tried to take files offline
and then take the necessary action to tell those users why this facility is unsupported.

Although this synchronization block will not stop mobile clients using OneDrive for Business, it does block Mac
clients. For this reason, you should not implement the block if Mac clients are in use in your tenant. Another
restriction is to constrain users to upload files with a set of known extensions. For example, if you do not want to have
users uploading executables, zipped files, and PSTs to OneDrive for Business, you can run the following cmdlets.

[PS] C:\> Set-SPOTenantSyncClientRestriction –ExcludedFileExtensions "exe;zip;rar;pst"

[PS] C:\> Get-SPOTenantSyncClientRestriction

TenantRestrictionEnabled AllowedDomainList BlockMacSync ExcludedFileExtensions

------------------------ ----------------- ------------ ----------------------

False {} False {exe, zip, rar, pst}

Separate each file extension with a semi-colon and do not leave any extra spaces. The Get-
SPOTenantSyncClientRestriction cmdlet can check that the proper restriction is in place.

If necessary, you can clear the set of excluded files by setting the property to $Null . The setting only controls the
OneDrive.exe Windows client, so you can add excluded files to a OneDrive for Business site with another client before
the exclusion kicks in to prevent file synchronization with the PC.

Blocking OneDrive Consumer : The current version of the OneDrive sync client can synchronize both OneDrive for
Business and OneDrive consumer accounts to a single PC. However, some organizations do not want to allow users to
synchronize their OneDrive consumer account to corporate PCs. It’s possible to block this through a Group Policy
setting called DisablePersonalSync . See this Microsoft page for information how to control the setting and to learn
about other Group Policy settings that can be used with OneDrive.

The last approach that you can take is to remove the temptation for users to synchronize files from certain document
libraries by hiding the Sync button for those libraries. If the Sync button is not available for a document library, users
cannot start the synchronization process. To do the job, you need to write some PowerShell code that uses the
SharePoint CSOM API. Fortunately, a script exists to help start the job. You can take that code and change it to meet
your needs. For instance, you might use a CSV as the input to specify the target sites to disable synchronization. It is
important to realize that removing the Sync button will not interfere with or stop synchronization if the user
previously set it up for a document library. However, it stops new synchronization.

It might seem that controlling client synchronization is a big issue for OneDrive, but it is not really. Similar issues to
those listed above often happen in how people use (or abuse) personal file shares. The best thing about OneDrive For
Business is that all the features of Office 365 security and data governance protect the information in user sites. In
addition, because OneDrive For Business is an area of focus for Microsoft, a reasonable expectation exists that more
features will become available over time.

The question of Shared with Everyone : Office 365 used to automatically create a “Shared with Everyone” folder
it created OneDrive for Business sites for new users. The folder is intended to be used as an easy way for people to
share information with others within a tenant. Unfortunately, experience proved that the sharing was just a little too
easy at times and resulted in information being disclosed inadvertently through tools like Delve. Users who already
have the folder can continue to use it. If you wish, there is nothing to stop you creating the folder manually after a
new OneDrive for Business site is created or, if you want OneDrive for Business to behave as before and create the
“Shared with Everyone” folder in all new sites, you can run the cmdlets shown below to update the SharePoint Online
configuration:

[PS] C:\> Set-SPOTenant –SharingCapability Disabled –ProvisionSharedWithEveryoneFolder $True

OneDrive For Business Files Restore

Ransomware has the nasty habit of infecting document, no matter where they are. If an attack happens against a
OneDrive site, you can use Restore OneDrive (click the cogwheel to expose the choice) to restore files from the
document library versions to a point in time. OneDrive comes with some out-of-the-box points (such as a week ago),
but you can select any point up to a 30-day limit (Figure 8-9 ). When you select a restore point, OneDrive prompts you
to confirm to go ahead with the restore, and if confirmed, OneDrive restores files as they were at that point.

Figure 8-9: File Restore in OneDrive For Business

Files Restore Coming also to SharePoint Online Sites : Files restore feature is also coming to SharePoint Online
Sites to give the ability to recover files in case something bad happens in a document library.

The consumer version of OneDrive also supports the ability to restore to a point in time. However, the feature is only
available if you have a paid subscription (Office 365 Home).

Known Folder Move (KFM)

KFM is simplifies the process to move user contents located in well-known PC folders (Desktop, Documents and
Pictures folders) to OneDrive. With KFM, user content is automatically synced to OneDrive and can be accessed from
all user devices. To start using this feature, end users only need to click on a KFM toast notification or click the
Update folders button under the Autosave tab of OneDrive Client settings. Before the OneDrive sync client starts
synchronization, it checks for the presence of any unsupported files, and then begins to copy the folders from the
workstation to OneDrive. KMF works on Windows 7, 8, and 10 workstations.

KMF also comes with the ability to silently redirect Windows known folders to OneDrive without user interaction. To
enable this setting, a group policy must be enabled.

Managing OneDrive For Business

OneDrive For Business, as any of the core Office 365 workloads, can be managed through its Admin Center or with
PowerShell.

Using the OneDrive Admin Center

OneDrive Admin Center is a central location to manage OneDrive for Business settings such as Sharing, Sync or
Device access. It can be accessed in three different ways by any user granted with an Office 365 Admin role or a
SharePoint Admin role:

From the OneDrive shortcut under the Admin center section in the left pane.
From the user’s OneDrive using the OneDrive Admin link in the lower-left corner.
Typing https://admin.onedrive.com in the browser.

Although the console gives administrators a GUI (Figure 8-10 ) to manipulate settings that are already available
through the SharePoint Admin Center or PowerShell, many administrators prefer to avoid configuring options through
a command-line interface.

Figure 8-10: Controls in the sharing section of the OneDrive Admin Center

Compared to the current SharePoint Admin Center, the OneDrive Admin Center gives a small subset of configuration
options covered below. Some of those options also change SharePoint settings.

Home : Intended to show recent message posts and other information relating to OneDrive for Business.
Sharing : Access to settings for sharing SharePoint and OneDrive for Business files with external users, what
sharing links are available to users, plus the external domains with which users can share files.
Sync : Allows control over synchronization of content to non-domain joined PCs and blocking of specific file
types. From this section, a OneDrive Admin can also choose to apply the setting that hides the Sync button in the
OneDrive UI.
Storage : Access to settings that control the storage assigned to users and the number of days that OneDrive For
Business keeps files after a user account is deleted.
Device access : Access to settings controlling when and where devices can access content held in OneDrive for
Business and a mobile device management policy to control access to content via the mobile OneDrive for
Business and SharePoint apps (for iOS and Android). A Microsoft Intune or Enterprise Mobility Security
subscription is required to deploy that device management policy.
Compliance : Links to options relating to OneDrive for Business available in the Security and Compliance
Center, including the Office 365 audit log, Data Loss Prevention policies, Retention, eDiscovery cases and Alerts.
Notifications : Controls the notification sent by OneDrive when file sharing occurs or when external users
accept invitations to access files.
Data migration: Provides access to a page with information about how to migrate content to OneDrive From
Business , including a link to download the SharePoint Migration Tool.
We discussed how to control sharing settings for SharePoint earlier. Office 365 synchronizes changes made through
the OneDrive for Business Admin Center to SharePoint Online, where the settings apply at the tenant level. One thing
to be careful about is that the sharing level set for OneDrive for Business cannot be more permissive than for
SharePoint. For instance, if the tenant does not allow sharing for SharePoint files, you cannot allow sharing in
OneDrive for Business. You can configure sharing in OneDrive for Business to be less permissive than allowed by
SharePoint, as in the case when you do not want users to share personal documents with external users who are not
already known in the tenant directory.

In most cases, you should not allow anonymous sharing (“Anyone ”) as anyone who receives a link to a document can
open it without a sign-in. Sharing with New and existing users means that users can share with any email address.
When that person receives an invitation to open a file, they can only redeem it once by proving their identity with an
access code generated by SharePoint or signing in with a guest account associated with the email address. Restricting
sharing to Existing external users means that a user can only send an invitation to people already known in the
tenant directory (guest accounts).

The advanced settings for external sharing allow you to create an allow list for external domains with which you are
happy for users to share files. Conversely, you can create a block list to prohibit sharing with specified domains. You
can have either a block or an allow list; you cannot have both. Another setting controls whether external users must
accept sharing invitations from the same email address used in the invitation. In other words, an external user cannot
sign in from DomainA to accept an invitation sent to them in DomainB . Finally, you can control whether external
users can share items they do not own (a bad idea, normally).

The OneDrive reports available in the Office 365 Admin Center and the Microsoft 365 usage analytics (Power BI
content pack) give some insight into the activity level for OneDrive for Business within a tenant. The depth of
information is acceptable for small to medium tenants but might need more to satisfy larger enterprises. For example,
knowing that OneDrive usage has grown over the last 90 days is good, but understanding how that growth developed
over the last two years is better. Another gauge of OneDrive activity can be assessed by the volume of OneDrive
events captured by the Office 365 auditing log. Chapter 21 describes how to use the audit log to search for and
interpret the events stored there.

Using PowerShell to Manage OneDrive for Business

Although Microsoft has no specific tools to manage OneDrive for Business with PowerShell, the same principles and
guidelines used for SharePoint Online apply. A SharePoint Admin can also use the SharePoint Online Management
Shell, PowerShell ISE, PnP Cmdlets and SharePoint and Office 365 APIs to manage OneDrive. As an example, the
following piece of PowerShell code extracted from this public PowerShell script creates a set of OneDrive For
Business sites using the SharePoint Online CSOM API (remember that you need to have CSOM binaries installed in
your PC to execute this code):

[PS] C:\> $spoCtx = New-Object Microsoft.SharePoint.Client.ClientContext($sSPOAdminCenterUrl)

$spoCredentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($sUserName, $sPassword)

$spoCtx.Credentials = $spoCredentials

$spoUserProfilesLoader=[Microsoft.SharePoint.Client.UserProfiles.ProfileLoader]::GetProfileLoader($spoCtx)

$spoUserProfilesLoader.CreatePersonalSiteEnqueueBulk($sODFBUsers)

$spoUserProfilesLoader.Context.ExecuteQuery()

$spoCtx.Dispose()

To add a secondary administrator to an existing OneDrive site, we can use the Set-SPOUser cmdlet as follows:

[PS] C:\> $sODFBUrl="https://spoforitpros-my.sharepoint.com/personal/gustavo_spoforitpros_onmicrosoft_com/"

$sSecondaryODFBAdmin="jcgonzalez@SPOForITPros.onmicrosoft.com"

Set-SPOUser -LoginName $sUserName -Site $sODFBUrl -IsSiteCollectionAdmin $true

Finally, as an example of the potential for managing OneDrive For Business using PowerShell, this script shows how to
get all the items in ODFB recycle bin :

[PS] C:\> $spoCtx = New-Object Microsoft.SharePoint.Client.ClientContext($sODFBUrl)

$spoCredentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($sUserName, $sPassword)

$spoCtx.Credentials = $spoCredentials

$spoODFBSite= $spoCtx.Site

#ODFB Recycle Bin Content

$spoRecycleBinItemCollection = $spoSite.RecycleBin

$spoCtx.Load($spoRecycleBinItemCollection)

$spoCtx.ExecuteQuery()

foreach($spoRecycleBinItem in $spoRecycleBinItemCollection){

Write-Host "Item:" $spoRecycleBinItem.Title "- Deleted By:" $spoRecycleBinItem.DeletedByName "- Deleted Date:"
$spoRecycleBinItem.DeletedDate

$spoCtx.Dispose()

Migrating to SharePoint Online and

OneDrive For Business
Preparing to Move from On-Premises SharePoint
Moving from an on-premises SharePoint deployment to Office 365 is attractive in terms of the reduction in effort
needed to manage SharePoint farms and other pieces of infrastructure but it’s not a project to be rushed into.
Experienced consultants who have been through the process multiple times suggest that a measured approach works
best, and that good project management is key. The following steps are often involved:

1. Achieve stakeholder consensus . Although moving to Office 365 might seem to be the right technology
approach, other players have a voice in the decision. SharePoint is often tied into or provides the basis upon
which business processes are implemented, so buy-in will be required from the business owners, compliance
department, and other interested parties. The transition will involve IT roles such as those responsible for SQL
Server, hardware and storage management, security, and so on.
2. Identify other software used with SharePoint . Just like Exchange Online, it is harder to use third-party
software with SharePoint Online because no access is available to the Office 365 datacenters. You can certainly
deploy monitoring and reporting software that can use remote probes and queries to retrieve information from
SharePoint Online, but any software that interacts with SharePoint needs to be tested to verify that it will
continue working against SharePoint Online. If not, decisions must be made about how the software can be
replaced. Third-party apps are relatively easy to identify, but to come up with a complete picture, you need to
also note every customization that has been applied to a SharePoint site, including things like custom content
types. At the end of the day, the same customizations will need to be applied to SharePoint Online (if the
customization is supported in the cloud).
3. Prepare your infrastructure . If the on-premises SharePoint infrastructure is showing any problems, now is the
time to fix them. Lurking issues that become known during a migration or when hybrid connectivity (to link
SharePoint on-premises with SharePoint Online) is put in place are likely to cause havoc. A solid, well-functioning
and reliable infrastructure is a prerequisite for the migration to start.
4. Know what you must deal with . Apart from software used with SharePoint, you also need to understand how
much data is involved, where that data is found, how it is accessed and used and by whom, the business owner,
and its importance to the company. This information will guide your decision as to when the data is moved to
Office 365. Sometimes it might be necessary to keep sensitive data on-premises to meet compliance or other
regulatory concerns. Sometimes you might be forced to delay some part of the migration until a solution is found
for some software that interacts with SharePoint Online. A detailed inventory of applications, data, and users will
give the essential information necessary to plan the migration. Each application should have a known owner and
purpose.
5. Audit ownership and site collections . Because of the complexities that might be encountered in moving
workload and content to SharePoint Online, it makes sense to begin with some basic principles and create an
inventory of applications and content running inside the on-premises SharePoint deployment. Analyzing the full
scope of the customizations, applications, and content that exist inside your current on-premises environment
will make it very much easier to plan the migration to SharePoint Online and to engage with consultants and
vendors of third-party migration utilities as you refine and settle on your migration strategy.

The inventory is likely to throw up some opportunities for rationalization and

efficiencies. You might be able to remove some old and now-obsolete applications and
sites that no one still needs. Examine the list of site owners to decide whether the names
are correct and still need the level of access that they have. Changing responsibilities
within companies often lead to a situation where someone can still access data that they
have no business reason to go near. Trim ownership and sites so that you know what
must be moved to Office 365 and whom will manage the data once it gets there. Here
are some questions that you might use to assess the situation:

What content exists in each site collection and who uses the content? How much of the content is still required?
Do you plan to move your content as it is or to a new Information Architecture designed and deployed to
SharePoint Online?
How much content needs to move to SharePoint Online?
 Do you need to move also content metadata, versions and security configuration?
Is SharePoint customized and can the same customizations be applied to SharePoint Online?
What applications have been developed based on SharePoint? Was the code developed in-house or bought from a
third party? Will it run inside SharePoint Online and if not, what changes are necessary and how long will the
changes take? Who uses these applications?
What workload and content can be first moved to SharePoint Online to confirm the migration process? Will the
content fit into the tenant’s default storage or is added storage needed?
What workload and content needs to stay on-premises? Do some blocking issues exist to make the platform
choice permanent or should it be possible to move the workload and content to SharePoint Online in the future?
Is hybrid search capability needed to find information across both on-premises and cloud platforms? Microsoft
has a cloud hybrid search service that enables data to be sent from on-premises SharePoint 2013, SharePoint
2016 or SharePoint 2019 servers to SharePoint Online. Setting the service up usually needs some effort in
planning and execution. Microsoft Press offers a free eBook ( Planning and Preparing for Microsoft SharePoint
Hybrid ) that is helpful when creating plans for this activity.

6. Decide how you will move . Many tools are available to help move information from SharePoint on-premises to
SharePoint Online, including a free tool from Microsoft .
7. Create an end state plan . The plan describes what will move to SharePoint Online and what, if anything, will
stay on-premises. It should also lay out the order that information and users will move. If hybrid connectivity is
going to be used to allow consolidation of on-premises and cloud data, details of how the connection is set up and
supported needs to be included.
8. Ask for advice . There is likely to be a local SharePoint or Office 365 user group where it will not cost anything
to ask others who have moved to SharePoint Online for their advice on the process. Some might even be willing
to review your plans and offer an opinion if improvements can be suggested. Conferences are a useful source of
information, even at the risk of being presented with too much information over a brief period. You might have to
hire an external consultant to help understand and solve some of the trickier questions in your migration plan. If
so, make sure that they have experience of SharePoint migrations and are not simply regurgitating the contents
of books.
9. Test. And then test again . Testing makes perfect. At least, thorough in-depth testing makes perfect. Testing
should not only feature members of the project team and IT department. You need to include representatives
drawn from across the user community to ensure that what will be delivered by SharePoint Online meets the
needs of everyone instead of just the project team.
10. Take it easy . SharePoint is not an application that you want to move through a big-bang, “we can get it done
over the weekend” effort. Move workload in phases beginning with the simplest sites and user communities and
gradually moving up to tackle the more complex sites that need customizations and third-party software. Pause
after each phase to understand what has been done and what needs to be improved in the next step. Do not take
on too much at a time as this usually ends up in failure.

Migrations are never as easy as you might think. SharePoint is particularly challenging because it is almost as
complex as an operating system. Unless you have left SharePoint on-premises alone and only use whatever features
are included in the basic software, you are going to spend time figuring out how best to achieve the same result in
SharePoint Online.

Moving Windows File Servers to SharePoint

Many tenants replace old-fashioned Windows shared drives with SharePoint Online libraries as part of the transition
from on-premises servers to Office 365. This is not a move that should go ahead on the basis that it is enough to copy
folders from the shared drives (and all the files they hold) across to equivalent folders in SharePoint document
libraries. This is a mistake often made by those who are less experienced with SharePoint. Like most migrations, some
up-front thought and planning usually generates a better outcome.

SQL Server databases lie at the heart of SharePoint Online. Documents, lists, and any other data used by SharePoint
exist in database tables and do not use the kind of file structure found in shared drives. Although SharePoint Online
can store millions of items in a document library, it used to be the case that performance problems arose when there
are more than 5,000 items in a single view in the document library. Microsoft has partially solved that problem by
means of the predictive indexing available for modern document libraries , but some other issues exist if you store
more than 5,000 items in a single view in the document library:

Usability : Complex nested folder structures often obscure the information that they hold. It is not good when
users must navigate through multiple folders to find the files they need. A general recommendation to classify
information is to use SharePoint Content Types and avoid using complex nested folders as a classification
mechanism.
URL limitations : If you have a complex folder structure in a document library, the URLs constructed to allow
access to the folders might exceed the 400-character limitation because the URL includes all the folder names.
Loss of Flexibility : Once a folder structure is in place, it is very difficult to change.

SharePoint experts recommend that you can avoid issues with folder structures by using indexed metadata to
organize files within libraries. If necessary, you can create views to sort and categorize the files in whatever order
users need to organize their work. Planning the metadata needed to organize documents and how to populate the
metadata as new contents comes into document libraries is not a project to take on lightly. If you are unfamiliar with
the concepts and processes involved in creating a suitable taxonomy, it is best to get some help so that the design is
right from the start.

Moving Files
Moving individual files from a network file share to a SharePoint document library, perhaps one used by an Office 365
Group or Team, can be a tiresome process. The OneDrive sync client is often used to move files between different
libraries, and you can use it to move files from network shares too. Here’s how:

Synchronize the target SharePoint document library to a workstation with the OneDrive sync client.
Map the source network file share on the workstation.
Use Windows Explorer to copy files from the source network share to the folder holding the synchronized content
from the SharePoint document library.
The OneDrive sync client will then synchronize the copied files to the document library.

This is an effective approach when you have a few hundred files to move. Things become more complicated when you
have more files to process. At this point you can consider:

Run a PowerShell script to move files. Microsoft’s SharePoint PowerShell module does not have cmdlets to
directly copy files to SharePoint Online or OneDrive, but it provides cmdlets to migrate on-premises contents by
creating a migration package that can be submitted to be imported by the Office 365 Import service .
The SharePoint Patterns and Practice (PnP) module provides cmdlets to copy contents to a SharePoint document
library. Here is an example of a script that uses the PnP module to copy files from a network file share to a
SharePoint document library.
Use Microsoft’s free SharePoint migration tool (see the section later in this chapter).
Use a commercial third-party product. These products vary in terms of capability and cost, so some testing is
necessary to figure out which best suits your circumstances.

Metadata
Document management systems depend on metadata to organize and find information. Although users do not need to
add metadata to documents (apart from a title and file name) when they add them to a SharePoint document library, it
is a good idea to coach people to change the habits that they might have from using file servers to pay more attention
to the titles, tags, and comments that they can add to documents, plus any other metadata considered necessary for
certain types of documents. The big pay-off is that documents with good metadata are easier to find with SharePoint
Search, Delve, and eDiscovery searches.

Versioning
SharePoint Online and OneDrive for Business support versioning. This is a useful feature that does not exist for
documents stored on local drives or file shares because it allows you to keep multiple versions of documents and other
items and revert to an earlier version if necessary, as in the case of corruption or a cyberattack. Some organizations
use versioning to keep all versions of a document for legal or audit purposes. By default, SharePoint and OneDrive for
Business enable versioning for document libraries (you need to enable it for lists). SharePoint Online support major
and minor versions while OneDrive for Business only supports major versions. However, there is no technical
difference between a major and minor version of a document. Both are updated files.

The latest updates of the click-to-run versions of the Word, PowerPoint, and Excel applications include an AutoSave
feature to automatically capture changes made to documents stored in SharePoint and OneDrive for Business
libraries. The idea is that a user can work on a document without having to worry about saving it because saving
occurs in the background on an ongoing basis. AutoSave uses versioning to capture edits made using the Office
desktop applications or the cloud apps (like Word Online). If you like, you can make registry setting to disable
AutoSave for the Office desktop applications, but you cannot affect how the cloud apps save document edits.

Because features like the AutoSave and OneDrive document restore features depend on the availability of earlier
versions of files, from 30 September 2018 onwards, the default will be that SharePoint Online and OneDrive for
Business libraries support a minimum of one hundred major versions . Microsoft will update existing libraries that
have versioning enabled for fewer than one hundred versions to support the new minimum. In addition, the document
library settings page will no longer offer the choice to disable versioning or to reduce the number of versions below
one hundred.

Organizations who don’t want to enable minimum versioning must run Set-SPOTenant to disable the feature before
Microsoft enables it by default. To disable the feature, download the latest version of the PowerShell module for
SharePoint Online and run the command:

[PS] C:\> Set-SPOTenant -EnableMinimumVersionRequirement $False

Microsoft recommends keeping the minimum of one hundred versions to enable maximum recovery of documents.

SharePoint Migration Tools

Microsoft’s free SharePoint Migration Tool can move documents from on-premises SharePoint sites (2013 version) and
file servers to Office 365. All you need is to be able to read from the sources and write to the target site collections.
After checking that access is available, the tool extracts the source documents and creates a set of XML manifests and
content. The tool then uploads the content package to a secure location in Azure and invokes a migration job in
SharePoint Online to fetch the content and import it into the target locations. In version 2 of the toolset , Microsoft
has provided features such as the capability to migrate SharePoint 2013 lists, create the destination site for a
migration in case if does not exist and JSON support for bulk migrations. In the new version of the SharePoint
Migration Tool revealed at Microsoft Ignite 2018, it will be possible to migrate full SharePoint 2013 sites . Microsoft
expects to deliver this feature in preview in January 2019 .


PowerShell support for the SharePoint Migration Tool : There are eight PowerShell cmdlets documented to
automate migrations to SharePoint Online using the SharePoint Migration Tool. A sample migration script showing
how to use the cmdlets can also be found in the cmdlet documentation

You cannot beat the zero-cost price point for the Microsoft tool. However, the tool offers a limited feature set and if
you have more complex requirements to move documents, schema changes, metadata, and customizations, you might
need a more sophisticated approach to migration such as those available from Metalogix, AvePoint, or Sharegate. All
SharePoint migration utilities use the same API and are subject to being throttled by Office 365. Third-party migration
products usually have their own ways to increase throughout or minimize throttling to move data to Office 365 more
quickly, and this might be a factor in your planning if you have large quantities of data to move. See this page for
more information about likely throughput for SharePoint and OneDrive for Business migrations.

Migration utilities typically work on the basis that they move documents and metadata from a source to a target
destination (a SharePoint Online site or site collection). Inevitably, some documents end up in the wrong place and
need repositioning into the correct destination afterwards. Until Microsoft upgraded SharePoint’s “Move to” feature
(January 2018), to allow movement of files to a different site, rearranging documents post-migration was difficult and
time consuming. Now, you can move documents to any folder in a SharePoint Online or OneDrive for Business site for
which you have write access. Moving is different to copying because a move includes all the versions of a file instead
of just the final version. In addition, moves preserve metadata between source and target destinations by matching
column names (if a column name does not exist for the target, the move drops that metadata). Moves also respect
compliance policies.

You should decide early on what migration tool to use for your project. Making that choice can absorb a lot of effort to
analyze, test, assess, and eventually decide. Do not underestimate the importance of this work.

On-Premises inventory/audit : Before starting a migration to SharePoint Online, it’s advisable to make an
inventory / review of the contents and structures to be migrated so any issue that can face during the migration can
be detected beforehand. Microsoft provides the SharePoint Migration Assessment Tool (SMAT) to help identify the
impact of migrating from SharePoint 2010 / 2013 to SharePoint Online. Of course, as happens with the SharePoint
Migration Tool, there are also commercial inventory tools that provide more detailed information about a SharePoint
Farm and issues that can arise when starting a migration project. The SharePoint Documentation Toolkit from SysKit
and Metalogix Expert from Metalogix are examples of third party inventory/auditing tools.

Deprecated SharePoint Online

Features
Like the other Office 365 services, SharePoint Online is in a state of constant evolution. This means that Microsoft
adds new features, improves existing functionality, and deprecates some features. Deprecation means that the feature
will be eliminated in the future. Table 8-4 shows a list of deprecated features in SharePoint Online and OneDrive For
Business since 2018. This list comes from the information published in the Office 365 Message center.

Feature Deprecation Details

Public Newsfeed in In June 2018, Microsoft will start removing one of the native social features in SharePoint
SharePoint Online Online: the SharePoint Public newsfeed. Basically, the following actions will happen:

The Public newsfeed will be Read-Only, so it won’t be possible to post new news there
or reply to existing ones.
The Option to implement this feature will be removed from the SharePoint Online
Administration.

As an alternative, Microsoft recommends using Team News, Communication Sites, and/or

Yammer.

Machine Translation Beginning September 2018, Microsoft will remove the user interface entry point for
Services for SharePoint Machine Translation Services from SharePoint Online. This deprecation has the following
Online impacts:

It won’t be possible to perform manual or on-demand translation requests from

SharePoint Online.
 It won’t be possible to create new variation labels for multilingual scenarios based on
the variations feature in SharePoint online.

As an alternative to this feature, Microsoft recommends using Microsoft Translator APIs .

Table 8-4: Deprecated features in SharePoint Online and OneDrive For Business

Other services that use SPO and ODFB

SharePoint Online and OneDrive for Business are not only part of the core workloads of Office 365, but also a key part
of other Office 365 services in combination with other services such as Exchange Online and Azure Active Directory:

Office 365 Video , the first Microsoft service designed to provide an enterprise video service for Office 365.
Office 365 Video uses SharePoint Online to store the original video files that are sent for processing to Azure
Media Services, so any user can play the videos with a great user experience, no matter the device and network
connection in use. Each time an Office 365 Video channel is created, a new SharePoint Online site collection is
created behind the scenes. This video service and its successor (Microsoft Stream) are extensively covered in the
Companion Volume (Chapter 9).
Office 365 Groups, the email-based collaboration and productivity solution built on top of the core Office 365
Workloads: Azure AD, SharePoint Online, Exchange Online and Skype For Business Online. Each time a Group is
created in Office 365, a SharePoint Online Site Collection is provisioned for the Group. Group members can
easily store files there, create lists and document libraries and, even, subsites. Office 365 Groups are explained
in detail in Chapters 11 and 12.
Microsoft Planner , the task management application in Office 365 is deeply integrated in modern SharePoint
Team sites through the Planner web part, which developers to incorporate plans in pages in a modern site.
Microsoft Teams , the chat-based workspace uses Microsoft Groups as its membership service, which means
each time a Team is created, a Group is created behind the scenes together with all its building blocks (an Azure
AD Object, an Exchange Online mailbox and a SharePoint Online Site Collection). Again, all the documents
generated through team activity are stored in a SharePoint Site collection. Teams also uses OneDrive For
Business to store files shared in personal chats. Chapter 13 explains Microsoft Teams in-depth.
Microsoft Yammer , the enterprise social network platform integrated in Office 365 that can be configured to
use Office 365 connected Groups in Yammer. In this way, each time a Yammer Group is created, a special type of
Office 365 Group is provisioned including a SharePoint Online Site, a Planner Plan and a OneNote. All the Group
files are stored in the Group site document library.

Create a Team connected to a group-connected SharePoint Team Site : You can create a Team from a group-
connected SharePoint Team site . There are too many teams and groups in the last sentence, but what it means is
that you can to link Microsoft Teams to a SharePoint team site that uses an Office 365 group to manage its
membership.

SharePoint Online can also take advantage of other cloud services available within Office 365:

Microsoft PowerApps , provides the ability to customize list forms or create standalone Apps that do not
require the author to write any code to create custom forms that are mobile ready . The user only needs to click
on the “PowerApps” action in a list to either create a custom PowerApp or customize the list forms. Ability to
create PowerApps for document libraries is not provided yet by Microsoft. A modern PowerApps Web Part ,
currently in preview, supports the rendering of PowerApp in modern SharePoint pages.
Microsoft Flow , gives users the ability to design and deploy business processes that work with modern
SharePoint lists and document libraries (see Chapter 23 for more information about how to build Flow apps).
Flow includes several SharePoint Online triggers and actions, so the following scenarios are covered by the
service:

A Flow can be started automatically when a document in a document library or an item in a list is added, updated
or deleted.
A Flow can be run on demand for a selected document in a document library or an item in a list.
An approval process for lists and document libraries can easily modeled by means of the approval actions
available in Flow.
Modern pages authored in a modern team site or a communication one can follow an out of the box or custom
approval process before being finally published.
A Flow can be run on a schedule, so it can periodically look for specific information in a SharePoint site and
perform required actions. For instance, the process to send a reminder to review new documents in a document
library could easily be modeled by means of Flow.

Access to the Flow platform and to the Flows created in a list or a document library is
available natively in modern SharePoint lists and document libraries (Figure 8-11 ).
Figure 8-11: Flow integration in a SharePoint Online document library

Moving Forward
Now that we know about how SharePoint Online and OneDrive for Business work, we can move forward to Delve, an
interesting application that’s only available in Office 365. Delve helps us organize and find documents stored in
SharePoint and OneDrive sites and has a couple of other tricks up its sleeve. Let’s delve into those details.

Chapter 9: Delve and the Graph
Tony Redmond

Delve is one of the more interesting applications that exist within Office 365. Some will consider it essential because
of Delve’s ability to find information stored in OneDrive for Business and SharePoint Online sites and attachments in
Exchange Online mailboxes, while others think of Delve as an overrated but pretty search utility. Like any application,
the value you extract depends on how you use the software. Let’s delve into the details.

Mastering Information
The sheer quantity of data accessed by humans has exploded in the last decade. We receive more messages than ever
before, have access to more web sites and pages, and electronic documents have largely replaced paper equivalents
as the basis for most business communications. At the same time, it has not become any easier for people working
within large companies to understand what knowledge existed within the company. A case perhaps of not being able
to see the forest because all the trees got in the way.

When you think about it, an Office 365 tenant is nothing more than a huge data mart with people adding more data to
the mountain every day by creating Word documents, OneNote notebooks, Excel worksheets, PowerPoint
presentations, and other files and storing those files in SharePoint document libraries or circulating them as
attachments to other people within their company. Perhaps valuable information is glimpsed once and then forgotten
in the flood of new data generated daily. It is reasonably common to find that people know about something but have
forgotten where to find the exact information when they need it. And project teams will work on information in
splendid isolation from similar efforts undertaken by other teams in the same company, perhaps even in the same
building. The aim of Delve is to help users master the information that already exists within their company by
exposing information to people who have access to knowledge but might not know that it exists.

Which brings us to Delve, defined by Microsoft Principal Engineer Alex Pope as “the primary profile experience and
destination for Office 365. Delve helps you contextualize, discover and refind Office 365 content about a person .”

In other words, Delve makes sense of the data inside Office 365 tenants to make it personal and valuable to an
individual. Delve delivers on this commitment by using signals and information gathered inside the Office Graph.

Office Graph
Before Delve can help users, it must be able to base whatever it surfaces on facts. The foundation of Delve lies in the
Office Graph, a massive database that uses graph structures for semantic queries with nodes, edges, and properties to
represent and store data. Essentially, graph databases set out to achieve an understanding of how objects relate to
each other. Many graph databases are available and in use today , but we can trace the heritage of the Office Graph,
which underpins Delve, from work done in Facebook to build the Open Graph to store web pages as objects in a social
graph. Yammer built on this work as the Enterprise Graph, which mapped the relationships between people and
information by recording interactions such the posts, replies, shares, and uploads which occur inside a social network
as well as the likes to show the value individuals place on these activities.

Microsoft bought Yammer in 2012. When Microsoft integrated Yammer into their operations, they looked at how to
expand the “signals” consumed by the Enterprise Graph to make sense of the actions taken by people working with
the Office applications to better understand the flow of information that occurs between individuals. These actions
include pieces of work like documents and messages, how sharing occurs between people and groups, who has
worked on what document and when, who sends messages to someone else, access to documents in SharePoint sites,
the instant messages sent by Skype for Business Online, who attends meetings and when meetings occur, and so on.
This work resulted in the Office Graph, which Microsoft announced at the SharePoint conference in early 2014.
Subsequently, Microsoft introduced the Microsoft Graph as a unified programming interface to multiple forms of
information including Office 365, Azure Active Directory, and Windows.

The Office Graph holds information about Office 365 objects (like users and documents) and the relationship that
exists between them (such as Tony has shared Document A with Paul), The data are stored as nodes and edges in a
graph index (Figure 9-1 ). Users and documents are nodes and actions such as sharing and updating documents are
edges. Interactions can be private (for instance, you create a document but do not share it with anyone) or shared
(you then share the document with a team via OneDrive for Business). Sharing is obviously an important signal, but so
are other actions like whom someone converses or meets with on a regular basis. Office Graph captures signals for
these interactions as people use Exchange Online, SharePoint Online, and other applications. The signals form the
nodes and graphs and connect the dots in a way that makes sense. That sounds simple, but it takes a huge amount of
CPU power and sophisticated machine learning algorithms to process and make any sense of the data. The data held
in the Office Graph is then consumed by applications using the Graph Query Language (GQL) and the Microsoft Graph
API.
Figure 9-1: Connections within the Office Graph (source: Microsoft)

The Office Graph does not assemble data drawn from across Office 365 into a single database. Instead, it stores
metadata and relationships about data that lives in other repositories, such as Azure Active Directory. The data stores
used by Office Graph are:

The Tenant Graph store: Bulk storage of data drawn from tenants optimized to allow analytics to be calculated
quickly.
The Active Content Cache : Provides access to active nodes and edges that are used by applications like Delve
for functionality such as “trending around you” or Office 365 Groups to suggest groups to a user that they might
like to join.
The Input Router : Responsible for notifying internal and external components of changes made the graph data
stored for a tenant.

To keep computation close to the source data, components running within each workload perform analytic
calculations and then push the resulting data to the tenant graph. The data in the tenant graph then becomes the
input for tenant-wide calculations. For example, Exchange Online has information about the people that users
communicate with in mailbox recipient caches. This data is an important input to calculate whom is important to a
user and makes up a set of known edges for the user’s graph. Exchange Online collates the data and then pushes it to
the tenant graph where it joins information gathered from SharePoint, Skype for Business, and other workloads to
build up a complete picture of a user’s activity within Office 365.

Microsoft exploits the Office Graph in many places within Office 365. For instance, any time you see a reference to
“trending data” inside Office 365, you can bet that the Office Graph is the source of authority for that information.
Because the Office Graph gathers signals about how people generate and share information between each other and
understands the organizational relationships that exist within a company, algorithms can figure out whether a
document, video, group, or other object might be of interest to a user. Features like Discover View in OneDrive for
Business , the way that Office 365 Groups suggests new groups a user might like to join, and information about
trending videos in the Office 365 Video Portal are all based on feeds from the Office Graph. Figure 9-2 illustrates
another example of the Office Graph in use. In this case, the People Cards used to display personal information about
Office 365 accounts. Contacts are fleshed out with data to show details of how and when the user interacted with
different individuals. As you can see, the information held in Office Graph also spans external recipients and guest
accounts as well as tenant users.
Figure 9-2: How People cards use Office Graph signals

The more interaction someone has with Office 365 applications, the more data about their activities is available in the
Office Graph and the easier it becomes to distinguish important work from merely interesting activities. In the same
way, the more interaction occurs between users within a tenant, specifically in terms of sharing or common work, the
easier it is to understand the paths of collaboration between teams and individuals within the company. Someone who
only stores their work on their personal PC will generate relatively few signals for the Office Graph (email
attachments sent to that person will generate some), so their experience of using Delve will be much less complete
than someone who uses OneDrive for Business and SharePoint team sites to share documents with their team.

The Graph Framework

Apart from its role to collect information drawn from multiple sources across Office 365, the Office Graph is also a
framework for applications that want to exploit that information. In this context, the framework is called the Microsoft
Graph, a collection of multiple APIs that allow programmers to build applications based on Office data. Each type of
data, such as messages, files, contacts, and so on, is reached by a well-known endpoint.

Together, the endpoints allow applications to mix and match data from multiple Office 365 sources. A good example is
how Microsoft Teams combines its own chat service with functionality taken from SharePoint, Planner, Skype, and
OneNote. Another is how MyAnalytics makes use of the Graph to derive intelligence about user activities to help them
work smarter.

Training material about the Graph is available online . You don’t have to write code to appreciate how the Graph
works, but it is helpful to go through some of the lessons to get some background into its functionality.

Delve and Office Graph

The Office Graph laid the foundation to collect and analyze information about how Office 365 users interact with other
people and the applications. Delve was the first application to exploit the information held in the Office Graph. Delve
was the first Office 365 application to use the Graph to succeed in its aim of revealing the most relevant content to
users based on the people with whom they work and the topics that they are working upon. Microsoft sometimes
refers to this as “pivoting on people”, which means that Delve is designed to take a people-centric view of information
(such as what they are working on) rather than an organization-centric driven-from-the-top view as is often
implemented in other systems. Another way of looking at the problem is to consider questions that occur every day
within large companies:

Who has the knowledge to help me with my work?

How can I connect to people across the company to solve the problems I am working on?
How are documents stored and how can I access work that is of interest to me?
What are other people working on?
What could I work on?
Traditionally, email has been the great connector within companies. Email continues to be a very powerful unifying
influence within the networks that bind companies together, but its power is limited by the need to make a first
connection with someone to “discover” that person and to understand what they do and how a relationship might be
mutually beneficial. Once people connect, email is a fantastic way to develop and enhance the relationship by sharing
information on a one-to-one (person) or widespread (group) basis. However, new entrants into a company must make
their own connections to become effective and that can often be a real challenge.

Delve presents information to users in two ways that help solve the problem. First, Delve calculates which users in the
Office Graph are most relevant to the current user. Second, Delve retrieves and presents the most relevant content
available to the user based on their interaction with those users, saving users the time and effort of having to go and
check repositories such as SharePoint team sites or the OneDrive for Business sites of other users to find whether
new and interesting information is available. Because Delve knows what is interesting to you, it has a role in breaking
down silos that occur deliberately or by accident by exposing information to you if your access allows you to see it.

Delve appears as an app in the Office 365 app launcher, and users can access it through a web browser or one of the
Delve apps. Delve is one of the "next generation" portals built inside Office 365 that Microsoft hopes will allow users
to make better use of information. The Office 365 Video portal is another. These portals both have a tight connection
to the Office Graph. In some ways, the role of Delve is to be the "search and discover" facility for Office 365. Many
people who work in large organizations will say that huge amounts of information are available to them, if only they
could find what they are looking for. Delve shows users the files that they have worked on most recently, which is a
great help in keeping track of things. Two other functions help too. First, Delve’s uses the Search Foundation to find
information, which makes Delve the easiest way to search across SharePoint and OneDrive sites, including those that
are on-premises (the SharePoint hybrid search capability allows the sharing of metadata from on-premises servers
with online services, thus exposing those items to Delve). Second, humans often remember the person who shared
some information with them rather than the exact details of the shared information that. Delve allows you to navigate
to that person’s profile and explore what they have been working on. If that person has shared something with you,
Delve will show it to you.

Delve never extracts content (like the text of a document) from the repositories where data exists. In Instead, Delve
presents users with a view (the cards) onto information that might be of interest together with links to connect to the
information. Delve calculates the files shown under a user’s “Activity” by the actions taken by that user (such as the
files they store in OneDrive) and whom they share information with through access to SharePoint libraries, as well as
explicit sharing of items stored in OneDrive for Business libraries, and Exchange Online email attachments. The
people with whom an individual user works can be figured out by observing the interactions between users, the sites
they share, the email traffic between them, and the meetings they attend. The information held by the Office Graph
gradually decays over time, so the decisions reached based on its contents reflect the current state rather an
historical trends and interests.

Delve is available to all Office 365 tenants with enterprise plans (E1 to E5 and their academic or government
equivalent). Delve is not available to tenants that use the standalone SharePoint Online plans.

Enabling Office Graph

The Office Graph runs in the background for all tenants to gather the data used by Delve. Before anyone can the
features of Delve, you must enable the Office Graph for the tenant. Or rather, if you do not want to use Delve and the
Office Graph, you must disable it because the default state is for Office Graph to collect signals for a tenant. To
change the Office Graph state, open the Settings section of the (old version of the) SharePoint Admin Center, and
then enable or disable access to the Office Graph as shown in Figure 9-3 . You can’t update this setting through the
new version of the SharePoint Admin Center (yet).

Figure 9-3: Enabling access to the Office Graph

Delve Browser App

The user interface of the Delve browser app (Figure 9-4 ) is a dashboard divided into a navigation pane to the left and
a large viewing pane to the right. From top to bottom, the navigation pane has several links to navigate Delve:

Home : The entry point to Delve, which reveals the files that Delve believes to be of most interest to the user
based on who has been working on items of shared interest, recent documents shared with the user, and so on.
Files sent to the user as email attachments also appear here.
Me : The idea is to give users an easy-to-use way to get back to their work or what they might be interested that
others are working on. Several tiles are displayed:

Go back to recent documents : Lists all the documents and files that the user has accessed most recently, but
only if the files are in SharePoint document libraries or OneDrive for Business sites. Items exposed here include
files stored in the document libraries belonging to Office 365 Groups. Shared notebooks also show up here as do
videos in the Office 365 Video Portal that the user has watched. Initially, a small number of the most recently
accessed files are listed but clicking “See All” expands to a much larger set.
See what people are doing : Lists documents and files that are being worked on by people with whom the user
interacts. This is section is intended to highlight new information that the user might not know about that they
might have an interest in seeing. You can also click on someone’s avatar to see information about their activities.
Update Profile : Displays personal information about the user fetched from their Office 365 profile.
Organization : Gives a graphic view of the reporting relationship for the user within the organization. This
information comes from Azure Active Directory.
Blog : Access to the user’s personal Delve blog. The idea of people writing and publishing blog posts through
Delve seems to be less popular with Microsoft than it once was. However, the functionality is still available.

Favorites : This view lists favorite “boards,” a navigation mechanism for Delve (we will discuss this concept
shortly) and documents tagged as favorites. Users can mark their most important documents as favorites so that
they are always nearby. Documents shown as favorites can only be in SharePoint Online or OneDrive sites. You
cannot mark a file on a local workstation, a file share, or SharePoint on-premises as a favorite.
If a user has a license for MyAnalytics , an “Analytics” link also appears to allow the user to access their
personal dashboard. MyAnalytics is a separate application described in the companion volume.

Logically, because Delve’s mission is to present information that is important to an individual, the dashboard seen by
one user has different information to that shown to anyone else. Delve calculates the data used to populate each of the
parts of its interface using canned queries against the Office Graph. Thus, the list of people shown are those whom
the user has interacted with recently based on signals such as email, meetings, or document sharing. Photos might be
missing for some of the users, which is a sign that they have not updated their profile by uploading a photo.

Apart from the links described above, the navigation pane has two other kinds of navigation to help users to get to
data quickly. The “People ” link lists other people whom Delve considers the user to work with closely (based on the
signals gathered in the Office Graph). Only users within the tenant show up here. Clicking on the link for a person
brings you to their Delve page and displays their profile and documents on which they have been working. However,
only documents that the user has permission to see are surfaced here. The second navigation aid is a set of links to
Delve boards. You can think of a board as a tag that you can add to documents to form a collection of documents that
have the same tag. We will discuss how to create and use boards shortly.
Figure 9-4: The Delve browser app

Delve has no role in compliance or the retention of information. It is purely a means of displaying relevant information
to users to encourage them to use that information more productively. Compliance issues such as data loss prevention
or the preservation and retention of information depends on the mechanisms used by the underlying applications. As
items age and no activities occur for those items, they lose relevance to Delve and are replaced by other items that
might be more interesting to the user.

Over time, as SharePoint document libraries and OneDrive personal sites are populated, more information will
become available and Delve becomes more useful to the average user, especially when they need to search to find
files. The key point to remember is that Delve cannot find anything if nothing exists in the target repositories available
to the user, and it cannot reveal anything if the user does not have permission to see content.

Delve Search
A major benefit of Delve is its ability to search Office 365 for relevant data, especially documents. Of course, like
anything else in Delve, if users do not store information in places where Delve can find it, nothing will ever turn up. It
is quite common to meet this situation at the start of Office 365 projects where the first focus is often to move on-
premises mailboxes to the cloud and nothing happens to convince users that they should be using SharePoint Online
and OneDrive to store business documents (including the document libraries used with Office 365 Groups and Teams)
instead of the file shares or personal storage.

The Search Foundation is very important to Delve because Delve depends on its content indexes to find information.
The Search Foundation began life as technology obtained through the acquisition of Fast Search and Transfer ASA
(FAST) in 2008 . The Search Foundation updates the content indexes for data created through applications like
Exchange and SharePoint (the on-premises versions of these products use the same technology) and the content
indexes are available for use by any Office 365 application.

The Search Foundation content indexes come from the content in documents and messages. Usually, searching
against the content indexes is efficient and usually very effective in terms of the results returned. However, users can
dramatically increase the chances of finding a specific document if they take the time to update document properties
like title, tags, and comments to include likely search terms. For instance, including “personal dashboard” as a tag for
the document file for this chapter might make it easier to find the file. Unfortunately, users are not good at populating
document properties and the names given to files are often strange, misleading, or quixotic, so finding files can often
be a challenge.

Indexing does not happen at once when a user adds a document to a library or updates an existing file, and it can take
a little while before a new item appears in Delve. Not all content that you expect might show up within Delve, but
there is usually a good reason why not. For example, if no one has accessed a document in the last three months, it is
unlikely to be an item of immediate interest to someone and therefore will not appear. In addition, Delve only supports
Word, PowerPoint, Excel, Notebooks, and PDF files as well as video files uploaded to the Office 365 Video portal.
Anything outside these boundaries does not appear in Delve, including common types such as Microsoft Project plans.

Possibly because users find it more difficult to find documents as tenants accumulate more information in OneDrive
for Business and SharePoint Online sites, Microsoft rolled out “intelligence-powered search” in February 2017 as part
of the transition to a new version of Delve. At the time, Microsoft said that the content indexes for Office 365 grow
10% month over month. We can interpret this to mean that more people use OneDrive for Business and SharePoint
Online sites to store documents instead of local drives. This is exactly the kind of mindset change that you want to see
if you want to maximize the use of Office 365, including search. However, the downside is that the swelling volume of
unstructured and ill-organized data can make it even more difficult for users to find information. It is a classic
example of looking for a needle in a haystack.

To solve the problem of how to find the right information among so much data, Delve combines the raw content
indexes with intelligence derived from the Microsoft Graph to generate more precise and focused search results.
Instead of returning results that match input queries, Delve filters the results to extract information that is most
relevant to users. Delve takes factors such as who created documents, the site holding the documents, and your
relationship to those people and sites into account when presenting search results. The idea is that you should see the
most relevant information first followed by less relevant content as the search progresses.

Searches start when a user types some characters into the search box as Delve begins to evaluate search terms and
presents information as the search returns results. Figure 9-5 shows what you can expect to see. In this case, we
enter “Office 365” as the search term. The response highlights a mailbox, some documents, and two boards. The
Office Graph heavily influences the results shown here, so the documents listed are usually the most recent matching
items the user has accessed. This makes sense because in many instances, users are interested in recent work.
Figure 9-5: Delve displays initial results

If the quick search results do not turn up the desired information, the user can go on to a complete search by pressing
the Return key or clicking the blue arrow to the right of the search box. Delve then starts a more exhaustive search,
which returns results like those shown in Figure 9-6 . Again, we look for items matching “Office 365” and Delve
responds with some users who might match the search term, including members of the author team and the mailbox
used for reader feedback. If you want to limit search results to documents (a generic term that also spans
presentations, spreadsheets, and so on), you can opt for “Documents” instead of “All result types” in the drop-down
choice in the menu bar. Much the same approach is used in with how Search works in the SharePoint Online home
page, the difference being that SharePoint Online does not include boards or people in its search results.

Clicking a user icon brings you to the user’s Delve page and shows you the documents they have worked on most
recently. To the right, we see a set of boards where matching documents exist. Below the users, we see a list of
documents returned by the search. Selecting a document in the list launches an online viewer to reveal its content
while clicking the location or person link to the right brings you to the storage location. Remember, Delve only shows
information to a user when that user has the permission to see it.
Figure 9-6: Delve search results

The integration of Office Graph into Delve search delivers more precise results than the earlier implementation.
Intelligent search does not mean that users can continue to add rubbish to Office 365 and expect Delve to make sense
of what people dump into SharePoint or OneDrive. Paying attention to document titles, properties and tags is still
critical to search precision and the ability of users to find documents quickly and easily.

Delve User Profiles

Apart from exposing details of a user’s recent activity, the Delve browser interface (Figure 9-4 ) includes information
about the user’s profile, which allows anyone who navigates through Delve to the page to discover information about
the user such as their job title and contact details. The organizational structure reported in the profile is based on the
information held about managerial relationships in Azure Active Directory while the information about other people
comes from interactions with other users as recorded in the Office Graph. The profile information shown here appears
in other places within Office 365, such as when someone views author information in SharePoint and OneDrive for
Business document libraries.

Users can update their Delve profile through Update Profile , which exposes a set of properties defining the contact
information for the user. Some of these properties are updatable and some are only editable by an administrator.
Contact information is defined as properties in the User Profiles section of the SharePoint management console. The
Delve profile only uses a small set of the available properties. Administrators can select which of the properties are
visible through the User Profiles section of the SharePoint Admin Center,, including properties that are updatable by
users, and the properties that are locked down. For example, the organization usually assigns the number for a user’s
Work Phone and the user cannot change it, so it doesn’t make sense to update that property in the Delve profile. On
the other hand, the user owns their Home Phone and is therefore writeable. The same logic holds true for the Office
and Office Location properties, where the latter serves as a user-friendly guide to the location of someone in a
building (for instance, “behind the large plants on the 2 nd floor”).
Figure 9-7: Updating personal information for a user

Tenants can define custom properties and make them available in user profiles. However, a critical point to
understand is that all writeable properties only update SPODS (the directory used by SharePoint Online) and changes
to these properties do not synchronize back to Azure Active Directory. The Delve profile also allows users to enter
some information about themselves (Figure 9-7 ). This information divides into:

About Me: A free-text personal description.

Projects.
Skills and expertise.
Schools and education.
Interest and hobbies.

Delve does not apply any checks to any of the information input and it is entirely up to the user what they choose to
include. All the information entered here is visible to other users when they view the contact details for the user.

Content cards
Delve show information to users through content cards. You can’t control the appearance of these cards or what
information they show. As you can see from the left-hand side of Figure 9-8 , a content card gives some added context
for the user to understand whether the object represented by the card is interesting to them. If present, a graphic
extracted from the item is used to highlight content along with the author name and picture. If several graphics are
available in an item, Delve selects the best graphic based on resolution. The author picture is from their Delve user
profile, which coms from the photo data in their Azure Active Directory account. The name of the item used in the
card comes from information inside the file, such as a Heading 1 title used in a Word document or the title used for
the first slide in a PowerPoint deck. Information about the item's location is also provided (for instance, the name of a
SharePoint library or a OneDrive for Business site that holds a document), together with a visual sign to show if the
item is associated with any boards. In this case, we see the number 1 beside the boards icon, so we know that the
document is tagged for a single board. The “titbit” or hint at the top of the card tells the user why Delve is showing
them this information. It could be that someone has recently updated a document or that some of your colleagues
have worked on a document.
Figure 9-8: Delve content cards

Cards for files stored in SharePoint and OneDrive sites have a count of views, which increase as users access the file
or because of edits to the document by the author. At first glance, an item with a high number of views might seem to
be very popular, but it could be the case that the views accumulate because of frequent edits by the user and no one
else has ever been near the file.

The ellipsis menu for a card (in the lower left-hand corner of the card) exposes some more actions. The user can send
a link to an item, copy the link to the clipboard (the links open the item using an online app, if available), chat about
the item in a Yammer group (if the tenant uses Yammer), or see who has access to the document. This choice takes
you to the access control list controlled by the native application and shows you to whom Delve might display the item
in a document view.

Some like the card-centered user interface, others would prefer a more traditional list interface. As it happens, if you
want to create your own interface you can build it on top of the GQL API. Many examples of using GQL to create new
interfaces to explore the kind of information handled by Delve are available online, including an interesting project
that uses Cortana to access Delve.

Delve Boards
Beneath the navigation links in the left-hand side of Delve’s interface, you might see entries for some "Boards", which
are an easy and convenient way for users to mark items as being of interest for some reason. Put another way, boards
allow users to assemble collections of items that belong together, no matter where they are stored and managed
within Office 365. The collection might be temporary, as in the case when a board is used to focus attention on items
of interest for a certain meeting, or have a more permanent nature, as when a board is used to tag all the items
accumulated during a long-running project. Boards also allow users to reuse and build upon information, as in the
case when you create a board for a new project and incorporate items from other boards to serve as the basis for
discussion.

Figure 9-9 illustrates a practical example of boards in use. We can see a set of boards in the left-hand navigation pane,
including those that I use to organize documents associated with projects such as my blogs, travel arrangements, and
presentations. Clicking a board reveals all the files associated with that board and so allows a board to become a fast
shortcut to the files that belong to a specific project or area of interest. In this case, the board reveals the set of
documents relating to a blog. A link to a board can be emailed (with the Send a link choice) to tell fellow workers
about the information tagged in the board. When the recipient uses the link, Delve reveals all the documents in the
board that their permissions allow them to see (some items might be restricted).
Figure 9-9: Documents listed on a Delve board

Boards allow users to gather collections of associated items together without having to seek help from the IT
department, who might otherwise be called upon to create a specific site to hold information belonging to a project or
other initiative. Items tagged with boards can come from many of the Office 365 repositories – team sites, group
libraries, or their personal OneDrive site. Items stay in place in their source repository and can appear in multiple
boards which, in effect, allows users to categorize information to reflect the changing needs of the organization
without the help of a professional knowledge manager or curator. After a user selects a board to view information,
Delve only shows them the items that they can access, even if many other items are tagged with the board.

You can only add an item to a board if the item’s card displays a board icon. File types that supports boards includes
Word, Excel, PowerPoint, OneNote, and PDF (Microsoft is working to expand the set of supported file types). To add
an item to a board, click the boards icon (for a card) to reveal a dialog where you can enter the name of a board. As
you enter the name, Delve shows a list of recently-used boards. However, you can input whatever name they like.
Boards are created by users and there is no administrator intervention to control the boards that are used or to create
boards on behalf of a user. A board assigned to an item by a user becomes an attribute of the item and is visible to
other users. When you add an item to a board, you can click the board name to find all the other items tagged with
that board name. Users can tag an item with many different boards to reflect the way that users categorize and
organize their information.

Board curation : Delve does not include any features to allow users to manage the boards that they create or for
administrators to later fix any problems that might be introduced by users (like spelling mistakes) or to impose a
taxonomy for cards across the tenant. Microsoft said that they might address this need by listing the boards created
by a user in their profile and allow boards to be created, renamed, or removed there. Although a common request
from administrators, especially those dealing with large tenants, Microsoft has given no sign whether it will be
possible to exercise administrative control over boards.

When you add an item to a board or create a new board, Delve automatically adds you to the list of followers for the
board. Following a board is a way of gaining quick access to the contents of the board as a list of the most recent
followed boards always appears in the left-hand navigation pane. Clicking Favorites in the navigation pane will bring
to you a page where all your favorite boards and documents are listed. To find boards that have been created
previously, start typing in the Search box. Delve will list individual documents that match your query along with
boards whose name matches the query.

If you’re not interested in following a board or want to free up space for a board that you think is more important, you
can use Remove from Favorites (available when viewing the contents of a board). Conversely, if you find a board
that seems interesting, you can use Add to Favorites to start following it.

Anyone can remove an item from a board by hovering over board name on the item’s card and clicking the X. This
removes the selected item from all boards. Removing an item from a board does not happen at once for all users as
the change takes time to ripple across the system.

Above the set of boards, you see a listing of other users (from the same tenant) that the Office Graph believes to be of
interest because they have had some interaction with the user, perhaps as an email attachment or through common
membership of a document library. Clicking a user’s avatar navigates to that person’s Delve dashboard and reveals
information that they have worked on that you can access. Another tile on the dashboard lists documents that others
“working around” that person own that might be of interest. Again, you won’t see anything here unless you are
allowed access to the information. The Office Graph acquires information about new users from Azure Active
Directory daily, so it might take a little time before a new user shows up in this list, but they will if they share
information with you.

Making Delve your Start Page

When a user signs into Office 365 via a browser, the default action is to display the Office 365 landing page
(http://portal.office.com). But we all have different working habits. Some people like to start every day in Outlook,
while others will want to see their calendar to understand the outline of the day ahead. As Delve becomes more
capable of surfacing information drawn from across Office 365, it makes sense to consider whether Delve should be
your home page. If you’d like to do this, click the cogwheel symbol in the Office 365 menu bar, select Office 365
settings , then select Delve as the Start Page , and then Save . The next sign in to Office 365 will bring the user to
the Delve Home page.

Privacy and Security

The mission of Delve is to bring pertinent information to users. This aspect of the application leads some to worry
about inadvertent information leakage. In this respect, it is critical to realize that Delve never reveals any document
or other information to a user that the user is not already entitled to see. Delve does not change the permissions
assigned to a document by its owner and enormous care is taken to ensure that access is controlled by the security
settings managed within applications. For example, you won't see an attached document unless you received that
attachment in email or you won't be able to see a document in a SharePoint library unless your Office 365 account has
permission to access that library. Today, only users with Office 365 credentials for your tenant can use Delve, so
there's no chance that someone outside your tenant can use Delve to gain unauthorized access to information

The impersonation question : SharePoint Online supports the ability to write code that executes in the context of
another user’s identity (impersonation). When code that uses impersonation processes documents in libraries, the
Office Graph notes this fact but registers the name of the person running the code rather than the impersonated user
as the account that last processed the document. This can lead to interesting results in Delve where someone
appears to become a hyperactive user and the people who apparently accessed documents do not show up at all.

Delve never presents information to a user if they cannot see that content. However, it is possible that people will see
information through Delve that the authors might prefer not to share. This is through no fault of the application.
Instead, it underlines the importance of setting and updating appropriate permissions on document libraries and
other sources of information so that they can be properly harvested by the Office Graph.

Because Delve only ever shows a user information they can see, if you visit a co-worker’s page (click on one of the
people listed in your page), the cards revealed by Delve belong to files that you can access rather than the complete
set of information available to the other user. You do not see all the documents belonging to your co-worker nor do you
see cards that Delve considers relevant for them. The search query executed by Delve when you navigate to the other
user’s page takes your credentials into account when it calculates what cards to display.

Some unverified reports claim that Delve has revealed documents to someone when they should not have been able to
access the information. Invariably, the root cause of these incidents proves to be permissions set on the document or
the hosting site in such a way that the user can view the content. Delve always trims the results shown in its views to
remove items blocked from a user by inadequate permissions. As Delve becomes better known within an organization,
perhaps it will be a catalyst for a review for how administrators secure SharePoint Online sites through permissions
plus the accounts that can grant administrative access to site collections.

Real World – Administrator permissions and Delve : tenant administrators are usually administrators of
SharePoint site collections. As such, these accounts have access to all documents stored in all sites. If these accounts
are used by administrators for their day-to-day work, Delve will surface information to them drawn from sites across
the tenant. Although Delve will filter this information to only present items deemed to be relevant to the user, a high
likelihood exists that some confidential data will eventually appear. Again, Delve is only following the rules – these
accounts have access to the information through their site collection administrator status, so it is OK to show the
user any file in the site collection. For this reason, it is best to ask administrators to only log in and use highly
permissioned and privileged accounts when they need to perform administrative operations and to use lower-
permissioned personal accounts to work with Office 365 applications.

Disabling Delve’s Ability to Show Documents Authored by a User

Some people might be uncomfortable at the prospect that Delve might reveal documents they work to other users.
Informing users about documents, videos, and other files generated by co-workers is a major part of what Delve does.
The intention is to make information more discoverable and available within the organization. Delve only exposes
documents and other data to other users if those people can view the information. Users receive that authority
through their access to the SharePoint site holding the data or because someone has explicitly shared a file with them.
Even so, a user who works with confidential information might be happier if they can block any possible disclosure
through inadvertent exposure through Delve. For instance, those working on a document describing a new salary
structure for the company might well prefer the existence of that document to remain a tightly guarded secret until
the time comes to reveal the new structure to employees.

Figure 9-10: Delve Sharing Options

To prevent Delve showing their documents to other members of the tenant, users can disable document sharing. To do
this, go to the cogwheel menu (Figure 9-10 ), and move the Documents slider from On (the default) to Off . This does
not disable document permissions that are in place. These permissions stay in place, and users can access documents
through normal sharing mechanisms or through their access to a document library. All the slider does is control the
ability of Delve and OneDrive for Business to show documents to other users if the Office Graph decides that those
documents are “of interest” based to those users. The set of Delve options shown in Figure 9-10 includes MyAnalytics.
You will see a different screen if your tenant does not have the necessary licenses for this application.

Even after someone disables document sharing, Delve continues to make other information available to the user, such
as the organization view and people with whom they work. However, the experience and usefulness of Delve is much
reduced. For example, if the user selects one of the people listed in the navigation pane, they will not be able to see
documents authored by that person and shared with the user because that view depends on documents sharing data
recorded in the Office Graph. Although the Office Graph will stop tracking document activity for the user, any
information previously recorded stays in the Office Graph, which means that Delve might reveal those documents to
other users until the Office Graph flushes the information.

In addition to users selectively disabling the sharing of their documents through Delve, administrators of SharePoint
sites can place a filter over what the documents the site reveals to Delve. You might have situations where complete
document libraries or specific documents have content that you want to remain invisible except when accessed
through the document library. As explained by Microsoft in an online post (or this post for an independent view on the
topic) , a site column called HideFromDelve can be used to mark the library or documents so that they never show up
in Delve results. The documents are still indexed by Search and are available for eDiscovery purposes. They also stay
visible to applications that query the Office Graph.

It is good to be able to control document sharing activity on a user-by-user basis, but this does not satisfy some
customers who wish to control the gathering of signals by the Office Graph and limit the collection of the wide range
of signals to track how people interact with Office 365. This data can be interrogated by applications, like Delve, via
the Microsoft Graph API. Some have voiced concern that the Office Graph gathers too many signals about user
activity and some customers have requested Microsoft to allow organizations to control the type of signals gathered
for a tenant. A smaller and less comprehensive set of signals might lead to poorer determinations of relevant
information for applications like Delve to show to users, but the belief is that this is an acceptable compromise to take
when personal privacy is involved. Microsoft has not said whether they will deliver a method to customize the signals
gathered by Office Graph for a tenant, so the question is still open.

Using Graph APIs : The REST-based Microsoft Graph API and the Graph Query Language (GQL) are available for
third-party developers to access the contents of the Office Graph database. You can get information about developing
against the Microsoft Graph online . An example of the kind of applications that can be developed against the Office
Graph can be seen in the “ MyWorld” application published by AvePoint.

Delve and Exchange Online

When Microsoft first made Delve available, only documents stored in SharePoint Online and OneDrive for Business
were exposed to users. The ability to see and search for documents proved the usefulness of Delve and has been
expanded over time to include other types of files like videos, but the fact remains that a great deal of information
circulates as email attachments in all sizes of companies. A fair case can be made that attaching a file to a message
was the first rudimentary form of sharing and that it is still the easiest and most common way for users to share
information with their colleagues today. Tenants using Teams probably find that fewer attachments circulate in email
because more files are shared in team channels, but for most of us, email attachments are a quick and convenient way
to share information with others, which is why Delve tries to expose interesting information found in attachments
stored in the user’s mailbox.

Delve's support for Exchange Online is based on the signals collected in the Office Graph. Some recent messages that
have attachments are displayed in the document view in the Home page. Not all messages with attachments appear
because Delve applies the same tests for relevancy to decide what should appear. In effect, Delve is filtering your
Inbox (for this is the only folder from which messages are selected) to figure out what messages are most important to
you taking all the signals gathered by the Office Graph into account. You might be surprised at the choice made by
Delve, but that's part of the delight of Delve as it might unearth something important that you had completely
overlooked as you processed the Inbox.

Figure 9-11: A Delve email card

Like any other object that appears in Delve, cards display information about email attachments. Figure 9-11 shows
what you can expect to see in such a card. In this case, the card is for a PowerPoint attachment I sent to someone for
their review.

Two methods exist to access the content of the message. You can click the message link icon (lower left-hand corner)
or the message subject. In either case, Delve launches a modified version of OWA to view the message and
attachment. OWA can display any file format supported by Office Online (including Adobe PDF) in this manner.
Although Delve doesn't create a full OWA session to view a message and you can't get to a folder list or access other
messages from this point, you can perform the following actions with the selected message:

Reply and Reply All (including reply all with a meeting request)
Forward
Delete
Mark as Unread
Print
Flag
Assign a retention policy
Print
View message details

The card for an email attachment does not offer the ability to add the item to a board. This is by design as
attachments cannot be added to boards. Although it might seem like a good idea to be able to tag an attachment to a
board, attachments exist inside user mailboxes. Attachments are therefore personal and cannot be shared in the same
way as files stored in SharePoint or OneDrive sites.

An Inbox can be quite a dynamic place and if you move a message to another folder (or the message is moved by the
Managed Folder Assistant because of a retention tag) then the data held in the Office Graph might not point to its
current location, which in turn causes OWA to fail when it tries to display the message and its attachment. In this
past, when people commonly moved items out of the Inbox to other folders, this might have been more of an issue.
Today, when people often leave everything in the Inbox (the “piler” syndrome), it is not a problem.

Hybrid Delve
Delve and the Office Graph become increasingly useful as the number of data sources available to them grows.
Microsoft has provided the Office Graph with the ability to consume signals from on-premises repositories deployed in
hybrid tenants. The Hybrid Cloud Search service is available as an add-on for SharePoint 2013 and is included in
SharePoint 2016. The service crawls on-premises content sources and transmits metadata to SharePoint Online. The
on-premises data is incorporated into the search index maintained by SharePoint Online, which acts as the
coordinating authority for searches. Users must connect to SharePoint Online to search across on-premises and cloud
resources. The metadata will also be made available to the Office Graph and so be exposed through in Delve. If a user
wishes to access an on-premises document that surfaces in Delve, they are redirected to the on-premises library to
access the file from there.

Although hybrid connectivity already exists for Exchange, the hybrid connection is not designed to send metadata to
the Office Graph. In any case, it would be unreasonable to take the same approach for email attachments. Only users
with cloud-based mailboxes can see attachments in Delve that they have received. Receiving an attachment is a very
personal interaction that cannot be compared to working with documents in a shared library. The only sharing that
has taken place is between the sender and the recipient, which is why Delve only surfaces email attachments in its
personal “Me” view.

In addition, Delve only displays attachments that it considers most relevant to the user. The decision to display one
attachment instead of another is taken based on the totality of the signals held in the Office Graph. For instance, an
attachment from an external correspondent (even one using an on-premises mailbox in a hybrid organization) might
be totally ignored by Delve while one sent by another person is surfaced. The decision is driven by the collection of
signals held in the Office Graph so the attachment from the hybrid mailbox is overlooked because no other signals
exist to show its relevance while the other attachment is considered more relevant because it can be connected to
meetings, Skype for Business Online calls, and other signals about a related project.

Delve Apps
Delve apps are available for Apple iOS and Android mobile devices. Although the card-based interface used by Delve
takes up a reasonable amount of screen real estate, the mobile clients are very usable. Versions are available in
multiple languages and like the web client, Microsoft enhances the Delve mobile clients often to support new features
as they become available. Figure 9-12 shows the iOS version of the client positioned in “My Work,” roughly equivalent
to the files shown in the “Me” section of the browser client. Clicking on a file invokes a viewer, and if you want to edit
the file, an app (if available). For instance, if the Delve for iOS app shows a Word document, you can edit it with Word
for iOS.
Figure 9-12: The iOS version of the Delve mobile app

Automating Office 365

In the early days of Office 365, the basic workloads brought their own approaches to automation to Office 365. After
its introduction in Exchange 2007, the focus for Exchange was PowerShell, which quickly became part of the
Exchange administrator’s toolkit as a powerful way to automate common operations. PowerShell is great at
automating common administrative tasks but is extremely limited when dealing with user content. To close this gap,
Exchange Web Services (EWS) supports programmatic access to data in user mailboxes. EWS is a very effective
method to access Exchange data, as you can see in the many code examples available on the Internet, including the
repository managed by MVP Glen Scales.

SharePoint also supports PowerShell, but the coverage of the SharePoint PowerShell module is less comprehensive
and powerful than its Exchange counterpart. For this reason, developers usually prefer the SharePoint PnP ( Patterns
and Practice ) initiative when they want to extend SharePoint functionality. For more information about SharePoint
PnP, see the discussion in Chapter 8.

The Microsoft Graph

Office 365 is a different environment from the familiar on-premises world you’re used to. PowerShell plays a key role
in the automation of Office 365 administrative tasks, EWS is available to access mailbox data and PnP continues to
flourish. However, for programmatic access to both user and system data, Microsoft’s focus is on building a common
access approach to all manner of Office 365 data from Exchange and SharePoint to Teams and Planner. This is the
Microsoft Graph, a single REST-based API to interact with Office 365 and Azure data via multiple endpoints. An
endpoint is something like mail, contacts, calendar, or groups. All the calls are via standard HTTP requests, which
means that programmers can use a wide variety of programming languages. For example, GET requests fetch data
while POST requests write data.

The Microsoft Graph is now a critical part of Office 365. It is used by Microsoft developers to create new applications
like Teams and the mobile apps for many workloads within Office 365. The Microsoft Graph also includes data from
outside Office 365, such as Azure Active Directory and Microsoft’s consumer cloud services, meaning that it is
possible for programmers to mix and match data drawn from multiple sources within the same application. Microsoft
has deliberately made the Graph approachable by including support for a wide choice of development technologies.
By using the Graph APIs, programmers can do the following:

Access data from multiple applications, such as Exchange Online and SharePoint Online.
Traverse data accessed by users to be able to track their usage of the service.
 Understand data trends to be able to find data that is important to the tenant or groups of users.
Build new tools and applications using Office 365 data.

As programmers start to use the Microsoft Graph to build applications on top of Office 365, Microsoft hopes that they
will view Office 365 as a complete fabric than limiting their vision in the same way as they would when working with a
single application. This is the approach taken within Office 365 to create applications like Teams.

The Microsoft Graph Explorer

Full documentation about the various endpoints available to Office 365 data through the Microsoft Graph is available
online . You will notice that some of the endpoints are marked with a version number (such as V1.0) and some are still
marked as “beta”, showing that these endpoints are still under development. Of course, there is nothing like using a
tool to get to know what it is capable of. To understand the capabilities of the Microsoft Graph, you can fire up the
Graph Explorer , a web console that allows you to execute different API calls against real data. The idea behind the
Graph Explorer is that it helps programmers understand the kind of calls they need to construct to interact with
various Office 365 endpoints and the format of the results returned through their interaction.

You can select “V1.0” or “beta” from the drop-down list to move between the sets of supported calls. The field holding
the selected call offers an IntelliSense-like auto-complete capability, so you can click it to reveal the calls available
from a specific point. For instance, assume that we select https://graph.microsoft.com/beta/me/ as a starting point. If
you now click the field, you see a list of calls organized (somewhat alphabetically) starting with
https://graph.microsoft.com/beta/me/businessPhones and then https://graph.microsoft.com/beta/me/city and so on.

Although you can use the Graph Explorer to examine data in a test tenant, the experience is more forceful when you
authenticate your account. When this happens, the Graph returns the information that the authenticated user would
see. After you enter a specific query or select one of the “Getting Started” examples, click Run Query to have Office
365 return the requested data. For instance, Figure 9-13 shows the results of a call to fetch data from
https://graph.microsoft.com/beta/me/insightstrending . “Insightstrending ” means the set of documents that the user
sees in the “Popular Content” section of Delve. Whenever you see “Me” used in a Graph call, you know it is a
pseudonym for the logged-on account.

A single authentication is enough to access all user endpoints, but you need to change permissions to gain
administrative access to others, such as the set of Office 365 Groups known in the tenant. You know when you need
access rights to access an endpoint because you will see the standard 401-access denied response when you try to
access data. The solution is to edit the permissions used with the Graph Explorer and then log-on again to create a
new session with the extended permission set.

Figure 9-13: Microsoft Graph Explorer

Apart from the obvious calls to play with, such as https://graph.microsoft.com/v1.0/me/messages (list messages in
your Inbox), other interesting endpoints to investigate are:

https://graph.microsoft.com/beta/groups : List all the groups in a tenant.

https://graph.microsoft.com/beta/me/joinedGroups : List all the Office 365 Groups that the logged-on user
belongs to in a tenant.
https://graph.microsoft.com/beta/me/joinedTeams : List all the Teams that the logged-on user belongs to in a
 tenant.

When you list sets of objects like Groups or Teams, you see an identifier for each object. You need the identifier (a
GUID) to navigate to the next level. For instance, this URL includes a group identifier
https://graph.microsoft.com/beta/groups/72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 , If you include the URL in a query
executed by the Explorer, you get back the properties of the group in JSON format. For example:

"@odata.context": "https://graph.microsoft.com/beta/$metadata#groups/$entity",

"id": "72ee570e-3dd8-41d2-bc84-7c9eb8024dd4",

"deletedDateTime": null,

"classification": "External Access",

"createdDateTime": "2016-09-29T18:37:00Z",

"description": "Exchange Grumpy Old Men (and Women too)",

"displayName": "Exchange's Grumpy Old Men",

"groupTypes": [

"Unified"

],

"mail": "ExchangeMVPs@office365itpros.org",

"mailEnabled": true,

"mailNickname": "exchangegoms",

"membershipRule": null,

"membershipRuleProcessingState": null,

"onPremisesLastSyncDateTime": null,

"onPremisesProvisioningErrors": [],

"onPremisesSecurityIdentifier": null,

"onPremisesSyncEnabled": null,

"preferredLanguage": null,

"proxyAddresses": [

"SMTP:ExchangeMVPs@office365itpros.org",

"smtp:exchangegoms@office365itpros.onmicrosoft.com"

],

"renewedDateTime": "2016-09-29T18:37:00Z",

"resourceBehaviorOptions": [],

"resourceProvisioningOptions": [],

"securityEnabled": false,

"theme": null,

"visibility": "Private"

You can see that the set of properties returned for a group is different from the set returned by the PowerShell Get-
UnifiedGroup cmdlet. This does not mean that the Microsoft Graph is not as good as PowerShell. Instead, the
capabilities available through PowerShell and the Microsoft Graph are very different. Where PowerShell deals with
administrative tasks, the Microsoft Graph handles some administrative tasks, but its major use is in the manipulation
of content. In the case of Groups, you can use the Microsoft Graph to read and write conversations, files, and the
shared calendar, none of which PowerShell can access. And in some cases, like Planner and Teams, applications have
limited or no PowerShell support apart from that available for Groups, and the Microsoft Graph is the only way to
interact with their content.
Clients
Delve is a very useful part of Office 365 to help users find information. Gaining access to that information is our next
step, so let’s talk about the various clients available to Office 365.

Chapter 10: Managing Office 365

Clients
Paul Robichaux

An Array of Clients
In the same way that users say things like “My Outlook is down” when they mean that their Exchange Server mailbox
is broken, most users think of Office 365 mostly through the lens of the client interfaces they use. Microsoft has a
sometimes-confusing mix of clients for the various Office 365 services, including desktop clients for Mac OS, Windows
10, and Windows 10 S; mobile applications for iOS and Android (and even the lamentably-gone Windows Phone); web
applications; and embedded clients built into desktop phones, Surface Hub devices, and so on.

Browsers
All the Office 365 services support web browsers. In fact, it is quite normal for new Office 365 services to initially
appear as web-only applications. Some services have support for desktop applications added later, largely because it
takes longer to upgrade desktop applications. Some services launch on all platforms simultaneously. Teams is the best
example as it launched into preview complete with web, mobile (iOS and Android), and desktop clients (Windows and
Mac). However, some Office 365 applications, like Planner, have no desktop versions and are only available through
web and mobile clients.

For web browser access, the Office 365 system requirements page recommends that you use the latest versions of the
Internet Explorer, Edge, Chrome, Mozilla Firefox, and Safari browsers. Internet Explorer 11 is the latest supported
version, which presents a challenge for businesses who are running older versions of Internet Explorer for legacy
compatibility reasons. Although older versions of IE and other browsers might work with Office 365, there's no
guarantee that they will work for all features or that the experience won't degrade over time as Office 365
development continues. I recommend that you stop using Internet Explorer altogether and switch to Edge or Firefox
for Windows instead. If that’s not possible, you should upgrade to IE 11 and then make use of Enterprise Mode for
backwards compatibility with older web applications running in your environment. In November 2018, Microsoft
announced that they will replace the core HTML rendering engine in Edge with the open-source Chromium engine,
and then ship a version of Edge for Windows 7. It will be interesting to see whether Microsoft meets this timeline,
and, if so, whether they accelerate their timeline for deprecating Internet Explorer 11.

Web browser access is available to all Office 365 users, providing them a choice of methods for interacting with their
data and applications depending on their needs at the time. There is also a specific license type, the kiosk or frontline
worker license, that gives access mainly via web browser and not via desktop applications. Kiosk users can also
access mailboxes using ActiveSync clients, as well as POP.

Protocols
Aside from web browser access, licensed Office 365 users can connect to mailboxes in Exchange Online using multiple
protocols and applications, cloud file storage and SharePoint libraries using the OneDrive for Business client and can
also connect to a variety of services such as SharePoint, Yammer, Groups, Skype for Business Online, and Project
Online using dedicated mobile clients. Exchange Online provides granular controls for each mailbox access protocol,
which are discussed in detail in the next section of this chapter. Access to most other services is either enabled or
disabled through licensing, or by disabling features within the service itself on a per-user basis. Access to services via
mobile clients can be controlled using mobile device management (MDM) or mobile application management (MAM),
which are discussed in Chapter 18.

Office 365 supports any version of Microsoft Office desktop software that is still in mainstream support. The Office
365 Business Premium plan, and Enterprise plans from E3 upward, include the right to install and run the Office
desktop applications (Word, Excel, PowerPoint, Outlook, and so on). Companies can also license the Office 365
desktop applications separately, without any of the other cloud services such as Exchange Online or SharePoint Online
included in the plan, and gain the benefits discussed below. But if you are reading this book, we can assume that you
are doing more with Office 365 than just using the desktop applications.

Outlook
The desktop Office applications have historically been available in two forms. First is the familiar “perpetual license”
version: you buy it once and keep it forever. Perpetual versions of Office have long been built on the Microsoft
installer (MSI) format. The second, newer, form is Office as a subscription-based service. When you license Office this
way, you only have rights to use it while you keep paying. In exchange for this ongoing fee, you get more rapid
product updates. You will hear people refer to subscription-based Office as “click-to-run” or “C2R” sometimes because
the Office apps themselves are built on an application virtualization technology (known as AppV) that allows
application updates to be streamed on-demand. It is challenging to mix C2R and MSI versions of Office on the same
machine, and Microsoft recommends that you not do it.

Real World: The Microsoft Teams desktop client for Windows and Mac OS doesn’t exactly use either of these
mechanisms. You can download a platform-specific executable installer that streams the Teams installation bits to the
machine, but after that initial installation the Teams client will automatically update itself. Bowing to user requests,
Microsoft also offers an MSI-based installer for bootstrapping Teams installations through System Center
Configuration Manager or Group Policy, but once the product is installed that’s the end of your control over its
updates.

Why would anyone ever be willing to pay an ongoing license fee for Office? That’s a fair question, but this approach
offers some concrete benefits. First is that Microsoft updates subscription-based applications regularly (and more
frequently than the perpetual versions). Secondly, and probably more importantly, getting Office apps as part of an
Office 365 subscription can be surprisingly cost-effective. Buying a subscription to Office apps through Office 365
offers a fixed, predictable monthly cost tied to the number of users you have. Some people argue that the cost of an
Office 365 subscription is more expensive in the long term than the traditional model of buying licenses at a one-time
cost. There are many variables to consider in that equation, including the upfront cost, the cost of buying new
versions to remain in mainstream support, and the cost of deploying and maintaining the software with updates. It's
important to make a fair comparison, and not try to compare prices for Home or Personal editions of Office with the
full Business or ProPlus suite, as Home and Personal can't be used to connect to Office 365 cloud services like
Exchange Online. The ProPlus versions allow customers to install on up to 5 machines, which may further reduce your
overall cost of deployment. For customers using Office 365 cloud services, the additional requirement to stay fully
within mainstream support will come into effect in the year 2020, requiring standalone Office users to upgrade to
newer versions, whereas right now it's possible to stretch out the lifespan of an Office install and connect to Office
365 with versions of Office that are in extended support.

Features and Subscriptions

The third reason can be a bitter pill to swallow. Microsoft has regularly started releasing Office 365 features, such as
Focused Inbox in Exchange Online and integrated support for streamed language translations in PowerPoint, only in
the Office 365 subscription-based applications. This irritates many people who have purchased the perpetually-
licensed versions, but there is no reason to expect Microsoft to add new features to a previously shipped version of a
product for free. To make things worse, there is a perception, mostly created by Microsoft, that even existing features
may break in the cloud—in December 2018, an ill-phrased comment from a Microsoft support representative
highlighted this issue. The C2R version of Outlook prefers Office 365 as the target for Autodiscover requests. This
behavior isn’t new, and, for most users, it works flawlessly whether they have an online or on-premises mailbox. There
are a few edge cases (such as a user who has an on-prem mailbox but also has an Exchange Online license assigned)
where this causes problems. Microsoft provides a solution (documented here ; you must add a DWORD value named
ExcludeExplicitO365Endpoint to HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Outlook\AutoDiscover), but
for customers who have happily been using on-premises mailboxes with C2R software, it might be disconcerting.

As a rule, the subscription-based Office 365 client applications deliver the most up to date, feature-complete, and
compatible version of Office for your users, and that will often outweigh any comparison based on price alone.

Real World: Many client applications such as Google Chrome, and the Click-to-Run version of the Office desktop
applications, can automatically update themselves when new versions are released. Some enterprises prefer to
disable automatic updates and control the deployment of updates. That is fine, if administrative control doesn’t
prevent the updates from ever being deployed. Your update strategy should be either automatic or manual. Not
updating at all isn't a valid strategy for Office 365. For smaller organizations who are content to allow automatic
updates, the IT administrators should at least maintain awareness of the vendor releases by subscribing to RSS or
email notifications from those vendors.

Office Clients
The pace of innovation and development in Office 365 makes it impossible to maintain support for older versions of
client software without incurring high development and support costs that would then need to be passed on to
customers. Part of the decision process about moving to Office 365 is an implicit commitment to maintain client
versions by deploying updates for those applications when they are available from the software vendor. If you allow
your client applications to become outdated, you can expect the quality of the user experience to degrade over time.
Eventually, users might be unable to use some Office 365 features because their client software is unsupported or
obsolete.
The current list of supported versions of Office includes:

Office 365 Professional Plus (“ProPlus”): the version that is delivered as part of eligible Office 365 subscriptions.
Office Professional 2019. Formally released at Microsoft’s Ignite conference in September 2018, Office 2019 is
essentially a snapshot of features previously introduced in Office 365 ProPlus.
Office Professional Plus 2016.
Office Professional Plus 2013.

Those are just the product names. In terms of specific versions, Microsoft rarely provides specific update numbers,
and instead recommends that you apply the latest updates. There’s no minimum update level specified at any given
time, and if you are experiencing client problems with Office 365 services the first step should always be to check for
available updates and, if there are new updates available, apply them to your client installations to see if your problem
is resolved.

The release of Office 2019 marks a milestone for Microsoft in that it’s the first major “year” release since Office 365
ProPlus became the primary Office application suite in Microsoft’s lineup. You can buy Office 2019 in “home &
business” or professional SKUs, but Microsoft is pushing customers towards Office 365 ProPlus instead, in part by
limiting the features in Office 2019. Some of the cloud-only features, such as document co-editing, require access to
the service; others, such as the ability to install the same purchased license on multiple devices, could have been
included in the standalone edition if Microsoft had wanted to.

Once a version of an Office suite or application enters extended support, it is no longer supported by Office 365.
Microsoft usually won’t deliberately seek to block or prevent you from connecting with unsupported versions of Office
applications, but no bug fixes or other updates will ever be applied to make them compatible with Office 365, and they
always have the option to block specific older client versions from communicating. They also can, and have,
deprecated protocols required for older clients, rendering them unable to connect.

Office Update Channels

In early 2016 Microsoft changed the update model for the Office 2016 versions of the Office 365 client applications
from “branches” to “channels”. In September 2017, they changed the model again. The Microsoft website now now
describes three channels, as shown in Table 10-1, and the support lifecycle for feature updates is now 18 months
instead of 12. Each channel releases during a specific month. This approach is designed to give customers more
control over the frequency of updates being received by their end users, as you will be able to update either once or
twice per year thanks to the longer support lifespan. The twice-yearly feature update cadence for Office 365 ProPlus
will also align with the Windows 10 update releases, allowing organizations to plan their deployments of Office and
Windows feature updates to occur together. Security updates will continue to be released monthly.

Channel Feature Updates Security Non-Security Products that default to this channel
Updates Updates

Monthly Channel Monthly Monthly Monthly Visio Pro

Project

Office 365 Business (available standalone

and with some Business plans)

Semi-annual Every six months, in Monthly Every four Office 365 ProPlus
Channel January and July months

Semi-annual Every six months, in Monthly Monthly None

Channel March and September
(Targeted)

Table 10-1: Office 365 Branches and Channels

The Monthly Channel is suitable for businesses that are happy to have their clients updated to the newest features
every month. Typically, this will be customers who do not have special macros or Office add-ins that are critical to
their business processes. Historically, these automatic updates have not caused major, widespread issues, and can be
considered safe to use if you don't have the type of customizations or integrations in your environment that might
break after an update.

If this seems confusing, maybe Microsoft’s description will help make things clearer. The monthly updates are
aggregated into the Semi-Annual Channel (Targeted) release, and then:

“…a new version, with feature updates, will be released to Semi-Annual

 Channel (Targeted) in March and September. Four months later, in July and
January, that version, with those feature updates, will be released to Semi-
Annual Channel and will be supported for the next 14 months.”

The Targeted release is your chance to review and test changes before deploying them to your organization. For
customers with concerns about updates breaking functionality, switching from the Monthly to the Semi-annual
Channel is recommended, so that adequate testing can be performed before updates are deployed to clients. Under
the current update cadence, the Semi-annual Channel still receives monthly security updates when necessary, but any
feature updates or non-security updates will be received every four months.

To control which channel an Office installation uses, you can set the channel during installation, or by using Group
Policy. The Group Policy Administrative Templates for Office 2016 include an Update Channel setting in Computer
Configuration. Setting the update channel using Group Policy has the advantage of allowing you to change clients to a
different update channel than was used during initial setup or allowing you to target different channels to different
parts of your end user population without designing a different installation configuration file for each of them.

Note: The update channel Group Policy setting is not available for Office 2013 clients. It also does not apply to
Office 2016 clients that were installed using the MSI package, only those installed using Click-to-run.

Each update channel for Office 2016 has a different period of support that is applicable. For the Monthly Channel,
support for a build (or release) is only applicable until the next build is released. If a security bug is found in the latest
build of the Monthly Channel, a patch will be released for the latest build, but no patches will be released for any
previous builds in the Monthly Channel. This means that the Current channel builds are supported for one month.

The Semi-Annual Channel (Targeted) channel builds are released every four months, and each build is supported for
four months. When a new Semi-Annual Chanel (Targeted) build comes out, the previous build becomes unsupported.
For example, on May 18, 2018 Microsoft released build 9126.2210 (version 1803) as a Targeted build, replacing the
prior version 1709. This list shows all the build numbers and release dates for Office 365 ProPlus going back to 2015.

The “regular” Semi-Annual Channel builds are supported for 14 months after their release in that channel. When a
build is released to the Semi-Annual Channel (Targeted) channel, that starts the 18-month support clock for that build;
because a build that is released in the Semi-Annual Channel (Targeted) will be promoted to the Semi-Annual Channel
after 4 months, there’s an additional 14 months of support remaining at that point. Microsoft supports both the
current and previous Semi-Annual Channel builds, so there is no immediate pressure to update when a new build is
released to that channel as there is with the other channels.

If a security bug is found in today's build of the Semi-annual Channel, a patch will be released for the latest build and
the previous build, but no older builds.

This update cadence and support period for Office 365 applications are important to keep in mind when you're
planning your update strategy. If you decide to manually control updates, you need to ensure that you deploy new
builds promptly and do not let your clients fall into an unsupported state that could put them at risk of a security
vulnerability.

With multiple update channels to choose from, you also need to strike a balance between providing a stable version of
Office for the bulk of your user population, while also ensuring that new builds are being tested on a sample of your
user population before all users receive the updates. As a starting point, plan to deploy clients with a variety of builds
as follows:

1-5% of clients are deployed with the Monthly Channel.

10-15% of clients are deployed with Semi-annual Channel (Targeted).
All remaining clients are deployed with the Semi-annual Channel.

Other update channels you should know about: The Windows and Exchange teams have their own individual
release strategies which are broadly like how Office does releases… but there are enough differences that you should
carefully examine them to ensure that you’re getting the desired mix of update frequency and stability. In particular,
understanding how the Windows Long-Term Servicing Branch (LTSB) works is critical for organizations that have
strict change control requirements.

The Office Insider program

Microsoft has two programs for those who like to add excitement to their lives by running early, and possibly unstable,
versions of software. You’ve probably run beta versions of software for test purposes in the past; the Windows Insider
and Office Insider programs offer a very similar experience. When you join one of these programs, you’ll have access
to builds of Windows or Office before they are generally released. When you enroll in the Office Insider program,
you’ll get builds in one of two additional channels: Insider (formerly “Insider Fast”) and Monthly Channel (Targeted)
(formerly “Insider Slow”). The Insider channel produces releases roughly weekly, which means you’ll get earlier
access to new features but also that you may run into problems with features that don’t quite work properly. In
exchange for granting early access to Insider members, Microsoft wants to collect user feedback and bug reports, and
they maintain a set of Insider forums for that purpose.

Interestingly, Office Insider exists both for Mac OS and Windows desktop; there isn’t really an equivalent for the
mobile or web-based Office applications, although Microsoft does occasionally conduct open beta testing for Outlook
mobile on iOS and Android.

Controlling client updates via the registry : Although applying individual updates to the system registry on PCs
can be a horrendously inefficient method of controlling which update channel is used for Office, it can be an effective
approach when you want to test new versions of Office on a few PCs. The registry key that controls updates is:
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\office\16.0\common\officeupdate

The values that can be used are:

Insiderfast, for Office Insider (Formerly “Office Insider Fast”).

FirstReleaseCurrent, for Monthly Channel (Targeted).
Current, for the Monthly Channel.
Validation, for Semi-Annual Channel (Targeted)
Business, for the Semi-AnnualChannel.

Note that those values correspond with the old update channel names that Microsoft used to use, even though the
channels themselves don’t use those names.

Skype MVP Tom Arbuthnot has released a useful PowerShell script that allows you to interactively change the release
channel without directly modifying the registry; this may be useful for pilot users during your testing.

Installing Office 365 ProPlus

Office 365 ProPlus is the version of the Office client applications delivered to users as a “Click-to-Run” service, using
application streaming and virtualization technologies to reduce the amount of time between beginning the installation
of the software and being able to start using the applications. Office 365 ProPlus is included in the Office 365
Business, Office 365 Business Premium, Office 365 Enterprise E3, and Office 365 Enterprise E4/E5 licenses, as well
as a standalone “Office 365 ProPlus” license.

Office 365 users who have a license for Office 365 ProPlus can install the Office applications by logging in to the
Office 365 portal and clicking Install Office . At the time of this writing, the Office 365 client versions that are
available as Click-to-Run packages are Office 2016 for PC and Mac. Microsoft ceased to support the Office 2013
ProPlus applications , including Project Pro and Visio Pro, on February 28, 2017. Earlier Mac versions have also been
removed from the portal. Users running old versions of these applications must upgrade to Office 2016 to continue to
receive support from Microsoft.

You can disable software downloads from the portal for your end users, which is often preferred by organizations who
are using existing software licensing for Office clients, or who want to fully manage the installation process and not
allow users to install the software at all.

1. Log in to the Office 365 admin portal with your administrator account.
2. Navigate to Settings , then Services & add-ins .
3. Select Office software download settings
4. Modify the user software options to suit your needs.
5. Click Save to apply the changes.

When users log in to the Office 365 portal, they’ll see installation options based on your selections above. Users must
hold local administrator rights on the computer where they install Office, which is not likely to be an issue in a BYOD
environment but may present a challenge for organizations that do not grant local administrator rights to end users.
We will look at how to automatically deploy Office 365 ProPlus to the computers of low-privilege users shortly.

Real World: Although your corporate-owned computers may prevent users from installing Office 365 ProPlus
themselves, each license entitles the user to install on up to 5 computers, so if they are allowed to download ProPlus
at all, they will be able to log in to the Office 365 portal from home and install the Office 365 ProPlus software there.

Installing Office 365 ProPlus from a network share

When you install Office 365 ProPlus, the small bootstrap installer you launch downloads the required software over
the Internet. This may not be desirable in situations where Internet access is unreliable, bandwidth is limited, or
bandwidth usage is metered and expensive. Fortunately, the Office 365 ProPlus setup files can be downloaded and
placed on a network share to avoid the dependency on the Internet connection.

To download the Office 365 ProPlus setup files you’ll first need to download the Office deployment tool . Run the
downloaded file to extract Setup.exe and a sample XML configuration file. Place the Setup.exe file in the network
share where you plan to store the Office 365 ProPlus source files. The sample XML configuration file should be edited
before it can be used. At a minimum, you should replace the \\server\share entries with the real server and share
names for your environment and review the product IDs that you need to be included in the deployment in your
environment.

Real World: In larger environments whose network includes many branch or regional offices, using DFS to host the
network share for the Office deployment files provides a unified namespace that can help make deployment and
updates simpler to manage.

You can also find a full reference for how to customize the XML configuration file on the Microsoft Support website .

Pay close attention to the “Product ID” that you configure in your XML file. The correct product ID must be used for
the Office 365 license assigned to the user. If the wrong product ID is installed on the computer, the user will not be
able to activate it with their Office 365 credentials and will need to reinstall the correct version. A list of product IDs
matched to license types is available on TechNet.

For Office 2016 Click-to-run deployments, the sample XML file that is included with the deployment tool has channel
of Current and includes Office 365 ProPlus and Vision Pro as the applications to install. You can edit the file to
customize it further. If you’d rather start from a clean slate you can generate a new XML file using the Office 365
ProPlus Configuration XML Editor , or the newer config.office.com tool. Both tools are maintained by Microsoft, have
similar interfaces, and produce a working XML file.

If you do not specify a branch or channel and none is specified via Group Policy, the default branch or channel will be
used (refer to the section about Update Channels earlier in this chapter).

After customizing the XML configuration file, copy it to the same network share that you placed the Setup.exe file in.
Open a cmd.exe prompt on a computer on the network and run Setup.exe with the /download switch to download the
Office 365 ProPlus source files.

C:\> start /wait \\obcdc1\installs\Office365ProPlus\setup.exe /download \\obcdc1\

installs\Office365ProPlus\configuration.xml

The source files will be downloaded and placed in a folder structure automatically. Do not change or rename the folder
structure created by Setup. After the download is completed you can deploy Office 365 ProPlus from the network
share by running Setup.exe with the /configure switch. This command-line can also be run using software
deployment tools such as Group Policy or System Centre Configuration Manager (SCCM).

C:\> start /wait \\obcdc1\installs\Office365ProPlus\setup.exe /configure \\obcdc1\

installs\Office365ProPlus\configuration.xml

A tutorial and sample PowerShell script is available to help in deploying Office 365 ProPlus using GPOs.

Real World: If you’ve already purchased licenses and provisioned users in Office 365, you can begin your client
deployments before you start moving mailboxes and other data to the cloud. The Office 365 client applications can
connect to on-premises servers such as Exchange, within the limits of the normal client system requirements for
those on-premises products.

Managing updates to Office 365 ProPlus Click-to-Run

Updates to Office 365 ProPlus can be managed in different ways to suit the needs of the organization. If the user has
installed Office 365 ProPlus from the Office 365 portal then updates are automatically downloaded from the internet
and installed as they are released by Microsoft, and no other action is required by the end user other than restarting
applications when prompted that an update is ready.

If Office 365 ProPlus has been deployed from a network share, then the XML configuration file will determine how
updates are applied by enabling updates and configuring an update path. For updates to become available for
computers that are not configured to download them from Microsoft's content delivery network, the administrator
must re-run the Office Deployment Tool for Click-to-Run to download the latest build of Office 365 ProPlus to the
appropriate location on the network. Once the updated build is available in the update path on your network, the
computers will automatically apply the updates.

If you need to disable automatic updates the XML configuration file needs to have the specific build number of Office
365 ProPlus defined. A list of build numbers is published on the Microsoft Support website. If automatic updates are
disabled in your XML file, any new builds that you download using the Office Deployment Tool for Click-to-Run will
need to be manually deployed to end user computers.

Registry Information About Click to Run

When you install the click to run version of Office on a PC, the installation updates several settings in the system
registry at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\ClickToRun\Configuration . Among the settings are:

O365ProPlusRetail .EmailAddress : The User Principal Name of the account used to install the software.
O365ProPlusRetail.TenantId : The GUID for the tenant.
VersionToReport : The version of the software installed on the PC. For example, 16.0.10827.20138.

Some administrators use PowerShell to read these values from the system registry and send the data to a central
location for reporting purposes. For example:

[PS] C:\> Get-ItemPropertyValue -Path "HKLM:\SOFTWARE\Microsoft\Office\ClickToRun\Configuration"

-Name "VersionToReport"

16.0.10827.20138

Managing Updates to MSI Versions of Office

The update mechanisms and Channels described so far apply only to the C2R versions of Office. Customers who
license Office 365 can also get access to the traditional MSI packages for Office client deployment. If the MSI
packages are used, then updates need to be managed by the customer using Microsoft Update, Windows Server
Update Service (WSUS), or a similar tool such as System Center Configuration Manager (SCCM).

For C2R clients, all update types are released at the same time on the second Tuesday of each month, also known as
“Patch Tuesday” among IT pros. The MSI clients are handled differently. Non-security updates are released on the
first Tuesday of each month, while security updates remain on the second Tuesday of each month.

MSI deployments are still widely used by customers for a variety of reasons, but as you can see the update
management is more complex and time consuming. For your own client deployments, consider the Click-to-run service
as your first choice, and only revert to the MSI package if you have a good reason to. It will save you time and effort
overall that can be better spent elsewhere.

Office 365 Support for Office 2016 MSI

In September 2018, Microsoft retracted their earlier stance that Office 2016 clients would not be able to connect to
Office 365 services after October 13, 2020. Although understandable that Microsoft should want tenants to use the
latest client software to take advantage of server-side features exposed in Office 365, many companies have Office
2016 deployed on desktops and don’t intend moving to the click-to-run version. Following customer pushback,
Microsoft decided that Office 2016 clients will keep connectivity to Office 365 until October 2023 .

Troubleshooting Office Clients with SaRA

Although Office 365 is a very reliable service, there will naturally be problems that occur from time to time. Part of
that is due to the scale and complexity of the service; it is impossible to keep that much software and hardware in
100% healthy condition all the time. Part of it is also due to factors outside of Microsoft’s direct control. A large
proportion of the support calls that Microsoft receives are client issues that are due to the software on the computer,
network issues, DNS issues, or other factors that can be difficult to quickly identity. The more time Microsoft spends
supporting these types of issues, the costlier it is to run the service.

That’s why Microsoft has released the Support and Recovery Assistant (SaRA) for Office 365. This downloadable
troubleshooting tool has a wizard-driven interface to allow users to identify the issue they are experiencing, and then
allow SaRA to perform diagnostic tests and suggest solutions to the most common client-side issues. SaRA can check
for issues around mail flow, access to shared mailboxes and calendars, Outlook freezes, and repeated authentication
prompts. Although it doesn’t exhaustively test for every possible cause, it’s still a useful tool.

SaRA requires administrative rights on the computer to be able to install the software and run the tests. For BYOD
users, or environments where local admin rights are given to end users, this won’t be a problem. However, you can
also imagine that the existence of SaRA may not be known to end users who do not have an IT focus. They’re likely to
call Microsoft for support anyway, but if they were to try raise a support ticket using the Office 365 portal SaRA can
be suggested to them automatically as part of that process. For customers with IT support staff it is a simple matter of
the IT personnel installing the tool on client computers and running the diagnostic tests.

It’s easy to think that Microsoft should just provide working support tools like this to customers. And indeed, with all
the data they have from telemetry and from analyzing support trends, Microsoft are able to make reasonable guesses
as to what are the most useful diagnostics to develop for SaRA. That said, it’s important for us as customers and IT
professionals to provide feedback that drives the development of tools like SaRA, by letting Microsoft know when the
tools don’t have any information about our problems, or when they fail to detect and suggest fixes for issues. To that
end, SaRA has multiple places where feedback can be submitted to Microsoft.

Outlook Clients
Outlook is still the most common desktop email client used to access both on-premises and cloud Exchange mailboxes.
Microsoft is trying to create a unified identity for Outlook across multiple platforms, which can lead to some confusion
when discussing “Outlook” in books like this. In other sections of this book you will find more specific references to
Outlook on different platforms, such as “Outlook Web App” or “Outlook for iOS and Android”, but Outlook means the
Outlook desktop application here. The Outlook desktop is available for both Windows and Mac OS X, with some
differences in the functionality between the two clients.

Outlook for Windows performs best when you configure it to use cached mode to connect to Exchange Online. In this
configuration, you can decide how much of the server copy of the mailbox Outlook should keep locally on the user’s
computer for faster access. The local cache is known as the OST file, and is on the hard drive of the user’s computer
in a default folder path within their local profile. You can use a slider control in the Outlook account settings to
manage how much of the mailbox Outlook synchronizes to the OST file, so that you can balance availability of data
and storage consumption. For instance, many users find that keeping the last year’s email in the OST is the right
balance. Outlook for Mac uses a similar approach to cache mailbox contents on the local drive.

If you use Outlook and the other Office applications on the Mac, you might be interested in the “ My Workspace ”
program. The program accesses the Outlook calendar to display the user’s schedule and shows details of recent Office
documents they have worked on with the intention of giving quick access to recent work.

A new version of Outlook makes its debut

In September 2018, Microsoft took a somewhat unusual step: they introduced a redesigned user interface for the
Windows desktop and web versions of Outlook on what semi-jokingly came to be known as “New Outlook Day,” but
they did so as an opt-in preview. The new Outlook UI is triggered by updating to the latest version of the Windows
Outlook client or by clicking a simple toggle switch labeled “Try the new Outlook” in the web client, as shown in
Figure 10-1 . The changes are a foretaste of the UI changes that Microsoft has incorporated in the other Office apps,
with simplified icons, a much simpler and more cleanly designed ribbon, and more use of text to indicate functions of
buttons (for example, in Word the button to share a document now has the familiar arrow-coming-out-of-a-box icon
plus the word “Share”.)

Figure 10-1: The new UI for OWA

Since the initial release of the new Outlook UI, Microsoft has released a new and simplified Office ribbon, featuring
new icons and a smaller set of commands, across the Office 365 ProPlus applications. The new ribbon, when
collapsed, the ribbon takes up much less vertical space; when it’s expanded, the icons and artwork use more subdued
colors and simpler line art. Microsoft has also announced and previewed, but not yet deployed, a brand-new set of
icons for the Office applications.

Figure
10-2: The new simplified Outlook ribbon, collapsed and expanded
Figure 10-3: The new UI for Outlook Mobile

At Ignite, Microsoft also showed preview versions of a newly redesigned Outlook mobile client for iOS, but they didn’t
commit to a release schedule for it. They have since released the preview to users through the Apple’s TestFlight
service, which allows developers to flight beta versions of applications to select customers outside the normal iOS App
Store channel. The redesigned Outlook interface (Figure 10-3 ) features a large, easily readable header (which, sadly,
wastes a good bit of vertical space), clear controls for searching and filtering messages and contacts, and a
harmonized use of contact pictures (where available) that matches what you see in OWA and Outlook on the desktop.

Using Outlook in a virtualized desktop environment

In virtual desktop environments, such as Citrix XenApp or XenDesktop, where multiple users log onto published
applications or desktops hosted on a farm of servers, some issues exist with Outlook when it runs in cached mode:

The OST files for multiple users accumulate on the hard drive of the server and can quickly consume all the
available disk space.
If a user logs on to a new session on a different server than their last session, the OST file may need to go
through a major resynchronization, or even recreate its copy of the mailbox from scratch, which causes both a
poor user experience as well as a server performance issue that can affect all other users on the same server.

Similarly, in virtual desktop environments that use non-persistent storage, Outlook needs to rebuild the OST file from
scratch each time a user opens Outlook to start a new session.

To avoid these issues while still keeping the benefits of cached mode, you can apply a combination of several
configuration options in the Outlook group policy template. A description of how to use the Outlook 2013 group policy
settings is available on TechNet. The same guidance applies to Outlook 2016 as well; you would simply need to use
the Office 2016 Group Policy administrative templates (ADMX files) instead. The settings to apply are:

Sync Settings – this setting allows you to specify the amount of data (by age) that Outlook should cache in the
OST file. For example, you could set this to 1 month for users logging on to the virtual desktop environment, and
then set it to 12 months for users who typically work from the same desktop or laptop each day. Users who
frequently work offline in remote locations may even prefer all their email to be cached always.
Fast Access – this setting allows Outlook to connect to Exchange Online in online mode while building the OST
file at the same time, and then switches to cached mode when there is enough data available.
Cache File – this setting can be used to specify a network location for storing the OST file, so that it can be
stored on persistent storage and accessed from any virtual desktop that the user is logging on to. Microsoft has
published guidance and some specific requirements covering this configuration scenario.

There are also third-party solutions, such as Liquidware’s ProfileUnity or FsLogix Office 365 Container , that can help
Outlook cached mode perform well in VDI environments.

Managing mailbox client protocols

A set of connectivity protocols is available to allow clients to connect to Exchange Online. By default, Office 365
mailboxes can use all protocols and features, which include:

Mail Application Programming Interface (MAPI) - Outlook clients use MAPI to connect to mailboxes. Older
Outlook clients are still able to connect using RPC over HTTP, but Outlook 2010 and newer clients running the
latest service packs and updates connect using MAPI over HTTP. RPC over HTTP is an unsupported protocol from
October 31, 2017.
Outlook Web App (OWA) - OWA is the webmail client for Exchange Online and connects via the HTTPS
 protocol.
Exchange ActiveSync - Most mobile devices and applications that connect to Exchange Online do so via the
Exchange ActiveSync (EAS) protocol. Note that Microsoft has no control over how a vendor implements the EAS
protocol in their email client, a fact that accounts for some "interesting" problems that have occurred with mobile
devices over the years.
POP3/IMAP4 - although these are now outdated protocols that do not support the advanced features found in
most email clients, some users still use POP or IMAP to connect to mailboxes, and IMAP is also commonly used
by external applications that need to ingest email items from mailboxes.
Exchange Web Services (EWS) - EWS is the API for application access to Exchange mailbox resources,
commonly used for integrating external applications. EWS is also used by Outlook to retrieve free/busy, out of
office, and calendar sharing information. It is also the only protocol used by Mac clients such as Mac Outlook
2011 and later, and some third party mobile email clients.
SMTP - Outlook can perform all its functions using MAPI, but POP and IMAP are access-only protocols whose
clients need to use SMTP to send email. SMTP is also used by external applications to send email.
Microsoft Graph / REST API - The Office 365 APIs allow applications to access data such as email messages,
contacts, calendars, and files. There are separate APIs for the various Office 365 services, as well as the unified
Microsoft Graph API.
OWA for Devices - This mobile application pre-dated the Outlook mobile application that is available for iOS and
Android today. Microsoft has deprecated OWA for Devices in favor of Outlook mobile. Effective May 15, 2018, the
OWA apps were blocked from Office 365.

Figure 10-4: Accessing the Email apps settings in the Office 365 Admin Center

You can manage mailbox client protocols through the Office 365 Admin Center, the Exchange Admin Center (EAC), or
PowerShell. Different capabilities exist through each interface, which we will explore in this section. First, the Office
365 Admin Center includes an “Email Apps” section for user properties. Select a user, and then click the Edit button
in the Email Apps section and you’ll see a list of available protocols as shown in Figure 10-4 . Enabling or disabling
access to a client protocol is merely a matter of moving the slider to the appropriate position.

To change the client protocols available to a user through EAC, go to the mailboxes section under Recipients , open
the properties of any mailbox, then mailbox features , and then scroll down the controls for the different protocols.

You can disable a protocol by clicking the disable link. To enable the protocol for the mailbox user again click the
enable link. Most protocols can be managed in the EAC, however, access to Exchange Web Services (EWS) can only
be managed using PowerShell. Managing EWS is described in more detail later. To view the status of each client
protocol for a mailbox user via PowerShell, use the Get-CASMailbox cmdlet.
[PS] C:\> Get-CASMailbox –Identity Kim.Akers | Select *enabled

ActiveSyncEnabled : True

OWAEnabled : True

OWAforDevicesEnabled : True

ECPEnabled : True

PopEnabled : True

ImapEnabled : True

MAPIEnabled : True

MapiHttpEnabled :

UniversalOutlookEnabled : False

EwsEnabled : True

To disable a client protocol in PowerShell, you could use the Set-CASMailbox cmdlet. For example, to disable the
ActiveSync protocol for a mailbox use the following command:

[PS] C:\> Set-CASMailbox –Identity Kim.Akers -ActiveSyncEnabled $False

You can repeat the same steps for each individual protocol. If you have a combination of enabled/disabled protocols
you want to apply to mailboxes by default, then you will need to develop a custom script to apply those changes as
part of your mailbox provisioning process. You may be able to avoid some of this tedium now that Microsoft have
given us the Set-CASMailboxPlan cmdlet . This cmdlet allows you to set up a template for protocol access, then apply
the template automatically to newly created mailboxes. By default, you get four CAS mailbox plans in the service; you
cannot create, rename, or remove them. The plan applied when a mailbox is created depends on the license assigned
to the mailbox: Exchange Online Kiosk plans get the ExchangeOnlineDeskless plan, Exchange Online P1 gets
Exchange Online, and Exchange Online P2 gets ExchangeOnlineEnterprise (which means that E3 and E5 subscribers
get it too). The fourth plan, ExchangeOnlineEssentials, is not currently documented. Suppose you wanted newly
created mailboxes for your enterprise users to have IMAP and POP disabled—you could easily do this by running the
following cmdlet:

[PS] C:\> Set-CASMailboxPlan –Identity ExchangeOnlineEnterprise -POP3Enabled $False -IMAPEnabled $false

Note: There’s one protocol in the Get-CASMailbox output that you can ignore for now. MapiHttpEnabled , is only
applicable to on-premises Exchange organizations. The second, UniversalOutlookEnabled , remains a mystery. The
Mail app for Windows 10 is also referred to sometimes as "Universal Outlook" in news articles and blog posts. The
term "Universal" refers to the app as being built by the Outlook team as a Universal Windows App. The Windows 10
Mail app connects to Exchange Online using ActiveSync and is managed like any other ActiveSync device. The
UniversalOutlookEnabled option that is visible in the output of Get-CASMailbox is not used to control this application,
or any other application at the time of this writing.

Outlook for iOS and Android

For many years, Microsoft pursued a terrifically successful strategy of licensing Exchange ActiveSync (EAS) as widely
as possible to third parties to enable their devices to connect to Exchange. EAS proved stable and both the cloud and
on-premises versions of Exchange include secure and basic management capabilities for mobile devices. The strategy
of licensing ActiveSync to all and sundry worked in terms of making EAS the de facto protocol for Exchange mobile
connectivity. The downside was that Microsoft ceded control over the user experience to the device vendors.
Companies like Apple and Samsung incorporate EAS into the mail apps running on their devices while exerting
absolute control over which features the clients expose to end users. Even Microsoft’s own clients do not use the
entire EAS protocol.

Exchange ActiveSync was always intended to do two different things: allow for item synchronization and provide
mobile device management (including remote device wipe, PIN enforcement, and so on). Despite its popularity, no real
advantage exists for the device vendors to implement all the functionality in the EAS protocol and they use EAS to
meet their own business goals. A good-enough job allows a user to connect to Exchange and access their mailbox. No
added value exists for Apple or Samsung or any other EAS licensee to amend their mail apps to offer more
functionality to Exchange users than they do when the apps connect to Gmail or another email server, although some
vendors grudgingly added support for a subset of EAS device management features. The mail app is “good enough” if
you can use it to process messages, meetings, tasks, and contacts. This attitude has existed for years and has resulted
in situations like the famous Apple iOS mail app bugs that caused excessive transaction log growth on Exchange
servers or calendar appointment “hijacking”. The net result is that EAS is now the lowest common denominator
protocol for mobile devices to connect to Exchange. The need to do better and to have control over functionality
available through mobile clients drove Microsoft to come up with a new two-pronged strategy.

The first prong of the strategy became visible when, in January 2015, Microsoft released the Outlook for iOS and
Android clients, rebranded and improved versions of the mobile email app previously known as Acompli (bought in
November 2014). The point of developing a set of high-quality Outlook apps for multiple platforms rather than the
earlier approach centered around licensing EAS is that it enables Microsoft to control the experience delivered to the
end user. Microsoft can update the Outlook apps to introduce a new feature without the need to debate
implementation details with mobile device vendors. No delay exists for end users to receive the latest functionality as
all they need to do is download an update from the Apple app store or Google Play. Pushing functionality out like this
is popular with users while creating some challenges for administrators who must figure out how to support updated
mobile clients, especially in situations where a Bring-Your-Own-Device policy pertains. The second prong of the
strategy was to improve the basic Exchange-native mobile device management tools available within Office 365 and
then to build Microsoft Intune, a much more comprehensive tool for mobile device and application management.

As was inevitable when an app came from the acquisition of a small company, large and complex corporations
expressed some security concerns about the first versions of the Outlook app, including the caching of mailbox data in
Amazon Web Services. All the issues expressed at the time are now resolved, and the intermediate processing of
mailbox data runs in an Azure-based service within the Microsoft cloud. The service performs translation and proxying
of protocol data to allow both older and newer versions of Outlook to communicate with the REST APIs that link to the
Exchange Online backend. (The component that does this is referred to in Microsoft’s documentation as the “stateless
protocol translator.”) The Azure-protocol translator runs within all Office 365 datacenter regions. The translator
removes the need to cache any mailbox data, which stays in the user’s mailbox. The automatic filtering of email to
support the Focused Inbox view occurs within the Exchange Online infrastructure. Unlike earlier implementations,
this architecture makes the complete mailbox available to mobile clients, including the ability to execute searches
against all items in the mailbox. A downside of this approach to synchronization is that users who access multiple
mailboxes hosted in different geographic regions may find they are only able to configure Outlook to connect to one of
the mailboxes at a time. This behavior is due to Microsoft respecting the data sovereignty regulations that exist in
some regions;

From April 2018, the infrastructure used by Outlook for iOS and Android is FedRAMP-compliant and approved for use
by Government Community Cloud (GCC) tenants in the U.S. See Microsoft’s instructions for details about how to
enable the Outlook mobile apps for use in GCC tenants.

Licensing Outlook for iOS and Android : Not all Office 365 plans include a license for the Outlook mobile clients.
Microsoft’s FAQ makes it clear that you need a suitable commercial Office 365 license: Business, Business Premium,
Enterprise E3 or E5, ProPlus, or the corresponding versions of those plans for Education or Government. Exchange
Online plans or the kiosk (front-line worker) plans do not include a license. However, these licensing restrictions
aren’t enforced, so it’s possible for you or your users to violate the license terms without realizing it.

In December 2018, Microsoft started deploying a new native mobile synchronization protocol (formerly codenamed
“Hx”) for Outlook mobile clients. The new protocol does away with the stateless protocol translator; Outlook mobile
clients attach directly to an endpoint in the service, meaning that it’s much easier for Microsoft to support multiple
instances of the service (such as GCC, the defense and aerospace-specific ITAR instance, and local instances in
various countries). Direct connection to the service also provides lower latency and better sync performance, and it
allows Microsoft to provide synchronization for both Android and iOS clients (which formerly used separate protocols)
with a single protocol. Interestingly, this protocol is not brand-new; it’s the same protocol used by the Windows 10
mail client, and it will eventually be used by the Mac version of Outlook. The new sync protocol will roll out to tenants
on a random basis, and at least as of now, Microsoft won’t give tenant administrators either prior notice of or control
over the rollout.

Perhaps more important than the infrastructure of Outlook for iOS and Android is the feature set of the client
application itself. Microsoft has kept up a rather astonishing cadence for feature improvements; in September 2018
alone, they added the ability to access shared calendars, support for creating and joining Teams meetings from the
Outlook client, integration with Office Lens functionality for photos and OCR, and the ability to search calendar
events.

Outlook Web App

Some time ago, Microsoft renamed Outlook Web App to “Outlook on the web.” This is a terribly clunky name, so we
won’t use it, especially as the vast majority of documentation for the product still refers to “OWA”.

OWA is the webmail interface for Exchange Online, providing users access to their mailbox, calendar, contacts, tasks,
as well as a launch pad for other Office 365 services such as the Office Online apps, OneDrive for Business,
SharePoint sites, Delve, and Office 365 Video.

In older versions of the on-premises Exchange product the webmail interface offered only a limited subset of normal
Outlook functionality. However, the modern OWA user experience is much richer, and rivals that of the full Outlook
client. In fact, for many users OWA provides all the functionality they need, with the convenience that it can be used
from anywhere that has an Internet connection without having to first install client software.
Keep in mind that the on-premises version of OWA lacks many of the features available in Office 365; this has been a
source of significant complaint from Microsoft’s customers. In hybrid environments, you will need to be aware of
differences between the cloud and on-premises applications and communicate to, and train, your users accordingly.

Web browser compatibility

Outlook Web App works with the latest version of major web browsers on current operating system platforms, such as
Microsoft Edge, Internet Explorer 11, Firefox, and Chrome on Windows, Safari on Mac OS X, and Firefox or Chrome
on Linux. Figure 10-5 illustrates the premium version of OWA, which is the interface that you expect to see when
modern browsers connect to Exchange Online, although you may see some minor differences as the OWA interface
evolves rapidly.

Figure 10-5: Outlook Web App

OWA supports a "light" mode that is automatically used for older versions of web browsers or operating systems such
as Linux that are not fully supported by Office 365. Being HTML-based, OWA light is used as the accessibility version
because it can work better with screen readers. As displayed in Figure 10-6 , the user experience is degraded when
compared to the full version of OWA, with some features not working or simply not available in the interface.

Figure 10-6: OWA "light"

With the pace of changes and new features in Office 365 maintaining updates to web browser clients is important for
customers. If you experience some unexpected behavior of OWA when running a supported combination of web
browser and operating system, you should check for recent updates to the browser. You can find the latest guidance
on OWA web browser compatibility on the Office Support website.

Outlook Web App mailbox policies

Outlook Web App mailbox policies provide administrators with control over which OWA features are available to
individual mailboxes. Examples of the features controlled by OWA mailbox policies include inbox rules, calendar
access, mobile device controls, and social network integration. Although an OWA mailbox policy controls access to
these features for OWA, it does not disable them on the mailbox itself. For example, a mailbox user who is prevented
from accessing their calendar due to an OWA mailbox policy can still access their calendar using Outlook or a mobile
device.

In the on-premises version of Exchange, the OWA virtual directories also provide a point of control for client
functionality. Obviously, you can't do this within Exchange Online, so you must work with OWA mailbox policies if you
wish to disable specific OWA features. A default OWA mailbox policy is created for each Office 365 tenant. You’ll find
OWA mailbox polices, including this default, in the permissions section of the EAC. The default policy enables all
features for OWA.

The OWA mailbox policy that is assigned to a mailbox user can be viewed and changed in the EAC by selecting a
mailbox and clicking View details under Email Connectivity . You can also retrieve the OWA mailbox policy that is
assigned to a mailbox user by running the Get-CASMailbox cmdlet.

[PS] C:\> Get-CASMailbox Kim.Akers | Format-List OwaMailboxPolicy

OwaMailboxPolicy : OwaMailboxPolicy-Default

The EAC will surface some, but not all, of the possible configuration options for OWA mailbox policies. The full set of
configuration options for OWA mailbox policies is available when using PowerShell. In this example, the Get-
OWAMailboxPolicy cmdlet is used to display the name of the OWA mailbox policy, as well as a setting that controls
whether Facebook integration is permitted.

[PS] C:\> Get-OwaMailboxPolicy | Select Name, Facebook*

Name : OwaMailboxPolicy-Default

FacebookEnabled : True

Your organization may only need a single, default OWA mailbox policy to which you make one or two changes to suit
your needs. Alternatively, you can create multiple OWA mailbox policies and assign to different mailbox users to limit
access to some features. For example, you might want to create an OWA mailbox policy that turns off Calendar access
for temporary employees or people working on an assembly line, along with another OWA mailbox policy that allows
Calendar access for other users.

After configuring a new OWA mailbox policy it needs to be assigned to mailbox users by running the Set-CASMailbox
cmdlet.

[PS] C:\> Get-OwaMailboxPolicy | Format-List name

Name: Limited Access OWA Users

Name: OwaMailboxPolicy-Default

[PS] C:\> Set-CASMailbox Kim.Akers -OwaMailboxPolicy "Limited Access OWA Users"

The next time the user refreshes their session or logs on to OWA the new policy is applied. To revert the mailbox user
to the default OWA mailbox policy, run the Set-CASMailbox cmdlet again.

[PS] C:\> Set-CASMailbox Kim.Akers -OwaMailboxPolicy (Get-OwaMailboxPolicy |

Where {$_.IsDefault}).Name

An OWA mailbox policy is not mandatory for mailboxes (the default policy is applied if one is not specified), so you can
also remove policies entirely from the mailbox using Set-CASMailbox .

[PS] C:\> Set-CASMailbox Kim.Akers -OwaMailboxPolicy $Null

Customizing the OWA URL

The OWA URL for all Exchange Online customers is the same, https://outlook.office365.com/owa . There are some
variations of that URL that will also work thanks to redirections that Microsoft has in place. For example,
http://outlook.office365.com will redirect from HTTP to HTTPS and from the root of the domain to the /owa virtual
directory.

Although this is a simple URL some organizations would prefer a custom URL that includes their own domain name,
which may be easier for their end users to remember. This can be achieved with a simple DNS CNAME record in the
organization’s public DNS zone. For example, a CNAME of webmail in the DNS zone for office365itpros.com with a
target of outlook.office365.com , will redirect any user that browses to http://webmail.office365itpros.com to
http://outlook.office365.com , where Microsoft’s redirection to /owa takes care of the rest.

Aside from the branding and ease of remembering, this is also a good strategy for retaining the same OWA URL that
you may have previously used for on-premises Exchange, by simply changing it to a CNAME that resolves to the Office
365 OWA URL instead. This avoids issues with your end users still having web browser bookmarks for the on-premises
OWA URL. However, this technique should not be used for Hybrid deployments.

Exchange Web Services

Exchange Web Services (EWS) is an API that enables client applications to communicate with Exchange Online.
Applications can use EWS to retrieve information from Exchange Online services, or to interact with data in Exchange
Online mailboxes. For example, an EWS application can retrieve information about calendar items for room mailboxes
to determine which items might have an organizer who no longer works for the company.

EWS is also used by Microsoft Outlook for calendar free/busy information, Out of Office settings, calendar sharing,
and other features such as MailTips. Other applications (including the Skype for Business client and the Mac OS X
Mail application) use it for access to mailbox data as well. As with the other protocols used to access Exchange
Online, there are controls available for administrators to use for a variety of scenarios. EWS controls can be managed
at the mailbox level or the organization level. The EWS settings for a mailbox are retrieved by running the Get-
CASMailbox cmdlet, and the organization level settings are retrieved by running the Get-OrganizationConfig cmdlet.

[PS] C:\> Get-CASMailbox Kim.Akers | Format-List ews*

EwsEnabled : True

EwsAllowOutlook :

EwsAllowMacOutlook :

EwsAllowEntourage :

EwsApplicationAccessPolicy:

EwsAllowList :

EwsBlockList :

[PS] C:\> Get-OrganizationConfig | Format-List ews*

EwsAllowEntourage :

EwsAllowList :

EwsAllowMacOutlook :

EwsAllowOutlook :

EwsApplicationAccessPolicy:

EwsBlockList :

EwsEnabled :

The Future of EWS

In July 2018, Microsoft announced that they will start to move away from EWS for Exchange Online in favor of the
Microsoft Graph when the need exists for programmatic access to Exchange data (in effect, the Outlook Mail API ).
The Graph doesn’t exist for on-premises servers, so on-premises versions of Exchange are unaffected. The decision
means that:

EWS will not receive future feature updates.

Microsoft will decommission basic authentication for EWS and exclusively use OAuth 2.0 . This change will
happen on October 13, 2020.
Microsoft emphasizes that they continue to support EWS for use in production environments. The change to the
Graph means that developers who build tools based on EWS must either change to use the Graph instead or continue
using EWS by replacing basic authentication with OAuth 2.0. Although the Graph and EWS largely overlap in terms of
functionality, some features used by EWS do not yet exist in the Graph. Microsoft is working to close this gap, but
until they do, some tools will have to stay EWS-based.

Using the EWS allow or block list

In June 2013, LinkedIn implemented a feature that invited users to enter their corporate email credentials on the
LinkedIn website. LinkedIn then connected to the person’s corporate email account and extracted email addresses to
suggest them as potential contacts for LinkedIn. The connection from LinkedIn used Exchange Web Services and
highlighted the need to monitor and control EWS access to Exchange on-premises and Exchange Online.

Disabling the entire EWS protocol because of one unapproved example of application access would deny your
organization the many good things that EWS allows. Fortunately, we can be selective in what we block or allow for
EWS by configuring an EWS application access policy. The EWS application access policy can be configured on a per-
mailbox basis or configured for the entire organization.

Real World: If your Office 365 tenant has “K” (Kiosk) licenses the EWS allow/block controls for your tenant will not
work at the organization level. For non-Kiosk mailboxes, apply allow/block controls on a per-mailbox basis. For the
Kiosk mailboxes themselves, direct EWS application access to the mailboxes is not permitted.

Continuing with the example of LinkedIn, to block EWS access by the LinkedIn user agent for the entire organization
there are two steps required that use the Set-OrganizationConfig cmdlet. First, set the EWSApplicationAccessPolicy to
enforce the block list.

[PS] C:\> Set-OrganizationConfig -EwsApplicationAccessPolicy EnforceBlockList

Next, add the LinkedIn user agent to the EWS block list.

[PS] C:\> Set-OrganizationConfig -EwsBlockList @{add='LinkedInEWS'}

The EWS block list is a multi-value attribute so it should be managed using add/remove methods to avoid overwriting
existing values when you are making modifications. For example, to add the OWA for Devices user agent to the block
list you run this command.

[PS] C:\> Set-OrganizationConfig -EwsBlockList @{add='MOWAHost*'}

[PS] C:\> Get-OrganizationConfig | Format-List ewsblocklist

EwsBlockList: {MOWAHost*, LinkedInEWS}

Similarly, to remove an entry you run this command.

[PS] C:\> Set-OrganizationConfig -EwsBlockList @{remove='MOWAHost*'}

[PS] C:\> Get-OrganizationConfig | Format-List ewsblocklist

EwsBlockList: {LinkedInEWS}

Unlike ActiveSync device access rules, the strings used for EWS block and allow lists can use wildcards for partial
matches. However, unlike ActiveSync controls there is no quarantine action available, only allow or block.

The example above blocks LinkedIn EWS access for the entire organization. If you only wanted to block it for a single
mailbox user the same process would be used, but the Set-CASMailbox cmdlet would be used instead of Set-
OrganizationConfig . Enforcing a block list will permit any EWS application that is not in the block list to connect. A
more restrictive approach is to enforce an allow list instead, which requires that any EWS applications be listed in the
allow list before they can connect.

[PS] C:\> Set-OrganizationConfig -EwsApplicationAccessPolicy EnforceAllowList

Enforcing a block or allow list for EWS has no impact on the Entourage, Outlook for Mac, or Microsoft Outlook
applications. These applications are controlled with different EWS settings which are discussed next.

Blocking/allowing Mac clients

There are two separate Mac clients that use Exchange Web Services for connecting to Exchange Online; Entourage
and Mac Outlook. Entourage is the oldest of these, with the Web Services Edition released in 2008 to allow
connectivity to Exchange 2007 using EWS. It has long since gone out of support, but you may still see it occasionally
on older machines. Prior to that Entourage used WebDAV, which was deprecated in Exchange 2007 and removed
entirely starting with Exchange 2010. Mac Outlook is the version of Outlook that ships with Office for Mac 2011, but
also refers to the newer version of Mac Outlook included in Office 2016 for Mac.

The use of Apple Mac computers is common in many corporate and education sectors but some organizations have
reasons to block the use of Mac email clients. For example, an enterprise that permits BYOD may choose to block
Entourage and Mac Outlook and require those users to instead make use of Outlook Web App or Outlook delivered
using an application virtualization infrastructure.

The Mac clients are allowed by default and can be blocked using Set-CASMailbox or Set-OrganizationConfig . For
example, to block both Mac clients for a mailbox user you would run the following command.

[PS] C:\> Set-CASMailbox Kim.Akers -EwsAllowEntourage $False -EwsAllowMacOutlook $False

Apple’s own Mail and Calendar clients use EWS, and you can block their access using the EWS block list described in
the preceding section; the EwsAllowEntourage flag doesn’t affect them.

Blocking/allowing Outlook EWS access

Microsoft Outlook can also be blocked for Exchange Web Services. This will impact free/busy information, Out of
Office, and calendar sharing. Blocking Outlook is a rare requirement and typically only applies to organizations that
want to limit specific users from being able to share calendars or see the availability of other users.

The Set-CASMailbox or Set-OrganizationConfig cmdlets are used to apply the block. For example, to block EWS
access by Outlook for a mailbox user you would run the following command.

[PS] C:\> Set-CASMailbox Kim.Akers -EwsAllowOutlook $False

Other uses for EWS

Exchange Web Services is used by many organizations for custom application development, such as creating
integrations between Exchange Online and their in-house line of business applications. It is also the API used for
integration between Exchange Online and other Microsoft services such as Skype for Business and SharePoint Online.
EWS applications can send and receive email messages, manage calendar items, and a whole lot more.

EWS is also used by many third-party migration tools as the protocol for accessing Exchange Online mailboxes. In
addition, an ecosystem of “cloud backup” products use EWS as the access protocol to extract mailbox data from
Exchange Online to copy to cloud storage and so meet the needs of many organizations who would like to make use of
Office 365 but are concerned about backup and recovery.

POP and IMAP

You can connect to Exchange Online mailboxes using the POP3 (POP) and IMAP4 (IMAP) protocols by using one of a
wide variety of email client software. Examples of email clients include Mozilla Thunderbird, Eudora, and the Mail
client for Mac OS X. Several mobile email applications can also use POP and IMAP for connecting to mailboxes. POP
and IMAP are also commonly used by third party applications that need to retrieve email items from a mailbox for
processing. Microsoft Outlook supports both POP3 and IMAP4 but given that Outlook supports the RPC-over-HTTP
and MAPI-over-HTTP protocols for connecting to Exchange Online, there seems to be little point in using Outlook as a
POP or IMAP client.

POP and IMAP are insecure protocols that transmit all client-server communications (including usernames and
passwords) in clear text that can be read by any intermediate network device. To secure the use of POP and IMAP,
Exchange Online enforces SSL/TLS encryption for all POP and IMAP client connections. While on-premises Exchange
servers permit administrators to relax the SSL/TLS requirement and use insecure POP/IMAP connectivity Office 365
offers no such choice to customers. You must use the protocols securely with SSL/TLS if you want to use them at all.

Client settings for POP and IMAP

Table 10-2 lists the POP and IMAP settings for Office 365. The SMTP settings are included because POP and IMAP
clients use SMTP to send email.

Protocol Server Port Encryption

POP3 outlook.office365.com 995 SSL

IMAP4 outlook.office365.com 993 SSL

SMTP smtp.office365.com 587 TLS

Table 10-2: POP and IMAP settings for Office 365

As an example, Figure 10-7 shows the correct Office 365 settings configured for IMAP and SMTP connectivity in the
Mozilla Thunderbird email client.

Figure 10-7: Setting up IMAP access to Office 365

POP and IMAP protocol defaults

The POP and IMAP protocols have configuration settings that can be changed to suit the organization. These settings
are applied on a per-mailbox basis. By default, each Exchange Online mailbox is configured with the settings shown
below, as viewed using Get-CASMailbox .

[PS] C:\> Get-CASMailbox ServiceDesk | Format-List pop*,imap*

PopEnabled : True

PopUseProtocolDefaults : True

PopMessagesRetrievalMimeFormat : BestBodyFormat

PopEnableExactRFC822Size : False

PopSuppressReadReceipt : False

PopForceICalForCalendarRetrievalOption : False

ImapEnabled : True

ImapUseProtocolDefaults : True

ImapMessagesRetrievalMimeFormat : BestBodyFormat

ImapEnableExactRFC822Size : False

ImapSuppressReadReceipt : False

ImapForceICalForCalendarRetrievalOption: False

These default settings work just fine for most mailboxes. But there are always a few cases here and there that need
some customization applied to one or two mailboxes, particularly when it comes to mailboxes being accessed via IMAP
or POP by software systems.

Before any custom configurations for a mailbox take effect the *UseProtocolDefaults settings must be set to $false .
You can set this for POP only, IMAP only, or set it for both if required.

[PS] C:\> Set-CASMailbox ServiceDesk -PopUseProtocolDefaults $False

-ImapUseProtocolDefaults $False

For both POP and IMAP, the EnableExactRFC822Size settings that are visible in the Get-CASMailbox output are not
configurable in Exchange Online. The Set-CASMailbox cmdlet simply doesn’t make that parameter available to you,
unlike on-premises Exchange. That leaves the following settings that can be configured.

Messages Retrieval MIME Format

The *MessagesRetrievalMimeFormat setting is used to specify the format for messages sent to the mailbox from other
internal senders. This setting has no impact on messages sent from an external sender. The default value is
BestBodyFormat , and you can also choose from:

TextOnly.
 HtmlOnlyHtml.
HtmlAndTextAlternative.
TextEnrichedOnly.
TextEnrichedAndTextAlternative.
BestBodyFormat.

Tnef.

Unless you have encountered a specific problem the default of BestBodyFormat can be left as is. However, if you do
identify an issue the format can be changed to one of the other values. An example of this would be a service desk
ticketing system that connects to a mailbox and ingests email items into its own database for processing. If the service
desk system is having issues reading the email messages, then the MIME format can be configured to a different
setting using Set-CASMailbox .

[PS] C:\> Set-CASMailbox ServiceDesk -PopUseProtocolDefaults $False

-ImapUseProtocolDefaults $False

[PS] C:\> Set-CASMailbox ServiceDesk -ImapMessagesRetrievalMimeFormat TextOnly

-PopMessagesRetrievalMimeFormat TextOnly

Force ICal for Calendar Retrieval Option

When a POP or IMAP client receives a meeting request from a sender within the organization, the meeting request
contains a link within the body of the email that takes the recipient to Outlook Web App for accepting or declining the
meeting. However, some POP and IMAP clients are compatible with ICAL messages, and so those particular users may
prefer to receive meeting requests as ICAL messages instead.

[PS] C:\> Set-CASMailbox Kim.Akers -PopUseProtocolDefaults $False

-ImapUseProtocolDefaults $False

[PS] C:\> Set-CASMailbox Kim.Akers -PopForceICalForCalendarRetrievalOption $True

-ImapForceICalForCalendarRetrievalOption $True

This option has no impact on meeting requests from external senders, which will arrive as an email message with an
.ICS file attachment instead.

Suppress Read Receipt

When a mailbox is accessed using either POP or IMAP there are two triggers for read receipts to be generated; when
the email message is downloaded by the client, and when the mailbox user opens the email message to read it.

The second trigger (when the mailbox user reads the email message) is user controllable. The user can configure their
email client to never send read receipts, to always send read receipts, or to prompt each time a read receipt is
requested. Given that this is something that the user can control there are no administrative controls to influence this
on a per-mailbox basis.

That just leaves the read receipt triggered when the message is downloaded from the mailbox. The read receipt
generated by that event may not be desirable in every situation. Just because an email message has been downloaded
by a piece of software, doesn’t mean it has actually been read. Furthermore, if the mailbox is for a high volume
transactional mailbox, such as an ordering system or help desk system, and the email messages are being accessed by
a software system the volume of read receipts generated could be incredibly high while serving no actual benefit at
all.

For those types of scenarios, you can suppress the read receipts generated when email messages are downloaded via
POP or IMAP. Suppressing the read receipt works for both internal and external senders.

[PS] C:\> Set-CASMailbox ServiceDesk -PopUseProtocolDefaults $False

-ImapUseProtocolDefaults $False

[PS] C:\> Set-CASMailbox ServiceDesk -ImapSuppressReadReceipt $True

-PopSuppressReadReceipt $True

Client Access Rules

Exchange Online supports the ability for a tenant to create and apply client access rules, which let you block or allow
specific protocols, authentication types, and IP address ranges. The New-ClientAccessRule cmdlet creates client
access rules ; as you would expect, Get-ClientAccessRule , Set-ClientAccessRule , and Remove-ClientAccessRule
cmdlets are also available. You must use PowerShell to create and manage client access rules, as there is no support
for them in the Exchange admin center.

To be more specific, you can selectively block or allow access through the Exchange ActiveSync, IMAP, EWS, Outlook
Anywhere, OWA, POP3, PowerShell, and REST API access, as well as access to the Exchange Admin Center and
PowerShell Web Services. You can also restrict or allow connections using AD FS, basic, certificate-based, or OAuth
authentication types. These selectors can be combined so that, for example, you can do things like block all OWA
access that’s not authenticated via AD FS.

By default, a tenant has no client access rules, so anyone with a valid mailbox can access Exchange Online from any
IP address using any protocol not blocked at the tenant or mailbox level. You can define up to 20 rules per tenant to
control what protocols clients can use over what connections. You can even create a filter based on some Azure Active
Directory account properties to control the scope of a rule (for instance, only apply this rule to users in Dublin). This
example creates a new rule to block ActiveSync connections such as those from Apple iOS mail app clients unless they
come from a specific IP address range corresponding to the corporate network.

[PS] C:\> New-ClientAccessRule -Name "Block ActiveSync" -Action DenyAccess -AnyOfProtocols ExchangeActiveSync -
ExceptAnyOfClientIPAddressesOrRanges 203.0.113.0/24

Each rule must contain an action (specifying whether the rule should allow or block access when its conditions are
met) and then a set of conditions. These conditions may either be positive or negative; that is, for a positive condition,
the rule matches if the specified parameter does match the rule, and for a negative condition the rule matches if the
specified parameter doesn’t match. The rule above is an example of a negative condition: it matches any IP that is not
in the specified range. You could get the same effect like this:

[PS] C:\> New-ClientAccessRule -Name "Block ActiveSync" -Action AllowAccess -AnyOfProtocols ExchangeActiveSync -
AnyOfClientIPAddressesOrRanges 203.0.113.0/24

However, this example wouldn’t actually do anything unless you created other rules, because by default all clients can
connect using all protocols from all addresses. To make this rule work, you’d need another rule that blocks everything
else… so it’s simpler just to use the first form unless you need a more complex set of rules to enforce your
requirements.

Exchange evaluates client access rules in priority order, which you can alter to promote or demote rules. As soon as
the conditions of a rule match, the rule is applied, and rule processing stops. When you create a rule, you can
explicitly apply a priority (0 is the highest priority value). If you don’t specify a priority, the first rule you add becomes
priority 0, the second priority 1, and so on. Changing the priority of an existing rule bumps down the priority of all the
rules after it; if you create a new rule with priority 5, the former priority-5 rule becomes priority 6 and all rules after it
follow suit.

PowerShell is one of the protocols you can control with client access rules. Microsoft strongly recommends that the
first rule you create is this:

[PS] C:\> New-ClientAccessRule -Name AllowPS -Action Allow -AnyOfProtocols RemotePowerShell -Priority 0

This rule guarantees that you’ll be able to use Remote PowerShell even if you later create a rule that might block it. If
you do manage to create a rule that prevents you from using PowerShell against your tenant, you’ll have to contact
Microsoft support to ask them to fix it.

You can create rules that apply to users instead of only to protocols and IP ranges, too. The
UserNameMatchesAnyOfPatterns , UserRecipientFilter, and ExceptUsernameMatchesAnyOfPatterns conditions are
useful for meeting requirements such as “Allow any users in the sales department to use Exchange ActiveSync but
block it for everyone else.” In that case, using UserRecipientFilter {Department -eq “Sales”} as a filter condition
would do the trick.

You can also scope rules to control whether they apply to users only or to all applications and clients. If you scope a
rule using -Scope All , connections from services and applications (such as EWS connections from an on-premises
hybrid Skype for Business server) will be affected by the rule.

Because you can create complex chains of rules, it is very important to validate the effect of the rules you build using
a test tenant. Even with the Test-ClientAccessRule cmdlet, there’s no substitute for a thorough test before you unleash
these rules on your environment.

 Real World: Microsoft offers several interfaces that developers can use to extend the various Outlook clients. For
the most part, there are three categories of plugins: those which work only in Windows Outlook, those which work
only in Mac Outlook, and those which are written using the Office extension APIs and can be loaded in both versions
of desktop Outlook, OWA, and even Outlook Mobile. For example, the familiar button that allows you to make a
meeting into a Teams or Skype meeting with a single click is an “add-in” (the preferred term for desktop-native
extensions to Outlook), while the FindTime tool is written with the Office APIs and can thus be used in multiple
clients once it’s enabled for the user. Troubleshooting Outlook add-ins on the desktop is tiresome; one of the most
common issues users report with Teams and Skype is that their add-ins don’t appear as expected, making it more
difficult to schedule meetings. Recently I ran into this with a customer whose users weren’t seeing the Teams plugin
in their Mac Outlook clients. After some tedious investigation, we found two separate problems. First is that the
Teams plugin required a specific Office Insider build of Outlook, which they hadn’t deployed. Once we got that
problem fixed, we learned that visibility of the Teams plugin is flighted in the service—not every tenant in every
region has access to it. As it happened, their tenant did not (although Microsoft support was able to upgrade the
tenant so that the poor Mac users got feature parity, in this one instance, with their Windows coworkers.)

OneDrive for Business

The OneDrive for Business client has come a long way in the last two years. It was originally based on the legacy
Groove.exe client, then Microsoft introduced what was then called the Next Generation Sync Client (NGSC). NGSC
morphed into what is now known as OneDrive or OneDrive.exe and has largely replaced the legacy client. The current
client offers full sync functionality for personal OneDrive folders, OneDrive for Business libraries, and SharePoint
Online document libraries (including those associated with Office 365 Groups), so there is little reason to use the
legacy sync client any further unless you are still syncing on-premises OneDrive for Business or SharePoint libraries,
both of which are unsupported by the new OneDrive client.

Customers with fewer than 250 Office 365 licenses must use the new OneDrive client. Microsoft still allows larger
clients to use the legacy client while they manage the transition to the new client. You should certainly be planning to
make the transition now, as Microsoft will eventually withdraw support for the legacy client.

Deploying the OneDrive Sync Client

The OneDrive client is available for Windows 7 or later, as well as Mac OS. If you've already deployed the latest Office
2016 applications to your computers using the ProPlus installation then the latest OneDrive client is installed,
however it does not automatically take over sync duties if the legacy client is already in use. If you need to deploy
OneDrive separately to Office then you can download the setup file from Microsoft . You can exclude the legacy client
from new Office ProPlus deployments by customizing the Configuration.xml file that you generate with the Office 365
ProPlus Configuration XML Editor to exclude Groove. For existing installations of Office, you can remove the legacy
client by re-running setup with a new configuration XML file that excludes Groove. A good opportunity to do this is
when you are rolling out a new build of Office to your user's computers as part of a normal update cycle.

You should also download and configure the OneDrive Group Policy template for your organization. If you plan to use
Group Policy to control sync settings for end users then you will need to modify the copy of the Group Policy ADMX
file that you place in your Central Store. You can edit the file using any text editor such as Notepad or Visual Studio
Code. Within the ADMX file replace any instances of the placeholder text for the tenant ID with your real tenant ID,
which can be found by running Connect-AzureAD to connect to your tenant. The tenant ID is the GUID string that is
displayed after a successful PowerShell connection is established.

PS C:\Scripts> Connect-AzureAD

Account Environment Tenant

------- ----------- ------

admin@exchangeserverpro.onmicrosoft.com AzureCloud 2b9bca49-687e-4e5f-8a52-21350b719b06

One of the quickest wins for OneDrive deployment is to migrate user home drives from a traditional file server into
OneDrive. Or, if your users are simply storing their personal files on their local computer, migrating them into
OneDrive will allow the files to sync to the cloud and to other computers that the user logs into, so that they can
access their files from anywhere. A demonstration of how to manage a migration of home drives to OneDrive can be
found here .

Before you begin the transition, you should run a script or a reporting tool to estimate the sizes of the user home
drives. All the data added to OneDrive will need to sync to the cloud, so bandwidth utilization needs to be considered.
If network bandwidth is a limited resource then a staged rollout of OneDrive would be sensible. Microsoft supports
silently configuring OneDrive using the user’s credentials to provide single sign-on, but this requires you to configure
the SilentAccountConfig registry key. If you don’t do that, plan on communicating to your users how to sign in to
their OneDrive client manually.

Managing OneDrive sync client updates

Whether you install the OneDrive sync client with Office 365 ProPlus or by using the OneDrive standalone installer,
the sync client application files are installed to the %localappdata%\Microsoft\OneDrive folder. This means that
OneDrive is installed as a per-user application, not as a per-computer application. It also means that OneDrive can be
installed and updated by users who do not have local administrative rights on the computer.

Microsoft releases OneDrive sync client updates through two deployment rings:

Production - Updates are released to a small percentage of clients in this ring first, and the remaining clients
receive the update within approximately two weeks. This is the default deployment ring for OneDrive installs,
and there are no controls available for including or excluding your computers from that initial small percentage
of clients that receive updates first. During the release of updates to the production ring Microsoft uses their
own telemetry to measure the success of the initial rollout. If a problem is discovered, updates will be stopped
while a fix is developed. The clients that received the problematic update will then be the first to receive the next
update to fix the issues, and then updating will continue through the rest of the production ring.
Enterprise - Updates are released to the enterprise ring only after they have been successfully deployed to the
production ring without any major issues. The enterprise ring receives updates over a 60-day window that starts
after the production ring deployment window has ended. Critical updates might be released to clients early in
that 60-day window, but otherwise you can expect your clients to update any time within that period.

The OneDrive sync client checks for newly available updates every 24 hours while it is running. If OneDrive has not
been running within the last 24 hours, it will check for updates immediately the next time it starts. Windows 10 clients
also have a scheduled task that will check for OneDrive updates every 24 hours regardless of whether the sync client
has been running. These mechanisms are all designed to keep OneDrive updated with the last bug fixes and feature
enhancements. If you're tempted to block OneDrive from checking online for updates then you'll need to periodically
check for an updated OneDrive standalone installer, download the updated application files, package them for
deployment, and then manage the rollout using your enterprise software deployment tool. That's an unnecessary
overhead for most organizations though. If you prefer to take a conservative approach to OneDrive updates then you
can configure clients to use the enterprise ring, but you should be aware that this will slow the release of new
features as well.

You can configure the update ring for OneDrive by using the OneDrive Group Policy template . In User
Configuration, Administrative Templates, OneDrive , there is a setting to Delay updating OneDrive.exe until the
second release wave . Enabling that setting places the client in the enterprise deployment ring.

Restricting OneDrive sync to domain-joined computers

For some organizations, there is a concern when deploying OneDrive for Business that users will access corporate
data from their personal computers. If the personal computers are not well secured, such as having encrypted drives
and good antivirus software, or if the personal computers are shared with unauthorized people, then the corporate
data could be exposed.

To address those concerns it’s possible to restrict OneDrive so that it only synchronizes files to domain-joined
computers. The general idea is that a domain-joined computer that is within the control of corporate IT will be more
secure than the average personal computer that staff own. OneDrive sync restrictions can be configured using the
OneDrive admin portal , or the SharePoint Online PowerShell module. For more information on configuring OneDrive
sync restrictions, see Chapter 8, “SharePoint Online and OneDrive for Business”.

Skype for Business

Skype for Business Online has broad client support for desktop, mobile, and web access by end users. Desktop users
can choose from the Skype for Business 2015 or 2016 client, Skype for Business 2016 Basic client, Skype for Business
on Mac, Lync 2013 client, and Lync 2010 client. Support for different Skype for Business Online features varies
depending on the version that you've installed, with the full comparison available on TechNet . Mobile clients can
install the Skype for Business app for iOS, Android, or Windows Phone from the app store for those platforms.

The latest version of the Skype for Business desktop client can be installed and updated as part of the Office 365
ProPlus client deployment. You can deploy it separately using the standalone installer that is available in the Office
365 admin portal. Standalone installations are appropriate if you are not deploying the full Office application suite, or
if you are using older versions of Office. If your users are licensed for Office 365 Business Essentials or Business
Premium their installation of Office 365 client applications does not include the Skype for Business client. Instead, you
will need to instruct users to download the Skype for Business Basic client and install it themselves, or deploy it
automatically using your normal software deployment method.

Updates to the Skype for Business client are managed through the same update channels as the Office 365
applications, which was covered earlier in this chapter. The features of Skype for Business Online and how they apply
to different clients are covered in more detail in chapter 16.

Teams
Teams is available to Office 365 Enterprise, Business, and Education tenants with client available for Windows, Mac,
browser, and mobile platforms. For Windows computers, the desktop client can be deployed automatically using group
policy or other distribution mechanisms .

Teams is a self-updating application. It will check for, and download, any available updates each time the user runs the
program. That makes the application simple to support (if you allow Teams to self-update), which means that
deploying Teams is basically a task of running the installer once, and then not running it again. The downside to this
approach is that you’ll never really know what versions of the Teams client you have on your network, or when they
are updated. As of June 2018, Microsoft maintains a change log of Teams updates , but generally they just shovel
updates out to the world-- there is no way to put different users in different update channels.

Proxies and Network Devices

Client applications and servers within your corporate network that will be connecting to Office 365 will need to have
access to all the appropriate endpoints. The general recommendation from Microsoft is to bypass any web proxies,
and bypass other network devices such as WAN accelerators, for connections to Office 365. That recommendation is
counter to the security policies of many organizations, with the adoption of cloud services making them more likely to
want to push all their internet traffic through a security or performance device.

There are good reasons for customers to want to secure their internet traffic. These days, clients on their network will
be connecting to more services over HTTPS, making the traffic invisible to traditional security measures. In response,
organizations implement security devices that can intercept, decrypt, and inspect HTTPS traffic to detect viruses,
data leakage, and ransomware. Similarly, with so much traffic now running over internet connections, organizations
try to reduce the impact on their bandwidth with WAN accelerators and traffic caching devices.

Unfortunately, such measures often result in degraded performance for clients, and in some cases, they completely
break connectivity to Office 365 endpoints, causing problems such as Offline Address Book download errors,
OneDrive for Business sync failure, and unpredictable Skype for Business calls. The specific will vary depending on
the vendors involved and their ability to provide guidance and templates for properly implementing their devices into
your network without impacting client traffic. If you’re going to attempt to implement such measures, approach it
with caution and do ample testing to ensure you don’t cause problems for your end users, particularly those on BYOD
devices that will require special considerations for any SSL traffic inspection you plan on doing.

There may be a happy middle ground if your chosen security devices are not fit for use with Office 365, but you want
to continue using them to manage other internet traffic. Bypassing security devices for clients connecting to Office
365 services is a reasonable approach, but can be poorly implemented if you’re not careful. The natural inclination of
firewall administrators is to apply controls based on IP addresses. But as Microsoft explains in their guidance , Office
365 IP address ranges are a constantly moving target. To keep up, you might be making changes to firewall
configurations as often as every 14 days, which is a heavy burden to carry for very little real benefit. Microsoft
recommends filtering based on URLs instead , which not all firewalls can support.

No matter the security devices at play, NATing of outbound connections to Office 365 services is also a concern.
Simply put, if too many clients are being NATed to the same public IP address, port exhaustion can occur which
results in degraded performance for the clients. NATing is fully supported, but Microsoft recommends limiting the
number of clients on one public IP to around 2000. This is mainly due to the number of connections a single client can
have open. Outlook alone has 2-4 connections open for a basic user scenario, and more if they are also connecting to
shared mailboxes, Office 365 Groups, or are performing an OAB download. Once you add in other applications such as
Skype or a web browser, the number of connections per client can easily be 10 or more. With a maximum of 64000
ports per public IP address, limiting NATing to 2000 clients per IP is a safe place to start. A pool of public IP
addresses will be needed for NATing outbound connections if you are running a very large organization.

Real World: If you use a proxy, security, or performance device for Office 365 client traffic, and you notice
problems, bypassing the device is a good troubleshooting step. The goal is not to point the finger of blame at the
network devices, rather it is to narrow down the likely cause of the issue. Ultimately if you choose to implement such
network devices in your environment, you are better off identifying issues and working to resolve them rather than
pretending they do not exist.

Mobile Devices and Applications

Although email is a very popular app for mobile devices, Outlook is not the only app that Office 365 implementers
should consider. Microsoft produces a wide array of apps for iOS (Figure 10-8 ), Android, and Windows Mobile that
are useful for Office 365, including:

Outlook (including the calendaring functionality from the deprecated Sunrise app).
SharePoint Online.
Skype for Business.
OneDrive (the same app can connect to both OneDrive for Business and OneDrive consumer).
Delve.
Word, Excel, and PowerPoint.
OneNote.
Office Lens (to capture information and post it to OneDrive or OneNote, Word, PowerPoint, or Outlook – you can
 also use the embedded version of Lens in some other applications like Word and PowerPoint).
Yammer.
Microsoft Teams.
Office 365 Video.
Sway.
Office 365 Admin.
Authenticator (used for multi-factor authentication).
Microsoft To-Do.
Planner.

Companion mobile apps for new apps typically appear soon after an application reaches General Availability.

Figure 10-8: Office 365 mobile apps installed on an iPhone

More detail about how to manage mobile device and application access to Office 365 is in Chapter 18.

On to Groups
Now that we know how to configure and manage clients, we can delve into the secrets of an Office 365 application.
Let’s go and consider the finer points of Office 365 Groups.
Chapter 11: Office 365 Groups
Tony Redmond

This chapter covers Office 365 Groups, or as they are sometimes called, "modern" groups. When Microsoft launched Office 365 Groups in November 2014, they positioned groups as a new
way to enable team working that brought Exchange and SharePoint together in a more integrated manner than had been possible before. At the time, Microsoft wanted to replace site
mailboxes, a previous attempt to integrate Exchange and SharePoint that proved successful in some customers but failed in many others. The combination of group mailbox and team site
worked much better than site mailboxes, especially for users who organized their working life around Outlook.

In April 2017, Microsoft said that “more than 10 million people rely on Groups in Outlook every month to work together and get things done.” At that point, Office 365 had roughly 100 million
active users, so Groups had 10% penetration. Since then, distribution groups persist, and the number of people who rely on Groups is much higher because of way that the mission for Office
365 Groups has evolved to be the membership service for a wide range of Office 365 applications, including Teams, SharePoint team sites, Yammer, and Stream. While continuing to be a
“better distribution group” for the email-centric community, the role of Groups in membership management is now more important within Office 365. Most of the recent developments have
occurred to make Groups more manageable, especially in large deployments.

A Group Identity and Membership Service

The basic idea behind Office 365 Groups is to bring together people, information, and applications to enable better communication and collaboration within Office 365. One way of thinking
about this is that Office 365 Groups deliver a single identity that is valid across the entire service. Using that identity allows application developers to break down the information silos that
often exist in large companies.

Figure 11-1: Office 365 Groups form the fulcrum for collaboration

Figure 11-1 illustrates how Office 365 Groups are now the central fulcrum for collaboration within Office 365, so much so that Office 365 Groups have become a consumable service rather
than an application. Each group has a common identity and membership. Base workloads like Exchange and SharePoint deliver functionality like email or document management that
engineers can combine with other components to build new applications. In effect, applications like Exchange and SharePoint serve as software toolkits for other applications. Signals that
capture user interaction with the applications accumulate in the Office Graph. Applications can then use the Office Graph data to make suggestions to users as to which groups they might
like to join, and so on.

The array of different groups available within Office 365 can be confusing, but the situation reflects a rapid evolution of the collaboration landscape as Microsoft leverages some of its on-
premises history and creates new offerings specifically for the cloud. The net result is that the term Office 365 Groups now refers to a cross-application membership service in addition to a
set of applications built around the service. Microsoft summarizes this concept in three steps:

1. Someone creates a group through their application of choice. The application depends on the user’s personal preference and the purpose for which they want to use the group. For
instance, if they believe that the group needs to collaborate via email and documents, they might choose to create the group with Outlook.
2. The new group is an Azure Active Directory group object. The new group object holds the membership of the group. Azure Active Directory synchronizes information about the new
group to other application-specific directories throughout Office 365 to make those applications aware that a new group is available. In some respects, because both types of group
control access to resources, you can think of an Office 365 group as very similar to a security group. The big difference between the two types is that an Office 365 group has both
members and owners while a security group has just members. In addition, a workload must recognize Office 365 Groups as a valid security mechanism before Groups can be used to
control access to resources.
3. Group members gradually connect and use the range of resources available to groups to build out the “group experience.” Another way of saying the same thing is that users select the
tools they need to get work done from the range of applications that support Groups.

We will explore how these steps work in practice through the rest of this chapter.

Applications that use Office 365 Groups

To distinguish their usage, Microsoft sometimes refers to the original “Office 365 Groups” application as “Groups in Outlook”. This is reasonable because the clients for these groups are the
Outlook family. The other applications that consume the Groups service to manage membership are Microsoft Teams, Microsoft Planner, modern SharePoint team sites, and Yammer Groups.
Table 11-1 lists the different forms of groups that consume the Office 365 Groups service.

Group Focus Storage

Groups in Outlook Email-based conversations within public or private teams. Limited to “more than” 1,000 members the groups used by Teams support 2,500 Exchange Online
members). Dynamic membership supported. mailbox

Microsoft Teams “High-velocity” chats within closed teams. Limited to 2,500 members. Azure data service

Microsoft Planner Task-based plans worked on by public or private teams. Limited to 2,500 members. Azure data service

Yammer Groups Open, idea sharing, collaboration at scale. The membership limit for these groups is up to 50K if synchronized to an on-premises Active Yammer
Directory and higher if not.

SharePoint modern Structured document and information management. SharePoint

team sites

Stream Controls access to video content in the Microsoft Stream service, Stream

Power BI Controls access to workspaces Power BI

Table 11-1: The various kinds of groups available within Office 365

Explaining the Member Limit

The 2,500-member limit for Office 365 Groups is based more on practicality than technology. In other words, the limit exists because we know that concurrent access to a shared mailbox
becomes problematic as the number of connections grow. Not every member will access the mailbox (to read conversations or look at the calendar) at the same time, so 2,500 members is a
“safe” limit, assuming normal usage patterns. If you have groups where members seldom access the mailbox, you can have more members. There are groups in production today that surpass
the 2,500-member mark by a considerable margin because the tenant knows that only a small percentage of the membership ever access the group concurrently.

The only actual limit for groups is imposed by Azure Active Directory , where “any number of objects can be members of a single group,” or, if you synchronize with an on-premises directory,
an Azure Active Directory group object is limited to 50,000 members. The higher limits supported for groups by Azure Active Directory does not mean that you should rush to create the
world’s largest Office 365 group. Instead, you should be aware of why the limit exists and take this factor into consideration when you plan how to use groups.

Core Principles
As seen in applications such as Teams and Planner, the single identity represented by a group and the ease of managing group membership are attractive for developers who need to enable
access to an application. If we look at Figure 11-2 , we can see how three core principles combine to form the architecture of Office 365 Groups. These are:

A single identity for the group exists in Azure Active Directory , which serves as a single point of truth for the ecosystem. Information about group properties and membership is
propagated by Azure Active Directory in a “forward synchronization” process to the directories used by individual applications to enable those applications to recognize and respect the
membership of groups. You can think of the single identity for a group as a form of “access all areas” pass that is valuable across all the resources available to the group.

Two writes occur in tandem to create a new Office 365 Group. The first is to Azure Active Directory to create the object that becomes the definitive identity for the new group. The second is
to the EXODS directory to create the group mailbox. This process ensures that the new group can begin to host conversations at once. After Exchange Online creates the new group mailbox,
it updates the group object in Azure Active Directory with information such as its email address. Provisioning processes create other components like the SharePoint Online document library
and the OneNote notebook. The information written into application-specific directories like SPODS use a background process to inform Azure Active Directory of details such as the URLs
used to link to the SharePoint site.

Once instantiated, future updates by administrative interfaces (including PowerShell) flow to the group object in Azure Active Directory, which then pushes the updates to the individual
applications used by the group. For instance, if a user updates the membership of a group using OWA, dual writes occur to update the membership information held in Azure Active Directory
and EXODS. A notification then occurs to make SharePoint Online aware of the new data. Much the same happens for groups that use Yammer to store conversations. In this case, three
writes occur because the group uses components from Exchange (the calendar), Yammer (conversations), and SharePoint (files).

Figure 11-2: The loosely coupled architecture of Office 365 Groups

Although notifications are the primary mechanism used to inform workloads of changes to which they should respond, applications might not respond at once to the notifications as they
might have other items to process. The forward synchronization process, which runs in the background on a timed basis, acts as a backstop by making sure that all workloads apply any
outstanding updates consistently. The focus of the synchronization process is scale rather than immediacy. In other words, a change might take one or two minutes (or sometimes much
longer, depending on the load on the service) before replication fully occurs across all directories. Although the change might not be immediate, it will happen. This is important because as
more applications use Office 365 Groups as a membership service, the interaction between those applications and Azure Active Directory becomes more complex. Typically, if an application
has its own directory (like Teams or Yammer), the application updates Azure Active Directory and its own directory with a change and relies on the background synchronization to spread the
change to all other directories.

Apart from working with dynamic Office 365 Groups (see Chapter 12), it is best to manage Office 365 Groups via the Office 365 Admin Center or EAC rather than the Azure portal.

Viewed in the context of Azure Active Directory, Office 365 Groups are both security enabled and mail-enabled. Administrators usually understand how Office 365 Groups function in terms of
email, but sometimes overlook the security aspect. Because they act like security groups, Office 365 Groups can control access to some other Office 365 resources, such as the plans created
though Microsoft Planner.

In hybrid environments, tools like Azure AD Connect can synchronize group membership to an on-premises Active Directory where the groups show up as normal distribution groups. You
need to pay some care and attention needs to the synchronization process to make sure that everything works as expected.

A set of federated resources is provided by applications drawn from across the service . Office 365 spans a growing set of applications, each of which delivers specific functionality.
Each application controls its own data stores. Federation allows Microsoft to select from the functionality available in existing applications and expose that functionality to end users through
new applications and their clients. Thus, an Outlook Group uses the following components:

A mailbox to store threaded conversations from Exchange Online.

The same mailbox includes a shared calendar.
A site collection from SharePoint Online. A team site including a document library allows group users to share documents and other files.
A shared OneNote Online notebook. The notebook is in the Site Assets document library of the SharePoint site.
Other applications can add their own components to the mix. For example, Teams adds its Azure-based data services to hold messages and graphics.

A Yammer-based group uses similar components, with the difference being that the mailbox only stores the group calendar. When a group is created by Microsoft Teams, it offers members
the ability to communicate through chats in different channels. If a group is used by Stream, it can control what videos are available to group members. Likewise, Power BI can use a group to
control the workspaces that are available to users.

The single identity for the group means that members automatically gain access to all the application resources available to the group and creates an integrated end-to-end user experience
across Office 365. Federation allows Microsoft to introduce new applications to Office 365 more easily and with less disruption to other applications than would be the case if each application
depends on its own resources. Members of the group can then use whatever functionality is exposed by the application. The functionality and data stays under the control of each application.

Applications are loosely coupled with groups and each other . The applications that contribute to Office 365 Groups operate independently of Groups and can exist independently of
each other. Applications are aware of each other because they share the common linkage through Groups and respect the common identity represented by a group. When changes occur in an
application, they can make other applications aware of updates through a synchronization process. For example, if a user creates a new group with Outlook, the notification that arrives from
Azure Active Directory to Exchange Online forces the provisioning of a new group mailbox to hold group conversations and the calendar. The presence of a new group object in Azure Active
Directory makes SharePoint Online aware that some work is necessary to update itself when someone tries to use the new group.

Although it is easy to see how Office 365 Groups can replace some forms of email distribution groups (and we discuss how to convert distribution groups to Office 365 Groups in Chapter 12),
they are not yet able to replace the other types of groups in widespread use within Exchange Online. For example, you cannot nest an Outlook Group inside other groups. In addition,
Exchange distribution groups allow multiple types of email-enabled objects to exist in group memberships. Because Office 365 uses groups to control access to applications and data, Office
365 Groups only support Office 365 user accounts as members while distribution groups can also include shared mailboxes, mail users, mail contacts, and mail-enabled public folders.

The fact that a group has a single identity with a single set of permissions is a major part of the value of Office 365 Groups. It means that groups can act as a means of access to different
Office 365 applications, which is how applications like Forms and Power BI use Groups. Forms, for instance, uses Groups as the way to share forms between users in what it calls “Group
forms.” Because a group is a single identity, when members join a group, they gain access to all the components available to that group with the same rights to the content as other members.
If you need to implement granular access to information, you should use whatever other specific permissions are supported by individual applications rather than trying to turn the team
identity of a group into a set of individual custom permissions.

The Evolution of Office 365 Groups

Before we examine the details of how Office 365 Groups work, we should first consider some of the basic ideas behind how Microsoft views these objects and how this affects their evolution
as a major component of Office 365. Here are some of those ideas:

A focus on self-service : Microsoft intends Office 365 Groups to be easy for users to setup on an on-demand basis. The default permission allows any user with a mailbox in a tenant to
create a new group (as we will discuss later, you can control the ability of users to create a group with an Azure Active Directory policy). The only thing a user must give is a name for the
group that Office 365 can use as the unique alias for the new group. Users creating objects that consume system resources without oversight flies in the face of traditional IT practices where
IT exerts control over what a user can and cannot do. The idea is that by making groups easy to create, users can collaborate as they see fit and work together to in ad-hoc or formal teams.
Remember that resources are more abundant inside Office 365 than is the case in most on-premises deployments, so a reduced focus on the need to control resources is an understandable
position to take.

Public by default : Originally, the thought adopted by Microsoft was to make groups public by default to encourage people to join and take part in as many groups as possible. That was the
thinking in 2015. Several years on, the experience of deployments and customer feedback moved Microsoft to believe that groups should be private by default. The change, which rolls out
gradually across all clients from March 2018, aligns Office 365 Groups with Teams as both apps use private as the default access type. It has a side effect of making groups less discoverable,
but it is easy to set a group to be public once it is created if you want to follow the original philosophy.

Simple permissions : The history of IT is full of stories about people getting into trouble with permissions. Either they do not have the right permissions to do the job or their account has
permissions that are unnecessary, which means that the user ends up accessing data that they should not. Once a user is a member of an Office 365 Group, they have full access to the
contents available to that group, no matter what application provides and manages that content. If someone leaves the group, they lose access to the information belonging to the group. It is
as simple as that. The downside of this approach is that non-members have complete access to information held in public groups, including the ability to remove data if they wish.

Sharing is easy : Users can share personal documents stored in their OneDrive for Business site with other people, including external users. Groups can share documents with other users
and other groups just as easily. The only difference is that group documents are owned by the members of the group. Thus, you should never put a document into a group document library
unless you want to grant full control over that information to every other member of the group.

A shared memory : As discussed before, administrators often bolted public folders onto distribution groups to create a shared memory for members of the group. Office 365 Groups
incorporate a shared memory from the start in that once someone joins a group, they enjoy full access to all the information that has accumulated in the group since its initiation. Nothing is
secret or hidden from group members. The shared memory makes it easy for new members to quickly familiarize themselves with the work of the group without the need for an existing
member to send them email, share documents, or update them with details of group meetings. Everything is present for the new member to explore.

A growing experience : Members might begin by taking part in group conversations via email, just as they have used distribution lists for decades. Over time they can become more fully
immersed in the work of the group and begin to use the other facilities to interact with other people. Since their introduction, Microsoft has steadily added new features to Office 365 Groups.
More importantly, as shown in their use by the Microsoft Teams and Microsoft Planner applications, Groups have become the keystone for Office 365 collaboration. Indeed, if you look at how
Microsoft presents the collaboration options now available within Office 365 (Figure 11-3 ), you see that Groups is one of the three foundational elements.

Figure 11-3: Office 365 collaboration options (source: Microsoft)

In all cases, the group object is both a single identity and membership list that applications can use to control access to resources. All the group types have an Exchange Online mailbox for a
group calendar, a SharePoint team site to store Files, a shared notebook, and a plan. The only difference between Office 365 Groups and Yammer is how conversations occur between
members and the storage for those conversations.

Some criticize Microsoft for making too many collaboration options available within Office 365. That could be true if you try to use everything. However, the experience is that tenants tend to
choose the options that match their business needs and culture best and stay with those choices. The important thing is that the available options accommodate the needs of customers and
that the options are becoming increasingly integrated over time.

Group Members
The functionality enabled through Groups is based on the single identity represented by the group objects stored within Azure Active Directory. The single identity and simple permissions
model create a low-touch need for administration. Membership information exists as sets of links that connect back to user accounts. Three types of members exist:

Owners (sometimes referred to as admins): These users can update group properties and add members and owners to the group. A user must be a group member before they can be an
owner.
Members : These users have full access to all the resources of the group. However, they must connect to the group to access conversations. Members can add new members to a public
group or request that an owner adds someone to a private group.
Subscribers : These are members who have opted to receive copies of conversations and group calendar events via email. A group does not have to have any subscribers, which is the
normal case for the groups used by Microsoft Teams where conversations occur through chats rather than email.

Membership is dynamic or static. You can only create dynamic Office 365 Groups through the Azure Active Directory portal where the membership types are “dynamic user” and “assigned.”
You cannot change the membership type for a group from dynamic to static afterwards.

An Office 365 user can be present in all categories of membership. Guests can only be present as Members and Subscribers and cannot be group owners. Members can elect to take part in
conversations via email, in which case they join the set of subscribers. Except for dynamic groups, administrators and group owners manage the three types of member by manipulating the
links belonging to the group. You can do this through group-enabled applications like OWA or Teams, the Outlook mobile apps, or by running the set of PowerShell *-UnifiedGroupLinks
cmdlets as discussed in Chapter 12.

A group can become ownerless when the account of its last owner is removed from Office 365. However, the group will continue to function. New members can continue to join if the group is
public but membership requests to join private groups stay unapproved because no owner exists to process requests. A tenant administrator can add a new owner by either editing the group
through the Office 365 Admin Center or EAC, or by running the Add-UnifiedGroupLinks cmdlet.

Public or Private Groups

The access type for a group can be private or public. A private group is one where the group owners control the membership. This is the most suitable group to use when group content is
confidential. If a group’s membership is open, anyone can view the membership of a private group but not the content. Users can ask to join a private group, in which case the group owners
receive their request to join. Any of the group owners can allow or deny a request to join. The default access type for a new group is public, but this will change when Microsoft rolls out an
update in late April 2018 to make Private the default. OWA is the first client to respect the new default, followed by the other Outlook endpoints (Windows and Mac desktops and iOS and
Android mobile apps).

You can change the default access type for Outlook-created groups in a tenant by updating the organization configuration. This value does not affect groups created from non-Outlook
endpoints.

[PS] C:\> Set-OrganizationConfig – DefaultGroupAccessType Public

Public groups are open to any user in the tenant. Anyone can join a public group if they wish. In fact, a user can enroll other users into public groups without their permission (the newly
enrolled users can then decide whether they want to take part in the group).

You can change the access type for a group using the following methods:

1. OWA: Select the group, edit its properties, and switch the type. For more details, see the section about creating and updating groups with OWA later in this chapter.
2. Outlook: Select the group, then Edit Group and update the setting.
3. Teams: Select a team and use Edit Team to change the setting.
4. Outlook mobile app: Select the group, edit its properties and change the privacy setting.
5. Planner: Select a plan and then Edit Plan to move the Make this plan public slider to Yes or No.
6. EAC: Select the group and edit its settings. The privacy setting for the group is in general settings.
7. PowerShell: Run the Set-UnifiedGroup cmdlet and set the AccessType property to be either Private or Public.

[PS] C:\> Set-UnifiedGroup –Identity TestGroup –AccessType Private

Interoperability with on-premises Exchange : Office 365 Groups are unique to Office 365 and do not exist in on-premises deployments of Exchange. If they wish, hybrid deployments can
synchronize Office 365 Groups to on-premises Active Directory with the AADConnect tool to make the objects available in the GAL. However, the synchronized groups then behave like
traditional distribution groups. Synchronization makes it more convenient for on-premises users to address Office 365 Groups, but even without synchronization, on-premises users can
contribute to group conversations by sending email to the SMTP address of a group. On-premises users can also receive and process meeting invitations from group calendars.
Conversations
Aside from sharing documents, the basic way members communicate within a group is through threaded conversations. The items that make up conversations exist in the Inbox folder of the
group mailbox for Office 365 Groups or the Yammer data store for Yammer Groups. In either case, clients create a view to sort the items and present them as threaded conversations. Users
can contribute to conversations through the clients or by sending email to a group. When users contribute to a group, they will not receive the message in their Inbox because the Exchange
transport system automatically suppresses that copy. Users can find copies of any messages they send to groups by going to the relevant group or by checking the Sent Items folder in their
mailboxes. Users can post items to conversations directly or through email. In either case, items flow through the normal transport processing pipeline. Any transport rules present in the
tenant, including data loss prevention rules to check for the transmission of sensitive data types, process items as they progress through the pipeline. The transport system rejects items that
fail checks in transport rules and generates a non-delivery notification for the originators.

Each contribution to a conversation is an individual item. Clients create a threaded view to arrange the items in a conversation using the conversation identifier to generate the correct order.
This is a similar mechanism to that used to sort items in personal mailboxes into a conversation view. The calendar is the other major folder used in a group mailbox. It functions much like
any other calendar in a personal or shared mailbox, with the notable exceptions that you can’t use granular permissions with group calendars.

Every email message in an Exchange mailbox has a property to mark if the user has read the message. Clients use the read status to know what to display to users for the count of unread
email. Groups has a similar concept in that clients also display information about the number of unread conversations, or rather, the number of conversations that include items that the user
has not yet seen. Email clients reset the unread status when a user accesses a message to read it. Groups takes a different approach in that it sets the read status for all unread conversations
in a group to read when a client moves its focus away from a group. For example, if you access a group with 50 unread conversations, and then move to another group, the change in focus
causes Groups to reset the read status for the 50 unread conversations. Because this behavior differs from email, it has caused some confusion with users. Microsoft is considering how to
solve the problem and hopes to have a solution in 2018.

An easy way to move items from personal mailboxes is to drag and drop items from mailbox folders to a group mailbox. For now, OWA is the only client to support this capability. You can
move items from any folder in your personal mailbox or any shared mailbox to which you have access. The moved items go into the inbox folder in the group mailbox and behave like any
other mailbox item. If you reply to the item, the recipient list includes all the addressees from the original message plus the group.

Accessing folders in group mailboxes : Although group mailboxes have a complete set of default folders that can hold items, clients usually only expose the Inbox and Calendar folders.
The folders function in the same way as they do in other mailboxes. For instance, junk mail sent to a group goes into the Junk Email folder, but no one knows about it. The Sent Items folder
stores copies of messages generated by clients, such as notifications to users that someone has assigned them a task using Planner. If a group is team-enabled, you also find compliance
records for channel conversations in the Team Chat folder, and if the group is subject to a hold, the folders used to keep copies of items until the hold lapses are present. You can view the
complete set of folders in a group mailbox by running this command:

[PS] C:\> Get-MailboxFolderStatistics -Identity GroupName | Format-Table Name, ItemsInFolder

Unless you are curious and want to explore what is in the mailbox, no real reason exists to access the other folders and you can safely leave them alone. However, if you want to, you can
access the folders in a group mailbox by adding it as a shared folder with OWA. When added, OWA treats the group mailbox exactly like a shared mailbox and you can open any of the folders
and examine the items stored within.

Document Libraries
Apart from conversations and the shared calendar, the most obvious difference between Office 365 Groups and traditional email distribution groups is the document management (or Files)
functionality available through the SharePoint team site owned by a group. When you create a new group, the provisioning process uses a site template called GROUP#0 to create a
SharePoint Online site collection for the team site. You cannot change the template to apply other settings, such as assigning a storage quota (which you can update afterwards using the Set-
SPOSite cmdlet). The site holds a document library and the group notebook. The site collection only supports a single site as some technical issues with permissions prevent Groups using
sub-sites. All the limits applied to SharePoint site collections apply to the site collections used by Groups.

Users can add content to document libraries using the following methods:

Drag and drop files from any location accessible to your PC (local drives, OneDrive for Consumers, file servers, etc.). However, you cannot drag and drop files from another SharePoint
Online site or group document library. We will discuss later how to move files documents from a SharePoint Online site to a group document library.
Upload files or the contents of entire folders.
Create files from scratch using the Office Online applications.
Save files into the document library from desktop Office applications.
Save attachments received by email into the document library using Outlook 2016 or OWA.
Use the OneDrive for Business synchronization client to upload and manage files. As explained in Chapter 8, you must use the new OneDrive for Business client (OneDrive.exe) to
synchronize SharePoint document libraries. The older (Groove.exe) version does not support SharePoint document libraries.

When a file is in a Group’s document library, the site controls its permissions. All documents in the library inherit the access to allow group members equal access to the content. SharePoint
checks permissions when a user makes a request to access a file by looking up group membership to find whether the user is a member of the group at that point.

The storage used by group document libraries counts against the overall allocation for the tenant. Each site collection can store up to 25 TB. If you use a single document library for the site,
it can hold the complete 25 TB if the library stays under the 30 million document limit for a library. However, in practical terms, it is more likely that you will meet the 5,000-file limit in a view
first. SharePoint imposes this limit for performance reasons, so if you want to use larger document libraries, you should consider using different views or split the documents across multiple
folders, each of which has its own views.

If the group is public, any user in a tenant can join the group and access the files in the document library. It is also important to remember that all content in the group document library is
owned by the group rather than the user who creates the file. If someone leaves a group, all the content they authored or amended is left behind.

The Files View : Telemetry revealed that many users do not use the Files capabilities within groups and continue to circulate attachments via email, even when a group document library is
available. The Files view uses information held in the mailbox to show all the files available to the group, including items in the document library as well as attachments to emailed
conversations. In addition, the Files view gives members the ability to work with attachments in the same way that they can for messages they receive in their personal email. Users can
download, view, or edit attachments and email the amendments back to group members and other recipients. When a group member edits an attachment, Groups stores the edited file in the
Email Attachments folder in the group document library.

Figure 11-4 illustrates a view of documents stored in a group document library. The interface used is the same as used for other SharePoint Online document libraries. You can change the
view to present items in the order you think most useful. For instance, some like to list items in document libraries in a last-modified order. In addition, you can pin up to 3 items to the top of
the list. Because all members have equal access to a group document library, it follows that anyone can pin a document and that anyone can remove a pinned document and replace it with
something they prefer.

Figure 11-4: A document library for an Office 365 Group

The following features are supported by group document libraries:

Edit a file with an Office Online application or with an application running on a PC.
Check documents in and out of the library to ensure exclusive rights to change content in a file.
Share a document with an individual user or another group.
Synchronize the library to make documents available offline.
Save email attachments to the library (logically, they go to the “Email Attachments” folder).
Rename, delete, move, or copy a file.
Attach a workflow to a document.
View the version history or properties of a file (only major versioning is supported).
See who shares a file.

The “Alert me” option is in the menu bar (for a folder) or the ellipsis menu (for selected documents). Alerts allow a user to set up notifications when the contents of the library or specific
items change. The alerts can be sent by email or SMS for different conditions such as when a user adds a new item to the library. You can configure an alert to occur at once or on a digest
basis (daily or weekly). The feature is especially useful when you need to keep track of changes made to a document updated on a frequent basis.

Document Library Regional Settings : Many of the document libraries created for Office 365 Groups use the Pacific time zone (UTC –8 hours) as their regional setting, which has the
effect that users see date and timestamps for documents as if they were in that time zone. You can change the time zone and locale (country) through the Regional Settings for the site that
holds the document library, or you can set the default locale and time zone for new sites as described in Chapter 8.

Use Cases
Before we head into the details of how to use Office 365 Groups within a tenant, it’s worth considering the two most common use cases for deploying Office 365 Groups for collaboration
purposes.

The most obvious use case is as a direct replacement for distribution lists. The advantage is that a group comes with a mailbox and calendar, meaning that conversations are captured and
accessible to all and that group meetings show up in a shared calendar. Because every group has a document library, group members have a way to share files with each other. The downside
is that Office 365 Groups only support cloud mailboxes while distribution lists support all types of mail-enabled recipients, including other distribution lists. Distribution lists also support
moderation. If you can live with these restrictions, Office 365 Groups are an excellent replacement for distribution lists, especially for organizations who have completed their move to Office
365.

Less obvious is when Office 365 Groups replace shared mailboxes. In this case, the desire is to give people who access the shared mailbox a common identity (membership) and access to a
document library while preserving the ability to process incoming and outgoing email. Office 365 Groups score highly as a replacement for shared mailboxes except that clients only interact
with the Inbox folder in the group mailbox. In other words, you can’t file messages in different folders, which is often needed for shared mailboxes that handle inbound customer requests. It’s
possible to use folders in the document library as a workaround. Again, if you can live with the restrictions, Office 365 Groups can replace shared mailboxes.

The Outlook for iOS and Android clients can access the mailboxes of Office 365 Groups and the SharePoint mobile client gives access to the document library. As they are purely an
addressing and delivery mechanism, distribution lists don’t need a mobile client. Shared mailboxes are not formally supported by the Outlook mobile clients (see chapter 7 for details).

Aside from these instances, the use case for Office 365 Groups is to be a membership service for applications such as Teams, Planner, and Power BI.

Implementing Office 365 Groups

Microsoft regards groups as a platform for user-driven collaboration. In an ideal world, or a small tenant, this approach works, and users create the right number of groups, each of which is
used for its assigned purpose. Allowing everyone to create new groups in larger organizations is more problematic because the potential then exists that underused or unwanted groups will
linger in the directory and absorb resources. The latter point really does not count because Microsoft controls the resources and tenants do not incur cost when a user creates a group. In
general, it is better to set out a well-defined set of rules to govern the creation of groups and communicate them to users up front than it is to try and apply rules retrospectively to control a
profusion of unwanted groups.

Here are some simple questions to help show the need for a new group:

What purpose will the group serve? If the person who wants to create a group cannot say why the group is necessary and how it differs from existing groups, then there is a fair chance
that the new group should not be created. Sometimes you cannot avoid creating new group, as in the case when a Planner creates a new plan, which results in the automatic creation of
a group to support the membership and conversations for the plan. See Chapter 15 for more information about Microsoft Planner.
Who is the intended audience for the group? Is the group to be public (anyone can join it) or private (restricted to a defined set of users)? Will the group allow people outside the group
to send messages to its members via email?
How will users interact with the group? OWA, Outlook 2016, and mobile clients are available, but the situation is less satisfactory with other clients which lack the necessary user
interface to present conversation, documents, and other group resources. If you expect that more than a thousand people might want to join the group, it might be better to create a
Yammer-based group and use Yammer rather than Exchange to host discussions.
Is an Office 365 group the right answer? Although it might be attractive to position Office 365 Groups as the answer for all possible collaboration requirements, this is not correct.
Microsoft Teams is a collaboration platform (see Chapter 13) that is a better fit for some scenarios where email communications and functionality such as the ability to work offline is not
important. A distribution group might be the right answer for a team that simply wants to swap messages with each other. A shared mailbox might be a better answer for a team that
needs to receive and process inbound messages from partners or customers and then respond using a common identity. It is important to select the right technology to meet the need of
the people who will use it instead of trying to force one answer for all.

If enough reason is available to justify the creation of a group, we then move to its name. Many organizations use naming conventions for user mailboxes, distribution groups, and other mail-
enabled objects to help users find recipients in the directory, such as listing users via last name followed by first name. The same logic applies to groups and is the reason why naming policies
are available within Office 365.

Managing the Lifecycle of Groups : The possibility always exists that a repository like a public folder, SharePoint Online team site, or group will gently decay over time as members lose
interest or the original reason to create the repository disappears. Public folders suffer badly from age rot. If you have the necessary licenses, you can apply a group expiration policy to
detect and remove groups after they reach a certain age. Group owners receive notifications before Office 365 removes a group and have the chance to renew it for another period. See
Chapter 12 for details about the group expiration policy.

Creating New Office 365 Groups

If the AAD policy for Groups gives them the necessary permission, a user can create new Office 365 Groups through the following interfaces:

The Office 365 Admin Center and the Teams and Skype for Business Online Admin Center (create a new team).
Email clients – OWA, Outlook 2016, Outlook for Mac (release 16.9 on), Outlook for iOS and Android.
Applications that integrate with Office 365 Groups, such as Power BI, Dynamics 365 , Microsoft Teams, Stream, and Microsoft Planner.
The Exchange Online Administration Center (EAC).
The PowerShell New-UnifiedGroup (Exchange Online module), New-AzureADMSGroup (Azure AD V2 module), or New-Team (Microsoft Teams module) cmdlets.
SharePoint Online and OneDrive for Business when creating a new team site or when linking an old team site to an Office 365 Group.
Yammer.
The Azure Active Directory portal.
The Microsoft Graph API .

Administrators most likely create new groups through the Office 365 Admin Center, EAC, or PowerShell while users are likely to use OWA or Outlook. To create a new Office 365 Group from
the Office 365 Admin Center, go to the Groups section and click Add a Group . Make sure that you select “Office 365 Group” as the group type to create (Figure 11-5 ). Next, you must
populate the basic properties for the new group. These are:

Group name . This is the name that is visible to users through the directory. Make sure that the name clearly conveys the intention and purpose of the group because it is highly likely
that this is the sole information people will use when they decide whether to join the group. It is sensible to check the directory beforehand to ensure that the name you want to use does
not clash with an existing object such as another group, a team, email distribution group or shared mailbox. If necessary, you can rename a group after creation by editing its properties
or by running the Set-UnifiedGroup cmdlet to update the DisplayName property. As explained in Chapter 12, a tenant can enable a group naming policy to apply a suffix and/or prefix to
the group name. The display name for a group can be up to 256 characters and can contain periods.
Group email address . Groups constructs a unique SMTP email address and Alias (or mail nickname) for the new group. Groups creates the identifier by removing any spaces from the
group name and checking against the directory to make sure that the value is unique. If not, you must change something in the identifier to make it unique (for instance, appending a
number to its end). You can add periods to the identifier if the period is surrounded by other text. For example, “Sales.Group.1” is OK, while “Sales.Group.1.” is not (see the
documentation for the New-UnifiedGroup cmdlet for more information).

Groups then combines the default email domain for the tenant with the identifier to form the SMTP address, which also must be unique. For example, if the
identifier is SalesProfessionals and the tenant default email domain is contoso.onmicrosoft.com, the SMTP address will be
SalesProfessionals@contoso.onmicrosoft.com. After creation, you can add more email addresses to the group or change the primary email address. For instance,
you might decide to assign an address from a vanity domain to the group. In addition, Azure Active Directory prohibits the use of certain well-known or “highly-
privileged” words in mail nicknames to avoid any possibility that users create groups with these words. The names are “abuse”, “admin”, “administrator”,
“hostmaster”, “majordomo”, “postmaster”, “root”, “secure”, “security”, “ssl-admin”, and “webmaster.” Only administrators can create groups that include these
terms in identifiers.

Description . A free-text description of the role and intentions for the group. You can add whatever you like here but the text should tell users and administrators why the group exists
 and its intended use. In a multi-lingual organization, you might like to include translations of a sentence outlining the group purpose in the most commonly used languages. If a group
will hold confidential information, it is wise to make that fact known to group members here.
Privacy . A group can either be Public, which means that anyone can join a group and access the resources available to the group (notebook, mailbox, calendar, document library, plan,
and so on) or Private (the default), in which case only nominated members can access the group resources. Office 365 does not mark private groups in any special way within the
directory, so they are listed alongside public groups. For this reason, it is a good idea to consider giving a confidential group a name that does not create interest (or gossip) on the part
of users. For instance, a group called “Planning 2017 Layoffs” is likely to attract unwanted attention! If users want to join a private group, they can apply to do so. A request then goes
via email to the group owners, who then decide whether to add the user to the group membership. After creating a private group that needs to remain confidential, you can hide its
existence from address lists to prevent the group appearing in the GAL. Remember that you can change a group access type later.
Language . The default language is the one for the user who creates the group. You can change the language at any time. Office 365 uses the selected language for messages sent to
group members, such as in the footer of copies of conversations. It has no impact on the content of group discussions or anything else belonging to the group.
Owner . The name of the user who will serve as the first owner of the new group. You must add at least one group owner who then becomes responsible for managing the group
membership. You can add some group owners and members when you create the group or build out the group membership afterwards by either adding new members or, for dynamic
groups, by changing the query used to calculate group membership when run against Active Directory. You cannot nest an Office 365 Group inside another group, nor can you add a
traditional distribution group to an Office 365 Group or vice versa.
Subscribe members . The default is “On” for groups created using the Office 365 Admin Center or an Outlook client. This means that the group automatically adds new to the
subscribers list. Subscribers receive copies of conversations and meetings via email and do not need to access the group mailbox, unless they want to. Members can contribute to
conversations by replying to the messages they receive. Individual users can disable or enable this setting as they like. Groups do not generate daily or weekly digests of conversations.
Groups created from Teams do not auto-subscribe members because their conversations are in Teams.

Figure 11-5: Creating a new Office 365 Group

Click Add when you are ready to create the new group. After a short pause, the new group is ready for use. Note that you cannot set a content classification for a new group when you create
it through the Office 365 Admin Center, OWA, or Outlook, but you can when you create a new group from SharePoint as part of creating a new team site. You can add a classification after
creation by editing the group properties or by running the Set-UnifiedGroup PowerShell cmdlet.

Updating Group Membership

The next step is to add new members (or owners) to the group. Select the group from the list of groups in the tenant and then click the Edit button in the Members section. You can then add
both internal and external (guest) members to the group by selecting them from the list (Figure 11-6 ). Click Save when you are ready to update the group.

The problem with using this approach to add members to a group is that a GUI only works well when tenants have small numbers of users. Scrolling up and down within a large list to find the
right person to add to a group can lead to mistakes, as can searching the GAL to find the right person when they have a common surname. Many tenants will, for instance, have multiple
users named “John Smith” or “Tom Jones” (this is the reason why large tenants often use organizational prefixes like department names to help people find the right user).

Figure 11-6: Adding members to the new group

OWA automates the process somewhat by allowing group owners to import members from distribution groups or other Office 365 groups. When this happens, Groups expands the
membership of the input group and adds eligible accounts to the group membership, so it is an effective way to add many members quickly. As we will discuss in Chapter 12, it is possible to
add members to groups programmatically, including the conversion of some types of traditional email distribution groups to become Office 365 Groups. Updating group membership through
PowerShell is not to everyone’s taste, but it can be the most efficient way to approach the task.

Limits : All software has limits and Office 365 Groups are no different. Apart from the limits imposed by the underlying applications such as SharePoint Online storage quotas or the number
of items that a folder can store in an Exchange Online mailbox (well over 100,000, so this should not be an issue in terms of the ability to store conversations), Microsoft has discussed some
specific limits for Office 365 Groups. The most important are:

Originally, an Office 365 Group could not have more than 10 owners. Microsoft increased this limit to 100 in July 2017. The client user interfaces (including PowerShell) will prevent an
attempt to add more. If you hit the limit, you must demote an existing owner before you can add a new one.
An individual user cannot create more than 250 groups. This limit exists to speed up retrieval of the groups owned by an individual from Azure Active Directory. However, the limit could
become a factor in large tenants where central control governs the creation of new groups. The simple workaround is to make sure that you reassign ownership for newly-created
groups to a different user and remove the account that created the group from its owner list. The limit does not apply to global tenant administrators.
Office 365 Groups with more than 1,000 users are supported with the caveat that access to the group mailbox (for conversations and calendar) will be slow.

A tenant can support up to 500,000 Office 365 Groups. These limits apply to all Office 365 Groups, including those used with Teams, Yammer, Planner, Dynamics 365, and Power BI. The
number can be increased by Microsoft if necessary.

Welcome Notifications
When you add someone as a member of an Office 365 group using the Outlook apps or the Office 365 admin tools (including PowerShell), the system generates a welcome notification for the
user and sends it to them via email. The notification tells the person that they are now a member of the group, whether the group is public or private, and how many others are members. A
link in the message (View group in Outlook) opens the group conversations using OWA. Office 365 generates the notification in the language of the user (based on mailbox settings), but the
footer information in copies of conversations received by group subscribers are in the language assigned to the group.

The exception to this rule is when a Teams client creates a group to manage the identity and membership of a team. The conversation activity for these groups takes place within Teams, so as
users join these teams, it is Teams rather than Groups that generates the welcome message. The content of the two types of messages is similar, with the big difference being that the link in
the Teams welcome message loads the target team in a browser.

You can suppress the welcome notification for a group by setting its WelcomeMessageEnabled property to $False . This can only be done with PowerShell:

[PS] C:\> Set-UnifiedGroup -Identity MyGroup -UnifiedGroupWelcomeMessageEnabled:$False

Creating Office 365 Groups from the Azure Portal

Although you can create a new Office 365 Group from the Azure Portal , this is not recommended. To create a new group, select Active Directory, then User and Groups and then create a new
group. Set the membership type to Assigned and click the button to enable Office features. After you add members, Azure Active Directory signals Exchange Online to create the new group
mailbox and SharePoint Online to create the new team site. The process of fully-provisioning group resources takes longer than it does when creating groups elsewhere too.

Eventually synchronization completes, and the new group is available. However, because the Azure Portal does not run in the same context as other more tightly integrated components of
Office 365, like the OWA and Outlook clients or the Office 365 administration tools, it does not allow administrators to populate many of the other properties that you might want to set when
creating a new group (like the group classification or whether members receive updates via email). In addition, the Azure Portal does not set a user-friendly primary SMTP address on new
groups, and you will probably end up assigning a new address to the group.

Updating Group Properties

Once the group exists, you can update its properties. The group maintenance operations that you can perform through the Office 365 Admin Center and EAC are:

Edit group details to change properties such as the group name or description. You can also update the group so that it can receive emailed contributions to conversations from people
outside the tenant.
Edit the membership and ownership of a group.
Delete a group.

It is not possible to update several important group properties through the Office 365 Admin Center or EAC. For instance, you cannot change the group email address, add a MailTip, or hide
a group or its membership from the address lists. You can manage these properties through PowerShell, which is one reason why some administrators script the creation of groups to ensure
that their properties are per whatever standards exist for the tenant. For example, this command updates several properties for a new group, including a new primary SMTP address based
on a vanity domain, a new alias, modification of the access type for the group, setting a classification for group content, and allowing external users to contribute to group conversations via
email.

[PS} C:\> Set-UnifiedGroup -Identity ProductMarketingGurus -PrimarySmtpAddress ProductMarketigGurs@Office365ITPros.com -Alias ProductGurus -AccessType Private -Classification Confidential -
RequireSenderAuthenticationEnabled $False

Non-Latin Group Names

Groups creates the primary email address for new groups based on the group identifier (alias), which is in turn based on the group name. When clients run in languages other than English,
group owners can input non-Latin characters in the group name. Behind the scenes, each client processes the name to generate the identifier and primary email address. Some clients use the
internationalized domain name (IDN) standard to deal with non-Latin characters. Although this is generally a good thing and results in valid identifiers and email addresses, it can result in
the creation of email addresses that are not user-friendly. For example, if you create a new group with Teams and use the Czech term Žlutí koníci (yellow horses) as the name, it results in an
address like xn--lutkonci-e2ad10n@tenant.onmicrosoft.com. To avoid the problem, you can make sure to specify a group identifier (alias) that only includes basic Latin characters.
Alternatively, you can assign a new primary SMTP address to the group post-creation.

Email Address Policies and Groups

By default, a newly created Office 365 Group receives a primary SMTP address belonging to the default domain configured for the tenant plus a secondary address for the Office 365 domain
(the one ending in onmicrosoft.com). You can assign extra SMTP addresses to groups after creation, but you might want to assign SMTP addresses from separate domains depending on who
creates the group. We can do this by deploying email address policies within a tenant to detect whom is creating a group and selecting an appropriate address based on that determination.
Chapter 3 in the companion volume covers the use of address policies within Office 365 and this Microsoft support article explains how to use email address policies in a scenario where some
groups are created by one set of users and other groups by another set.

Assigning Send As and Send On Behalf Of Permissions to Groups

How to assign the Send As and the Send On Behalf Of permissions to mailboxes is explained in Chapter 7. You can also grant these permissions to Office 365 Groups, or rather to the group
mailboxes, in order to let group members (and non-group members) send messages as if they were the group (Send As ) or on behalf of the group (Send On Behalf Of ).

You can assign permissions by editing the group properties in EAC or through PowerShell. With EAC, edit the group and go to the group delegation settings. Enter the names of the users to
whom you wish to assign the Send As and Send On Behalf Of permissions and then Save . Two cmdlets manage the permissions through PowerShell. The Add-RecipientPermission cmdlet
assigns the SendAs permission and the Set-UnifiedGroup cmdlet assigns the Send On Behalf Of permission. Here is an example of how to set the two permissions:

[PS] C:\> Add-RecipientPermission Office365Gurus@Office365ITPros.com -AccessRights SendAs

-Trustee Kim.Akers@Office365ITPros.com

[PS] C:\> Set-UnifiedGroup -Identity "Office 365 Gurus" -GrantSendOnBehalfTo John.Smith@Office365ITPros.com

After users have been assigned permissions, they can use OWA or Outlook to send messages as if they were the group.

Using Office 365 Groups with OWA

OWA was the first client to support Office 365 Groups, so it seems proper to start a discussion about client access here. If you expand the Groups section in OWA’s left-hand navigation pane
a list of your most commonly used Groups appears. You can click More to expand the full list. In this case, OWA lists the groups that you have added to your Favorites list, followed by all
groups of which you are a member (Joined Groups). Obviously, a user interface is always limited by available screen real estate, so it is wise to mark any group that you know you want to
access on a regular basis as a favorite as this makes it much easier to find, no matter what client is used. You can use the right-click menu to view information about a group or to add or
remove it from your favorites list. An option to copy the email address of the group is also available in the right-click menu. Sometimes it is useful to have the name and email address of a
group to hand to be able to share that information with another person, as in the case when you would like them to join a group.

Click on the name of a group to access its contents. The first thing you see are the conversations in the group mailbox (Figure 11-7 ). A list of conversations in chronological order (last in,
first out) appears to the left and the items that make up the selected conversation appear in the viewing pane to the right. You can set the read status for individual conversations to be read
or unread, just like you can for messages in your mailbox.
Figure 11-7: The OWA interface for group conversations

Users can reply to the selected conversation at the bottom of the list of contributions, which invokes a modified version of the OWA editor. You can use the @ sign to address individual people
(including those who are not members of the group). If you add someone outside the list to a conversation, they receive a copy of all earlier items in the conversation and Groups adds their
email address to the conversation to ensure that they receive later contributions. You can use the special @All mention to send a conversation to all group members, including those who have
not subscribed to the group. This method uses the members list to send the message instead of the list of members subscribed to the group, so it is useful when you want to be sure that all
group members receive a communication. When the reply is complete, click send. OWA submits the message for processing through the messaging system to allow Exchange Online to apply
transport rules before delivering the item to the group mailbox. Once delivered, the item appears in the conversation. The Sent Items folder of user mailboxes holds copies of their
contributions to conversations.

Other UI options include being able to “Like” an item in a conversation and, in the drop-down menu revealed if you click the icon at the top right-hand corner, options to remove a
conversation (all the items rather than a single item) and to copy a link to the conversation to your clipboard. The link to the conversation only makes sense to a computer, so you should paste
the link into a message and send it to whoever you want to see the conversation. Of course, if the group is private, the link only works for members of the group. Non-members cannot gain
access to a private conversation with a link and the error message tells them that they need to join the group to see the content, all of which means that they will probably ask why you sent
them the link.

https://outlook.office.com/owa/?
exsvurl=1&ispopout=0&ConvID=AAQkADk4ZGQzZTFjLTgyNWMtNDE4Ny04ODg5LWRmZjc2Njg0MDdkOQAQACkh8cEvPEwLrD6rFeIwPsU%3D&path=/group/O365ExchPro@office365itpros.onmi

In the menu bar we see choices to switch between:

Conversations : The threaded conversations held in the Inbox folder of the group mailbox.
Files : The group document library in SharePoint Online. This view shows all the files and email attachments available to the group. To access the document library, click the Browse
Library button in the Files view.
Calendar : The group calendar. By default, the group calendar displays alongside the user’s personal calendar using the normal OWA calendar interface.
Notebook : The group shared notebook. Other OneNote notebooks can be stored in the document library.

The ellipsis menu reveals:

Planner : Access the default plan for the group – if one does not exist, it will be created.
Site : Access the home page of the SharePoint team site for the group.

The cogwheel (options menu) reveals:

Manage group email : Decide what messages you receive from the group.
Edit Group (for group owners): Edit group settings, like whether it is public or private.
Connectors : Connect the group to various network data sources. We explore how to use Connectors with Office 365 Groups in Chapter 12.

The ellipsis menu has choices to Leave the group, Invite someone else to the group, and add or remove the group from favorites. The menu bar also shows the membership status of the user.
In Figure 11-7 , we see that the status is “Not Following”, which means that the user is not in the group’s subscriber list, so they must open the group to see conversations. Otherwise you see
“Following,” meaning that the user receives copies of messages posted to the group via email.

Deleting conversation items : Sometimes people post contents to groups that should not appear. Perhaps a conversation includes some inaccurate information or maybe it is just a matter
of formatting, spelling, or the choice of words that makes someone want to retract a conversation or a reply to a conversation. OWA allows a user to remove a complete conversation if they
start that conversation. Group owners can remove conversations started by any user. OWA also allows users and group owners to remove individual items from a conversation and user who
starts a conversation can remove an individual reply from that conversation, even if they are not its author. Removing items during a conversation is not something that should happen
without thought as someone else might be replying only to discover that their reply does not make sense when posted. Users and group owners can also remove conversation items with
Outlook, but Outlook does not support the deletion of individual items within a conversation.

Deletion affects all replies, including those from other users. Unlike personal mailboxes, when a user removes a reply or conversation from a group mailbox (or a Teams compliance record),
the deleted items do not move into the Recoverable Items folder. Instead, Exchange Online moves the deleted items into the Deletions sub-folder of Recoverable Items in the group mailbox.
The deleted items stay in Deletions for 14 days after which the Managed Folder Assistant permanently removes them from the database. The exception is when the group is under the control
of a hold, such as those placed by an eDiscovery case managed through the Security and Compliance Center (see Chapter 20) or must be kept because an Office 365 retention label (see
Chapter 19) exists for an item. In this case, the deleted items stay in Deletions for 14 days as normal and then move to the Purges sub-folder, where they stay until an administrator releases
the hold or the hold expires.

Discovering Groups
Naturally, you might want to find some more groups to join. To see what is available, click Discover and OWA lists a set of cards displaying information about groups that you might want to
join (Figure 11-8 ). Queries against the Office Graph calculates the set of groups to display, based on the people with whom the user interacts inside the tenant and other signals. Frequent
interaction with someone is a strong sign that you might like to be part of a group that the person already belongs to. You can click the X in the top right-hand corner of a card (exposed when
you hover the cursor over the card) to hide it from view and show that you are not interested in the group.

The group name and its description are critical to convey the intent and purpose of the group to a potential member. For example, “HR Working Group” is a reasonable title for a group as it is
easy to understand that this group is probably associated with the Human Resources department and is to do with some item of work that the department needs to do. However, in this case,
the description is clearly inadequate and gives absolutely zero help to the reader when it comes to understanding what the purpose of the group might be.

The search box (under “All Groups”) allows you to look for a group based on its name. One immediate shortcoming is obvious in that no facility is available for users to narrow their search.
For example, you cannot look only for public groups that have a certain word in their title or use other search criteria to find the right group to join.

When you search for groups, OWA divides the groups available in the tenant into “active” and “inactive” lists. An active group is one where some sort of conversation activity has happened in
the last 30 days. In other words, a group member has started a new thread or replied to an existing conversation (this includes the message automatically generated when a group is
created). The “active” designation takes no account of any activity in the group document library nor if something happened in a plan or team associated with the group. It is simply an
indicator to allow a user to avoid joining a group that is obviously inactive.
Figure 11-8: Viewing Office 365 Groups that a user might want to join

Inviting Others to Join a Group

OWA allows group members to invite other tenant users to join a group. From the Conversations or Files tabs, click the cogwheel menu and select Invite others. You then have the choice to
send an email with the invitation to the person you want to join the group or to copy a link that can be pasted into a message and sent to the potential member. The link is of the form:

https://outlook.office.com/owa/MyGroup@office365itpros.com/groupsubscription.ashx?action=join&guid=2d5f933d-1287-3777-2e87-d7c199bb85e0

In either case, when the invitee receives and clicks the link, they can join the group at once if it is public. If the group is private, the group owners receive a request to approve them as a
member. Group members cannot invite guest users in this manner because a group owner must add guest members.

Searching Group Content

Exchange Online and SharePoint Online automatically index the information stored in Office 365 Groups to make sure that the content is discoverable and usable for the purposes of
compliance and eDiscovery. Users can select the “Your Groups” search choice to look for information held in any group to which they belong. OWA searches only include items and
attachments stored in group mailboxes. If you want to search across the entire group, you must conduct separate searches for the files in the group document library and the conversations in
the group mailbox. Content searches can include group data held in SharePoint Online and Exchange Online in the same operation. However, content searches do not include plan metadata
that might be associated with a group.

Creating and Managing Office 365 Groups Through OWA

The process to create a new Outlook group with OWA essentially follows the same steps as when you create a group using the Office 365 Admin Center or EAC. Two stages occur. First, you
give the essential information about the group (left-hand screen of Figure 11-9 ).

The display name of the new group. This is the name that will be visible in the GAL. If your tenant uses a Group naming policy, Office 365 applies the policy to the name you give to
create the display name.
The “Group ID” or alias. The alias is the basis of the email address for the group.
The description.
The access type of the group: whether it is public or private.
The language used for copies of system notifications sent to group subscribers.
Whether to send copies of group conversations to members. If you decide to do this, new members automatically join the group’s subscriber list. Each member can decide later if they
want to continue receiving updates from the group.

After setting the group properties and clicking Create to set up the new group, you have the chance to populate the group with the first set of members (right hand screen in Figure 11-9 ).
You can add individual users, or you can select an email distribution group or an Office 365 group to add multiple members at one time. If you select an email distribution group, only the
members who have Exchange Online mailboxes can join the new group. Using an existing group is a very useful method to build out the membership of a new group. However, extracting
members from a distribution group is a one-time operation and no link exists afterwards between the source distribution group and the new Office 365 group. Office 365 does not replicate
any change made afterwards to the membership of the source group to the new group. If you decide not to add any members to the group at this point, the only member will be the user who
creates the group. Later, you can choose to nominate some of the members as group owners to allow them to administer the group (Figure 11-10 ).

Figure 11-9: Creating a new Office 365 group with OWA

Figure 11-10: Managing group membership with OWA

Editing Group Properties

The Edit Group choice available through the cogwheel menu allows group owners to update group properties such as whether external people can communicate with the group via email,
change its access type from Public to Private or vice versa, update the group’s classification or description, or decide whether members should receive copies of group communications in
their inbox. For example, to enable external communication, set the "Let people outside the organization email the group" checkbox. This updates the RequireSenderAuthenticationEnabled
property of the group to be $False and means that anyone outside the tenant can send email to the group (if they know its email address). Messages received from external users will either
begin a new conversation with the group or are added to an existing conversation, if the message is a reply to a message received from the group.

Figure 11-11: Updating group properties with OWA

You can add a graphic icon or thumbnail to help users understand its purpose. If you do not upload an image file for the group, Office 365 constructs a default logo using the initials of the
first two words of the group’s name. If the group name is just one word, Office 365 uses the first two letters of that word. The result is that you end up with boring default icons like many of
the ones visible in the Groups Directory (Figure 11-8 ). To make things more interesting, you can upload a JPEG, PNG, or BMP format file to use as the group icon. Although OWA will scale
the graphic file when it saves the icon in the thumbnailPhoto attribute for the group object in Azure Active Directory, it is best to create and upload a file sized around 300 x 300 pixels to
prevent the need for excessive scaling up or down of the image, or if you want to crop the picture to use a certain part of the image as the icon.

Group usage guidelines (click the circled I icon) and a Classification drop-down list appear in Figure 11-11 . These properties only appear if you configure the AAD directory settings object
for Office 365 Groups with the necessary settings as explained in Chapter 12. Apart from updating the classification for a group through a client, you can also change it with PowerShell. For
example:

[PS] C:\> Set-UnifiedGroup -Identity SharePointFans -Classification “Internal Only”

The value passed for the classification must be in the set specified in the AAD policy for Groups.

Controlling Group Email Traffic

When they create a new group, the group owner or administrator decides whether members will receive emailed copies of group conversations and events. The default is to distribute copies
to all members, including tenant users and guests. Members can control the communications they wish to receive through Manage Group Email in the cogwheel menu, selecting whether to
receive all messages, replies to their posts, calendar events, or no messages.

The group’s AutoSubscribeNewMembers property stores the setting and is True to subscribe members or False if not. You can change the AutoSubscribeNewMembers settings by editing the
group properties with OWA, Outlook, by updating the Subscribe members setting for the group in the Office 365 Admin Center, or by running the Set-UnifiedGroup cmdlet. For example:

[PS] C:\> Set-UnifiedGroup -Identity Office365ITPros -AutoSubscribeNewMembers:$False

As you might expect, Office 365 Groups created by Teams set the AutoSubscribeNewMembers property to $False because members of these groups communicate using team chats. To see a
list of groups that do not distribute copies of conversations and events to members, use the command:

[PS] C:\> Get-UnifiedGroup | ? {$_.AutoSubscribeNewMembers -eq $False } | select Displayname

To receive copies of group conversations and events, members join the group’s subscribers list. You can run the Get-UnifiedGroupLinks cmdlet to report the subscriber list for a group:

[PS] C:\> Get-UnifiedGroupLinks -Identity Office365ITPros -LinkType Subscribers

Oddly, there is no way to view the subscribers of a group through any interface except PowerShell. The membership view presented by OWA splits members into “All,” “Owners,” and
“Guests” while the group toolbar displayed in Outlook desktop shows Following if a user is part of the group subscriber list or Not Following if they are not. The user can toggle the switch
to join or leave the subscriber list as they like.

When a group is “chatty” and many conversations take place, some users might decide that they do not wish to receive updates via email and are quite happy to access the conversations in
the group mailbox instead. Users belonging to the tenant can control the updates they receive for a group through email through settings available in OWA, Outlook for Windows, Outlook for
Mac, and Outlook for iOS. Guests can ask a group owner to remove them from the group’s subscriber list if they do not want to receive updates. Even after they are not in the subscribers list,
group members continue to receive any conversations or replies posted to the group where replies include the special @ mention in the text (for instance, @Tony) or where someone uses the
special @All mention to include all members in a conversation. The same is true when you explicitly add someone to the TO: or CC: list of a message sent to a group. Remember that the
settings to control emailed updates are specific to a group. If you want to stop updates arriving in your Inbox for every group you belong to, you must remove your account from the
subscribers list for each group.

In OWA, the group card allows users quick access to different group resources (like conversations and files) and settings including the ability to control subscriptions. To access a group’s
card, find a message sent to the group and click on the group name in the message header. You can then move the Follow in inbox slider in the group card from Off to On (Figure 11-12 ) or
vice versa to control whether you receive updates via email. Behind the scenes, moving the slider to Off removes your account from the subscribers list for the group while moving it to On
adds your account to the subscribers list. Outlook for Mac and Outlook for iOS also support group cards with the Follow in Inbox setting for a group.
Figure 11-12: The setting in a group card to control emailed updates for a group

Using Office 365 Groups with Outlook Desktop

Because it is much easier for Microsoft to update a web-based user interface to accommodate the needs of new functionality, OWA was the first client to support Office 365 Groups. For the
same reason, it continues to be the case that new Groups functionality appears in OWA first. However, in terms of numbers of clients used, Outlook is the single most important client for
Exchange Online, which is one of the reasons why support for Office 365 Groups was so important for the overall adoption of Office 365 Groups within tenants. Outlook 2016 is the first
version of the desktop client to incorporate the necessary user interface elements to be able to work with Office 365 Groups. With that point in mind, we will look at how Outlook works with
Groups.

Accessing Groups with Outlook

A user must satisfy two requirements before they can use Outlook to access Office 365 Groups. First, you must use the right software. Because it has all the latest fixes and enhancements,
the best software is the click-to-run version of Outlook 2016 included in Office 365 ProPlus. Alternatively, make sure that you have the latest available build of Outlook 2016 Professional Plus.
Second, the Autodiscover service for the tenant must deliver information to clients about what groups are available in the tenant. This is a straightforward matter (you should not have to do
anything) for a cloud-only tenant but can be more complicated for a hybrid environment. We will discuss how Outlook consumes the information given by Autodiscover in a little while. OWA
does not have to concern itself with Autodiscover because that client can only function when connected to the server while Outlook often functions in what can be described as a “partially
connected” mode that is designed to accommodate low-fidelity network links.

Groups appear in two places in Outlook. First, Outlook includes any groups marked by a user as a favorite (with either OWA or Outlook 2016) alongside other favorites such as folders in the
user’s own mailbox, shared mailboxes, and public folders. The mailbox holds details of favorite groups to make them available across all clients. In other words, if you add a group as a
favorite in Outlook, you will see the same group listed as a favorite in OWA. You should add groups that you want to access frequently to Outlook favorites. However, there is limited display
space available on any PC screen and this in turn imposes a limit on how many Groups Outlook can list in its set of favorites alongside the other types of favorites (mailbox and public folders).

Second, if the user clicks the Groups root in Outlook’s resource navigation pane, Outlook expands the set of Groups to which the user belongs. If the user leaves a group, Outlook updates the
list and removes the group.

One thing to consider here is that the list is flat because groups do not use a hierarchy. Every group is a peer of the other groups. As you can see in Figure 11-13 , a scroll bar is activated for
the window when the number of groups reaches a certain point (dependent on the size of the screen), but it can be easy to “lose” sight of an important group if it is at the end of the list.

There is both goodness and badness in this approach. If you consider the example of public folders, many of those responsible for managing very large deployments (more than 100,000
folders) would acknowledge that a large percentage of the total folders form the hierarchy are empty. The deeper the hierarchy, the more folders are needed to flesh it out. Office 365 Groups
do not incur this overhead, but the alphabetical listing of all groups available within a tenant can make it difficult to find the precise group that someone might want to access or join.

Figure 11-13: Group conversations displayed by Outlook 2016 (Windows)

The interface used to display the contents of group conversations should seem very natural to Outlook users because it follows the same kind of approach used to display the content of
mailbox folders. The list of conversations appears in the middle pane and the viewing pane (to the right) displays the messages in the selected conversation, with the first contribution shown
on the top and later messages listed afterwards. This is a difference with the approach taken to display items in other email conversations as Outlook typically displays the latest message at
the top of a folder.

Like all the other Office applications, Outlook makes use of shared components. The code behind the special threaded conversation view comes from Word and can display anything that you
can put into a Word document. The view also includes the option to “like” a contribution.

As with OWA, the idea is that conversations are short and snappy, so a full-featured editor is not necessary. However, because Word is the editor, all its features are available, albeit in a small
space – but you can always use a full-size editor and then paste the text into Outlook. And if you need to work on the information in more detail, you can click on the small fly-out icon shown
on the right-hand side. This causes Outlook to display the normal message create window where you can use the full range of text formatting controls to create the text of your contribution.

When you contribute to a conversation, the content goes via email to the group mailbox (you will find your messages in the Sent Items folder of your mailbox). As shown in Figure 11-14 , a
user is typing the response to a conversation into the in-line editor. However, if you click the “fly-out” icon to the right of the message’s title bar, Outlook displays a full message form to allow
more room to compose the text for the new message.
Figure 11-14: Contributions to group conversations from Outlook go via email

Given the way that people use Outlook, it is almost inevitable that many of the interactions with Office 365 Groups is via email. Users will receive messages sent to conversations via email
and respond to them via email. And like email conversations in user inboxes, a lot of autosignature overhead might accumulate and obfuscate the valuable content contained in user
contributions. This is a difficult problem to solve. It is possible to build a transport rule to apply an autosignature to outgoing messages that will not include the autosignature if it exists in a
message (for example, as in the case of a reply). Third-party ISV products are also available that support central management of autosignatures. However, none of these solutions can block
users inserting their own autosignatures in messages across the spectrum of clients that can connect to Office 365, so it is probably an overhead that we must accept.

Groups Toolbar
When you select a group in Outlook, the client switches its normal ribbon to a special groups toolbar. Options are available are to start a new conversation, switch to the group calendar, or
open the shared group document library or notebook. Selecting Files or Notebook opens web pages to display the documents in the library or the contents of the shared group notebook.
Outlook calls the desktop version of OneNote if available on the PC. Otherwise, Outlook invokes the browser version to access the group notebook.

As you can see in Figure 11-13 , the toolbar shows that the selected group is public. The icon created for the group appears along with the joined status of the user. The Joined button allows
the user to subscribe or unsubscribe to a group. A nice touch is that when a group is part of the recipient list for a message, Outlook displays a “Go to Group” option in the command bar
above the message text. Clicking the link moves focus to the conversation view within the group.

Group Calendars
When you add a group to the favorites list, the action also adds the group calendar to the set displayed under the “My Calendars” section in Outlook’s calendar. To access the calendar, click
on its name to include it in the set calendars displayed on-screen. You can then add events to the calendar.

Two ways exist to add an event to a group calendar. You can create an event in your calendar (in which case, you are the organizer) and add the group to the list of attendees. The event
invitation goes to the group calendar as well as all members of the group. However, if you create the event in the group calendar, the group is the organizer. Email notifications for the new
event will only go to group members who subscribe to the group. The difference in behavior is because a group member creates the item in the calendar instead of the group receiving a
meeting request via email. In this case, the mail transport service expands the group membership and subscribers receive copies of the event.

Groups in the GAL

Figure 11-15: Joining a group found in the Outlook Address Book

Like all mail-enabled objects, Office 365 Groups are in the Address Book, where they are in the Global Address List (GAL) and in the All Groups address list. Users can browse the GAL to find
new groups or to examine the properties of groups. If you find an interesting group, you can click the Actions button to reveal the group properties. If the group is public, they can use the
Join button to join the group. If private, the Request to Join button (Figure 11-15 ) emails a request to the group owners to add the person as a member.

Navigating through a large directory and scattering groups here and there between all the other objects that make up the GAL can create a search challenge for users. It is helpful when a
tenant deploys a naming policy for groups so that they are all gathered in a single place in the GAL. Once a user has found and joined a group, they can add it to their folder favorites to
create a short-cut to the group.

Even with a naming convention in place, the prospect of looking at hundreds of groups to find just the right one to join can be an intimating task. To make things easier, users can find groups
to join with Browse Groups (accessed in the Outlook ribbon or by right-clicking the Groups resource in the navigation pane). Like OWA’s Discover feature, Browse Groups displays a set of
suggested groups that the user might be interested in joining (Figure 11-16 ). Office 365 calculates what groups to suggest using data held in the Office Graph that tracks interactions
between users. Simply put, if three other people whom someone interacts with often use a group, a high chance exists that the person might also be interested in the group.
Figure 11-16: Browsing suggested groups from Outlook 2016

Users can see what is happening in a public group by clicking View . This opens the group and shows the user the most recent conversations. Clicking All reveals a full alphabetic listing of
available groups, including those that the user has already joined. It is the equivalent of viewing the All Groups list through the Address Book.

Creating a New Outlook Group with Outlook

If the Groups policy allows a user to create new groups, they can do so using Outlook desktop. Three methods exist:

Click on Groups in Outlook’s resource navigation tree and then the + (plus) sign.
Click the New Items icon in the Home section of Outlook’s ribbon and then select Groups from the menu.
Click New Group in the group ribbon.

In all cases, the dialog shown in Figure 11-17 appears to collect information about the new group. The classification list box reveals the set of classifications defined in the tenant policy for
groups while the privacy list allows you to select whether the group is private or public (its access type). You can change both settings afterwards. When all details are input, click OK to
create the new group.

After Office 365 creates the new group, you can add members to the group. Just like editing membership for a group, this is simply a matter of browsing through the directory and selecting
people or other groups to join the group. If you add a group, Outlook expands its membership and adds any cloud mailboxes found in the membership to the membership of the new group.
The account used to create an Outlook group joins the group automatically as an owner. Outlook does not allow owners to add guests to a group. This can be done with PowerShell, an
administrative interface, or through OWA.

Figure 11-17: Creating a new Office 365 group from Outlook for Windows

Updating Group Properties with Outlook

A user who is a group owner can update the properties of the group. As you can see from Figure 11-18 , you can change the name of the group, its description, its membership, classification,
access type, the language used in messages generated by the group, and whether people from outside the group can send email to the group to take part in group conversations. Other
properties, like the primary SMTP address, are either invisible or you cannot change through Outlook. One of the members shown in this group is a guest.

Figure 11-18: Editing the properties of a group from Outlook for Windows

You can also manage requests to join groups with Outlook. Figure 11-19 shows a notification message sent to a group owner when a user tries to join a private group. The “go to Group
members” link in the message calls a dialog to display the current group membership, which you can then manage. Naturally, you can only view or change group membership when working
online.
Figure 11-19: Outlook for Windows processes a user request to join a group

Offline Access for Office 365 Groups

One of the biggest advantages of Outlook is its ability to enable users to continue working when a network connection is unavailable (or flaky). Since its first release in 1997, Outlook has
used the Offline Storage File (OST) to cache mailbox content for offline access. The addition of support for Office 365 Groups introduces the Group Storage File (GST). The GST is located in
the same folder as the OST and has an .nst extension.

For performance reasons, Outlook synchronizes any group accessed by the user into the GST. Only conversations and the group calendar are synchronized as the other information shared by
groups (files, notebook, etc.) is only available online. Outlook must run in cached Exchange mode for synchronization to occur, which means that Office 365 Groups are not available when the
client runs in online mode. In fact, groups do not even appear in Outlook’s resource list when the client works in online mode. This is an issue for deployments where Outlook runs on thin
clients and local files are undesirable. A partial workaround exists by configuring Outlook to synchronize a minimal amount of data (1 week for Outlook 2013 or 3 days for Outlook 2016). The
“slider” that controls how much mailbox data is synchronized by Outlook also controls how much data Outlook must synchronize to make Groups available offline. However, given that much
of the information shared by groups is only available online, OWA might be the better client choice in these situations as one of Outlook’s strongest selling points (offline access) is not
required.

Multiple connections : If you click the Outlook icon in the system tray, you can use the CTRL-Click key sequence to reveal the Connection Status option, which allows you to view the
connections maintained by Outlook to various resources – your mailbox, shared mailboxes, public folders, and Office 365 Groups. You might be bemused to discover that a HTTP connection
exists for every Outlook group to which the user belongs. If you examine the connections, you will see that a connection exists to the mailbox for each group in your favorites list. If you
access the Groups link in Outlook’s resource list and expand the set of groups that you are a member of, Outlook will make connections to their mailboxes. The connections are used for
background synchronization of new content to the offline copy of the group held in the offline groups (GST) file.

Offline access to data in the group document library and shared notebook is not dependent on Outlook. You can synchronize the group document library with the OneDrive for Business sync
client to have copies of those files offline, but you cannot get to this cache directly from Outlook. Instead, you can access the synchronized files through Windows Explorer and make changes
there. The group notebook can also be accessed offline if you have used it with the OneNote desktop application. Any change made in the desktop copy of the notebook will be synchronized
back to the online version when the network is available again.

Like mailbox folders, any updates for group conversations created when using Outlook to work offline will be synchronized back to the server the next time a network connection is available.

How Autodiscover tells Outlook About Office 365 Groups

Typically, Outlook learns about new mailbox resources such as a new shared mailbox through the XML manifest returned by Autodiscover. Although they show some of the same
characteristics as regular shared mailboxes, group mailboxes are not accessed in the same way. After it returns information about the user’s own mailbox and any other mailboxes that a user
can access, Autodiscover issues a call to list all the Office 365 Groups that the user has joined. Details of each group are retrieved and stored in a set of XML files in the C:\Users\
<username>\AppData\Local\Microsoft\Outlook\16 folder. A separate XML file, prefixed with “AutoD”, is maintained for each group and is refreshed hourly.

Outlook also stores information about the user mailbox, shared mailboxes, and other resources in the same folder. An example of the information retrieved for a group is shown below. Like
the information returned for a normal user mailbox, this information tells Outlook how to connect to and open the group mailbox.

<?xml version="1.0" encoding="utf-8"?>

<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">

<Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">

<User>

<DisplayName>EHLO Blog Entries</DisplayName>

<LegacyDN>/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=d09163c8097a462bbc03a2456c64b60f-ehloblog</LegacyDN>

<AutoDiscoverSMTPAddress>ehloblog@Office365itpros.onmicrosoft.com</ AutoDiscoverSMTPAddress>

<DeploymentId>bddca944-897a-4555-a83f-73ab13c75fd9</DeploymentId>

</User>

<Account>

<AccountType>email</AccountType>

<Action>settings</Action>

<MicrosoftOnline>True</MicrosoftOnline>

<ConsumerMailbox>False</ConsumerMailbox>

<Protocol Type="mapiHttp" Version="1">

<MailStore>

<ExternalUrl>https://outlook.office365.com/mapi/emsmdb/?MailboxId=ca754c8a-f2f3-4ad8-927f-b22085d12b89@Office365itpros.onmicrosoft.com

</ExternalUrl>

</MailStore>

<AddressBook>

<ExternalUrl>https://outlook.office365.com/mapi/nspi/?MailboxId=ca754c8a-f2f3-4ad8-927f-b22085d12b89@Office365itpros.onmicrosoft.com

</ExternalUrl>

</AddressBook>

</Protocol>
Outlook for Mac
Any version of the Outlook for Mac 2016 client released after July 2017 supports Office 365 Groups. Earlier Outlook for Mac clients do not include the necessary code to support groups.

Groups and Transport

Every group has an SMTP email address. Members and others (if allowed by the group settings) can use that address to route email to groups just like any other recipient. The messages
eventually arrive at the Exchange Online mailbox server that hosts the active copy of the group mailbox. At this point, the Exchange transport service processes the messages before delivery
to the group mailbox. All messages are subject to whatever processing the tenant implements through transport rules.

In addition to sending email to groups, clients can post new conversations and replies to existing conversations. Depending on the client, the mechanism to post to a group is slightly
different. Outlook handles a message posted to a group conversation much like any other message. The client posts the item into the local copy of the group that synchronizes to the
workstation. The normal Outlook synchronization mechanism picks up the change in the local folder and dispatches the message to the server where it is processed by the transport system.
Eventually, Exchange delivers the message to the group mailbox where it shows up in a conversation. The item posted to the group stays in the user’s Sent Items folder.

OWA and mobile clients work online do not have the same synchronization mechanism and offline copies as Outlook. Because all posts must go through the transport service, the possibility
exists that a short delay might be obvious between the time when a user sends a post to the group and the post showing up in a conversation. The delay comes about through the need for the
transport system to process the message through its pipeline to allow rules and routing to occur. To give the user confidence that their post succeeded, OWA and mobile clients create a
“fake” post in the conversation that is only visible to the submitter. Behind the scenes, email processing continues and if a transport rule decides for some reason to reject the post, it does not
deliver it to the group mailbox. The submitter will receive a non-delivery notification to tell them that an administrative rule blocked their contribution. At this point, OWA removes the fake
post so that it no longer appears in the conversation. The transport service usually processes messages very quickly and the time elapsed between submission and removal is very short, but if
delays occur, users might report a “disappearing item” if they ignore the non-delivery notification. It is all by design.

Mobile Groups
The Outlook for iOS and Android apps support access to Groups along with regular mailbox folders. The logic behind including Groups in Outlook is to make group conversations more
accessible to users. In effect, Outlook treats the list of groups that a user belongs to like a big folder. Each group shows up as a sub-folder. When you open a group, Outlook displays the
conversations like it would any other message in a folder. You can then reply to a conversation or create a new conversation just like you would reply to a message or create a new message.
The integration also supports “Group Details” to allow users to see the membership of a group, to mark a group with “Follow in Inbox,” which means that the user joins the subscribers list to
receive future updates via email or leave the group. Users can add events for the group calendar to their personal calendar. The mobile apps both do a better job of suppressing unwanted
material (like email autosignatures and the text of earlier replies) when they display conversations.

Because the Outlook apps cannot connect to Yammer, you cannot access conversations in Yammer Groups through the Outlook apps.

Outlook exploits the Microsoft Graph API to access groups. The API allows developers to create code to find and open groups, navigate through conversations, create new conversations, or
take part in existing threads, open the shared notebook, and interact with the files stored in the group document library. The Groups app comes under the control of the Azure Active
Directory policy for Groups, so only users who are on the permitted list can create a new group. You can also update the name of the group, its logo, description, and its membership through
the app, but you cannot perform other administrative operations such as switching a group from private to public or amending the group alias or email address. The mobile app also supports
the removal of a group.

Like all mobile apps, the kind of device you use limits the amount of work you can do with Groups. The screen size is the most crucial factor in terms of viewing data or overall usability.
Although you will probably pause before plunging in to edit a large and complex document, casual browsing and taking part in group conversations is more than possible, even on a relatively
small-screen device like the Apple iPhone 6S. The original Groups mobile app is now retired.

Removing and Recovering Office 365 Groups

The nature of all IT systems is that some groups become unwanted soon after creation or that the use of a group declines over time to a point where it falls into disuse. You might then decide
to remove the group from the tenant. As noted previously, you can remove a group from the Office 365 Admin Center, EAC, OWA, Outlook, a mobile app, or from a group-enabled application
like Teams. When you remove a group, Office 365 removes all the connected resources belonging to the group from SharePoint, Planner, Teams, Yammer, and so on. In addition, group owners
or tenant administrators can remove a group by running the Remove-UnifiedGroup or Remove-Team cmdlets. In this example, we remove a group and suppress the prompt to continue that
the cmdlet usually needs before it proceeds:

[PS] C:\> Remove-UnifiedGroup -Identity SkypeToTeamsMigration -Confirm:$False

Auditing Group Deletions : When you (soft) delete a group. Azure Active Directory notes the details in a “Delete Group” audit record. Later, when Office 365 permanently removes the
group, Azure Active Directory captures a “Hard Delete Group” audit record. Permanent deletion happens when the Microsoft Online Service Garbage Collector process runs, so the exact
period when a group is in a soft-deleted state will vary from 30 days to perhaps a few days afterwards.

In addition to explicit removals by owners or administrators, if you operate an expiration policy, Office 365 removes groups automatically when they expire after a set period. In all cases, a
removed group and any associated resources first goes into a soft-deleted state and remains there for 30 days. During this time, an administrator can restore the group to make it accessible
again to users. Alternatively, they can force the permanent removal of the group, which means that the resources of the group become irrecoverable. Table 11-4 lists the data recovered for a
soft-deleted group.

Group Resource Data restored

Azure AD Object Group object, properties, and membership.

Exchange Online mailbox Group mailbox holding conversations and shared calendar.

SharePoint Online Team Group document library, including the shared OneNote notebook and the folders used by channels within Teams.
site

Planner Group shared plan, including the plans created for channels within Teams.

Teams Team chats and metadata such as team and channel properties.

Unified Group object SMTP email address. Note – if the SMTP address has been assigned to another group, you cannot restore the original group until you update the SMTP address
for that group.

Yammer Conversations in Yammer data store (for Yammer groups that use the Office 365 Groups service).

Table 11-4: Data restored when recovering a soft-deleted group

You can recover a soft-deleted group through the EAC or with PowerShell. To recover a group with EAC, open the Recipients section of the console and then Groups . EAC lists the groups
known in the tenant, including distribution groups and dynamic distribution groups. To include the groups that can be recovered in its list of groups, EAC uses this command:

[PS] C:\> Get-UnifiedGroup -IncludeSoftDeletedGroups:$True

To display the properties of a soft-deleted group, EAC runs Get-UnifiedGroup again and extracts the WhenSoftDeleted property to calculate how long ago the deletion occurred.

To find groups awaiting recovery, click the Status field in the header and then look for groups that have a deleted status as shown in Figure 11-22 . The details pane tells you when the
deletion occurred. In this case, it happened seven days ago.
Figure 11-22: Recovering a soft-deleted group with EAC

To restore the group, click the restore link in the details pane. EAC prompts you to confirm the recovery and if you agree, proceeds to recover the group. After a short period, the newly
reactivated group is available to users.

Recovering a Deleted Group with PowerShell

When it recovers a group, EAC runs the Undo-SoftDeletedUnifiedGroup cmdlet and passes the object identifier for the selected group:

[PS] C:\> Undo-SoftDeletedUnifiedGroup -SoftDeletedObject 'c4afcaa8-e772-46be-b668-7e6b9c264ec1'

You can mimic what EAC does to recover a soft-deleted group as follows. First, retrieve the object identifier for the group you want to recover:

[PS] C:\> $Object = (Get-UnifiedGroup -Identity "Group to Recover"

-IncludeSoftDeletedGroups).ExternalDirectoryObjectId

Now, use the identifier to recover the group.

[PS] C:\> Undo-SoftDeletedUnifiedGroup -SoftDeletedObject $Object

This technique works, but Microsoft’s recommended approach to recover soft-deleted groups with PowerShell uses cmdlets in Version 2.0.0.137 (or later) of the Azure Active Directory
module, which must be installed on the workstation used for the recovery. The reason why EAS uses different cmdlets is that these cmdlets are wrappers around the Azure Active Directory
cmdlets tailored for use with EAC.

Scanning for Soft-Deleted Groups

Any Office 365 Group deleted by a groups-enabled application like Teams or Planner or PowerShell goes into a soft-deleted status. Our first task is to run the Get-AzureADMSDeletedGroup
cmdlet to expose the set of soft-deleted groups. The object identifier for a deleted group is critical information because you need it during the recovery process. In this example, we see that
two soft-deleted groups exist, one private and one public. The data is sorted by the time and date when the deletion occurred with the oldest groups listed first.

[PS] C:\> Get-AzureADMSDeletedGroup | Sort DeletedDateTime | Format-Table Id, DisplayName, DeletedDateTime, Description -AutoSize

Id DisplayName DeletedDateTime Description

-- ----------- --------------- -----------

3b42ada9-0bb1-4553-a676-c5a4553ec615 Mobile devices 16-Jan-18 1:16:33 PM Team to discuss mobile

3fff1844-cbe8-4a1f-a375-e8e38d179268 TeleWorkers 1 2-Feb-18 10:29:57 AM Teleworking processes

You can use the SearchString parameter to find a specific group. This only works with characters at the start of the DisplayName attribute and does not search any other value.

[PS] Get-AzureADMSDeletedGroup -SearchString "Engineering Testers"

Another way of doing the same thing is to use a filter. In this case, we look for a string at the start of the DisplayName attribute.

[PS] C:\> Get-AzureADMSDeletedGroup -Filter "startswith(DisplayName,'CEO')"

Sometimes you might want to look for soft-deleted groups that Office 365 will remove in the immediate future. One reason why is because time is slipping away for recovery if the tenant
needs to keep one or more of these groups, perhaps because the group resources (like the document library) hold some important information. In this example, we search for soft-deleted
groups that Office 365 will remove in the next week and highlight them for further attention.

[PS] C:\> $CheckDate = (Get-Date).AddDays(-7)

$Today = (Get-Date)

$Grp = (Get-AzureADMSDeletedGroup -All:$True | Sort DeletedDateTime | Select Id, DisplayName, DeletedDateTime, Description)

ForEach ($G in $Grp) {

If ($G.DeletedDateTime -le $CheckDate) {

$TimeToGo = ($G.DeletedDateTime).AddDays(30) - $Today

$Line = $G.DisplayName + " is due for permanent removal on " + ($G.DeletedDateTime).AddDays(30) + " You have " + $TimeToGo.Days + " days and about " + $TimeToGo.Hours + " hours to recover the group."

Write-Host $Line -Foregroundcolor Red

Restoring a Group
The Azure Active Directory cmdlets use object identifiers (a GUID) to know what object they should process. To make it easier to recover the right group, we store its object identifier in a
variable. In this example, we put the object identifier for the “Teleworkers” group in a variable called $RecoveryGroup .

[PS] C:\> $RecoveryGroup = (Get-AzureADMSDeletedGroup -SearchString "Teleworkers").Id

The next step is to run the Restore-AzureADMSDeletedDirectoryObject cmdlet to instruct Office 365 to recover the selected group.

[PS] C:\> Restore-AzureADMSDeletedDirectoryObject -Id $RecoveryGroup

The technique works to recover groups that store conversations in either Exchange and Yammer.

Because many different applications use Office 365 Groups, the recovery of a soft-deleted group involves several complex operations. Recovery of a simple group (one that uses Exchange and
SharePoint resources only) can occur within a few minutes and should complete within an hour. Depending on the current load within the service, recovery of a group that is team- or plans-
enabled might take up to 24 hours before Office 365 recovers and reconnects all the data. The added delay is due to the need to synchronize multiple directories and often to ensure that data
for Teams and Planner stored in Azure data services are restored correctly. It is also the case that when you restore a dynamic Office 365 group, it can take up to 24 hours before Azure
Active Directory runs the query to calculate the group membership.

The first stage in recovery is to restore the Azure Active Directory group object. After the group object is in place, Office 365 can restore each of the attached workloads to gradually build up
a fully-functional group. To verify that the progress of the recovery, you can run the Get-AzureADGroup cmdlet to check that the group object exists in Azure Active Directory. Once the group
reappears in Azure Active Directory, the background synchronization processes which link Office 365 workloads will pick up details of the restored group and make them available to users.
You can track progress with cmdlets such as Get-UnifiedGroup , which shows that the group is now available to Exchange Online, and Get-TeamUser , which shows when it is known to
Teams. The last code snippet checks with SharePoint Online to see if the URL for the group team site is known to it.
[PS] C:\> Get-AzureADGroup -ObjectID $RecoveryGroup

[PS] C:\> Get-UnifiedGroup -Identity $RecoveryGroup

[PS] C:\> Get-TeamUser -GroupId $RecoveryGroup

[PS] C:\> $SPO = (Get-UnifiedGroup -Identity $RecoveryGroup).SharePointSiteUrl

[PS] C:\> $Sites = Get-SpoSite $SPO

[PS] C:\> ForEach ($Site in $Sites) { if ($Site.Url -eq $SPO) {write-host "Site found"}}

Restoring the directory objects is only the first stage in the recovery process and does not mean that users can use the membership represented in the directory to access group contents. To
perform further validation and before telling users that they can work with the group, you should check each of the workloads used by the group using clients to confirm that the expected
conversations, files, plans, and chats are present. If you use a browser, make sure that you clear the browser cache or create a private session to avoid any problems caused by stale data.
Teams is usually the last application to have its data restored and available to users.

If a workload is not quite ready an hour or so after the restore, wait for another hour, and check again. Because the restoration process involves many different connections, it could be that
one of the steps in the process is waiting for another step to complete.

You have 30 days to recover a group following its deletion. After this period elapses, you might still be able to recover some information for a deleted group. For example, administrators
might be able to recover files belonging to its document library from the SharePoint recycle bin. If the group was on-hold and the hold is still in place, you can run a content search to find
information in the group mailbox and document library and export those items. However, you will have individual items and cannot reassemble them into a functioning group because the
group object no longer exists in Azure Active Directory.

To purge a soft-deleted group and permanently remove its contents from Office 365, run the Remove- AzureADMSDeletedDirectoryObject cmdlet and pass the object identifier for the group to
purge.

[PS] C:\> Remove-AzureADMSDeletedDirectoryObject -Id $RecoveryGroup

PowerShell does not ask you to confirm the purge, so be careful when you run this command.

Recovering Individual Documents and Items

The possibility always exists that someone will remove a document or conversation in error. The next step is to ask how to restore the now-deleted item. The first place to look is the recycle
bin where items stay for up to 93 days after deletion. If someone removes the item from the recycle bin, you might be able to use the fact that Microsoft takes regular backups of SharePoint
sites and holds those backups for 28 days. Within this period, you can file a support request to ask Microsoft to restore a site from backup. The problem here is that the restore occurs for the
complete site instead of just the missing document and so might overwrite updates to other documents that happened since the backup job ran. One solution is to mark important documents
(or complete document libraries) with a label that has a retention action. Users cannot remove files from SharePoint when they have a label with a retention action.

When someone removes a conversation from a group, the item goes into the Deletions folder in the group mailbox and stays there for 14 days, after which the Managed Folder Assistant
permanently removes the item from the mailbox database. You cannot use the standard Recover Deleted Items feature in Outlook or OWA because it is only available for personal mailboxes.
During the 14-day retention period, you can recover the item by running a content search against the group mailbox to find the item and then export the item to a PST file. Another potential
method is to recover a copy of an item sent to group members who subscribe to receive updates via email. If you move items back into a mailbox, you can drag and drop them into a group
with OWA. But the most effective method to recover deleted items in a group mailbox is to run the Get-RecoverableItems and Restore-RecoverableItem s cmdlets to view and restore deleted
items. See Chapter 19 for information.

Backup solutions from ISVs offer the ability to backup and restore documents selectively, which is a better outcome than having to restore a complete site. AvePoint has a backup solution for
Office 365 Groups that can restore conversations.

Yammer and Office 365 Groups

Microsoft bought Yammer for $1.2 billion in June 2012 to gain a presence in the enterprise social networking market. Facebook defines the concept of a social network to many people, a
feeling emphasized by the film of the same name that charts the development of Facebook from an application used to connect college students together in an electronic yearbook to a
platform that supports billions of users today. Facebook is a public social network that is open to anyone who cares to join. An enterprise social network, like Yammer, offer much the same
communication and collaboration facilities as you find in Facebook, but inside the confines of a single enterprise. Of course, you can stretch the concept of a single enterprise to a public
space, as in the case of Microsoft's now-defunct Office 365 Network where Yammer hosted over 88,000 members discussing technical issues and ideas in the many groups that reflect the
different interests of Office 365 customers. Microsoft transitioned the Office 365 Network from Yammer to a new platform in September 2016.

Microsoft enables Yammer for all Office 365 enterprise tenants. Once activated, you can use Yammer in diverse ways. Some find that Yammer is an excellent way to introduce users to the
new world of the cloud in the early stages of an Office 365 deployment. Yammer uses a web client and behaves very much like other social networking networks (think Facebook or LinkedIn)
and users who are familiar with those applications can quickly become productive with Yammer. Desktop and mobile Yammer clients are also available. Given a well-curated set of groups for
people to go to find and share information, Yammer can quickly become a success factor in an Office 365 roll-out, especially when the number of potential users who might contribute to
conversations is more than can be accommodated by Office 365 Groups or Teams.

Apart from the basic social networking aspect, Microsoft has integrated Yammer in various Office 365 applications. For example, if you use the Delve blogs capability and want to comment on
a post in someone’s blog, you will do so through Yammer. The same is true if you want to comment on a video playing in the Office 365 Video portal (Figure 11-20 ). To continue with the plan
to make Yammer the social layer within Office 365, Microsoft will make a new Yammer web part for SharePoint Online sites and a new connection capability for a Teams channel available in
early 2019.

Figure 11-20: Adding a comment to a video via Yammer

Yammer supports sharing across groups (in other words, you can cross-post a conversation to have it appear in multiple groups). It is also possible to search for content across the groups to
which you belong. And of course, Yammer comes with the compulsory marks of approval seen in other social networking products - the infamous “Like” and emoticons. Yammer supports
browser, Apple iOS, Android, and Mac clients. Yammer also boasts integrations with various other applications such as Dynamics 365 and SalesForce.com.

Yammer brings its own distinct blend of collaborative capabilities to the Office 365 mix. It is collaboration writ large, created with the enterprise in mind. Yammer is best when used by
hundreds or thousands of contributors meeting in groups dedicated to different topics, especially when contributors are geographically remote from each other. Some examples of Yammer
groups usage are for cross-company knowledge sharing or to distribute information on behalf of specific parts of the business, such as a HR discussion group or an IT suggestions group.

Other solutions (like Office 365 Groups or Teams) can work better when teams or small groups of people need to come together to figure out the answer to a specific problem, especially
when the discussion centers around shared authoring of common material rather than conversations. Groups that need for document management or compliance functionality are also
examples where Yammer might not be the best fit.

Some consider Yammer as a competitor for Exchange, or even as a complete replacement for email. Beauty is in the eye of the beholder and possibly this feeling arises more often in
situations where email has been badly deployed or managed so that an unreliable or underperforming service is delivered to end users, or where user education has suffered, and people
simply do not know how to use email effectively in business situations. It is also true that a generation bred on the kind of social interaction epitomized by Facebook is likely to be attracted to
a similar means of communication within their employment. If they wish, a user can interact with Yammer completely by email because they can receive contributions to group conversation
via email and respond via email. Some people have gone to great lengths to tailor a set of Inbox rules to process messages generated by Yammer to sort them into various folders. In this
scenario, the user would never have to go near Yammer except when they wanted to join or leave groups or adjust their Yammer settings.

The Evolution of Yammer Groups

Anyone who looks at the complete Office 365 feature set might well conclude that some overlap exists. Why use Office 365 Groups instead of Yammer or vice versa? How do these
applications compare to Microsoft Teams, which some commentators speculate will eventually replace Yammer. All these applications allow users to contribute to discussions hosted in
“groups” (or teams) and preserve contributions so that they are accessible to other users who join the group after a conversation starts. The applications support the sharing of files and are
accessible through a browser interface – and that is pretty much where the similarities end. Yammer’s focus is on social networking capabilities for large enterprises (the largest customer
Yammer networks operational today support well over 100,000 members) while the membership of both Office 365 Groups and Teams are limited to much smaller populations.

Because Office 365 Groups can function like distribution groups, they are easy to deploy for any tenant who has ever used Exchange. Users who interact with Office 365 Groups through
email and never access the shared resources will probably think of them as just like distribution groups. You send messages to groups and copies arrive in the mailboxes of any member who
subscribes to the group. Given the large numbers of Exchange Online mailboxes, Office 365 Groups has a lot of room to grow.

Yammer is different. You can certainly interact with Yammer through email, but the experience is less seamless than with Office 365 Groups. Yammer discussions are best experienced
through the browser interface or the Yammer mobile client. Even though Microsoft enables Yammer for all Office 365 enterprise plans and does the heavy lifting of implementation, the
decision to deploy Yammer inside a company needs more thought and preparation than simply turning it on. Such an approach will invariably result in a couple of active groups (for some
period) and a lot of stagnation. Any project to introduce a new form of collaboration to an enterprise, even though people are aware of the power of social networking through their exposure
to consumer versions, needs planning, evangelism, leadership and endorsement from executive level, and a whole lot of energy applied to get the network going, including the identification
of suitable ways to use the network to solve real-life business problems. An electronic chat room is nice to have but worthless in the long term; an active social network that brings people
together on an ongoing basis to debate, refine, and drive solutions to identifiable and quantifiable business issues is invaluable.

Yammer groups are the core of any deployment because this is the way that Yammer organizes conversations into discrete areas within the tenant network. Traditionally, Yammer groups have
only functioned within the Yammer ecosystem and have not had many points of integration with other parts of Office 365. The current generation of Yammer groups use Office 365 Groups to
manage their identity membership. Members of new-style Yammer groups can access all the resources that you would expect from of an Office 365 group. Over the long term, Microsoft plans
to make Office 365 resources available to older Yammer groups.

Requirements for New-Style Yammer Groups

You continue to use traditional Yammer groups until your Yammer network meets the prerequisites to use the Office 365 Groups service. The requirements are as follows:

You must enable Office 365 Group creation for the tenant. If your tenant does not allow the creation of Office 365 Groups, users can still create old-style Yammer groups.
The Yammer network configuration is initially only available for 1:1 networks. In other words, a single Yammer network exists within an Office 365 tenant. If multiple networks exist, you
must combine (merge) the networks before Yammer can use Office 365 Groups.
The tenant must enforce Office 365 identities for Yammer. Office 365 Groups are based on Azure Active Directory, so Yammer must use Azure Active Directory as its source of
authentication and identities. Yammer continues to have its own directory in the same way as other workloads do, but Azure Active Directory is the directory of record. Switching to
Office 365 identities is effective at once after an administrator makes the change.
You cannot add external users (of another Yammer network) to new-style Yammer Groups. This is because those external users are not known to Azure Active Directory and therefore
cannot join the membership of the underlying Office 365 Group. Yammer groups do not support the guests used by Office 365 Groups.

After meeting the requirements, any new Yammer groups that you create use the Office 365 Groups service. When you create a new-style Yammer group, much the same provisioning
processes as used to create Office 365 Groups swing into place to create:

The new group object in Azure Active Directory as the group identity and hold its membership. The group appears in the Exchange GAL.
A new SharePoint team site to allow the members of the Yammer group to store documents, the shared notebook, and to access the other components of the team site.
A new group mailbox to hold the shared calendar.
Yammer storage to hold the threaded conversations for the group.

You cannot convert a group that uses Exchange to store its conversations to use Yammer or vice versa. Apart from sharing a membership service, the two types of group operate
independently of each other and Microsoft has no migration tools to move one type to the other. If you make a mistake and create a group with the wrong conversation storage, you must
remove that group and recreate it in the correct application.

When you have a Yammer group connected to an Office 365 group, you can work with it using the Azure Active Directory and Groups cmdlets. For example, you can set a new primary email
address for a Yammer group or its classification by running the Set-UnifiedGroup cmdlet. You can also recover a deleted Yammer (new-style) group using the process to recover a deleted
Outlook group. Although you can add members to Yammer groups using the Add-UnifiedGroupLinks cmdlet, it is best to manage membership through Yammer and this minimizes any
directory synchronization lag.

Table 11-2 compares traditional Yammer Groups against Yammer Groups connected to Office 365 Groups.

Old-style Yammer Groups New-style Yammer Groups (connected to Office 365 Groups)

Group created and managed in Yammer directory Azure Active Directory

Membership managed by Yammer Office 365 Groups service

Discussions stored in Yammer data store Yammer data store

Access to shared calendar No Yes – in group mailbox

Access to SharePoint Online team site No – documents and other files stored in Yammer Yes. From November 2018, all new files posted in these groups are stored in SharePoint.

Table 11-2: Comparing old-style and new-style Yammer Groups

Allowing Yammer to exploit the shared identity offered by Office 365 Groups to access resources is a good step forward. Some rough edges exist that will be smoothened over time. For
instance, because both old-style and new-style Yammer groups can exist within a network, you might end up in the situation where some documents are in the Azure-based Yammer Files
storage and some are SharePoint Online. Documents are best stored in SharePoint to take advantage of the compliance and record management features available in that platform. A
comparable situation arises between Yammer Notes and OneNote. The long-term direction is to use OneNote, but some groups might like to continue to use Yammer Notes.

Yammer Directory : Although new-style Yammer groups use Azure Active Directory and the Office 365 Groups service, Yammer manages the membership of the groups in its own directory.
In line with the general recommendation that you should manage applications using their own tools, it is best to update the membership of Yammer groups through Yammer.

Yammer Discussions
Leaving Microsoft Teams aside, Office 365 tenants therefore have a choice of technology to host group discussions: Yammer or Exchange (Outlook Groups). The difference between new-style
Yammer groups and Outlook Groups is that Yammer groups hold their conversations in the Yammer data store while an Outlook group stores its conversations as items in the Inbox folder in a
group mailbox. A Yammer group that uses the Office 365 Groups service is easily identifiable because it lists links to the Office 365 Resources provisioned for the group (Figure 11-21 ).
Figure 11-21: A threaded discussion viewed through the Yammer browser interface

Although you cannot transform a Yammer-style group to become an Outlook-style group or vice versa, it is possible to run the two types of group alongside each other. After all, both use a
common group framework and the only major differences are the storage of conversations and the clients that people can use to access a group. Outlook and OWA cannot access
conversations in a Yammer group. Access to conversations in Yammer groups is only possible through the Yammer browser and mobile clients. Although they do not use Exchange Online for
messaging, members of Yammer groups can still send messages and users can also send messages to Yammer groups.

Any consideration of collaboration within Office 365 should take Yammer into account. You should lay out the needs of the organizations and map those needs against the types of groups
supported inside Office 365. You can then decide which type of group is best suited to specific scenarios. Although the question seems a tough one to answer, the reality is that most
organizations will find the choice relatively straightforward. It is likely that most tenants will standardize on a certain type of group and use those groups wherever possible. Those who have
used email in the past will continue to do so with Outlook-based groups while organizations that deployed Yammer will pick up the new-style Yammer groups when their Yammer networks are
ready for the change.

Yammer Files
Traditionally, when people uploaded files to Yammer, Yammer used its own storage. In November 2018, Microsoft told tenants that they had started the roll-out of code to switch storage for
all new files to the SharePoint site owned by Yammer groups connected to Office 365 Groups. Existing files stored in these groups, files owned by private messages, and files belonging to old-
style Yammer groups are unaffected. The plan is to complete the roll-out by March 2019. This is an important move because it means that the files can then be processed by Office 365 data
governance policies such as retention policies, Office 365 sensitivity labels, and so on. Over time, Microsoft will migrate all the files stored in Yammer storage to SharePoint. The migration
will be done automatically with no need for administrator action.

Evaluating Yammer, Groups, and Teams

Occasionally, Office 365 offers too many ways to get work done. Selecting the right tool for team collaboration is one of those times. Office 365 Groups offer a solid choice for teams to work
together based on email while Microsoft Teams delivers a chat-based workspace that works well for small to medium teams. Both Office 365 Groups and Teams support external access.
Yammer completes the picture.

Because many factors drive the choice of the best collaboration tools for an organization, it is hard to make a choice and say that any one tool will handle all circumstances and contexts. In
fact, no collaboration tool ever invented in the history of technology has been able to make that claim in any credible sense. To help you figure out the best choice of tools for use inside your
tenant. Table 11-3 lists some of the points that you can consider when comparing the different group/team-centric collaboration methods supported by Office 365.

Characteristic Office 365 Groups Yammer Groups Microsoft Teams

Office 365 Exchange Online (conversations and calendar). Yammer (conversations). Exchange Online (calendar). Azure data services (messages and media items).
technologies SharePoint Online (team site) SharePoint Online (team site). OneNote (shared notebook). Exchange Online (calendar). SharePoint Online
used Planner (plans) (document library).
OneNote (shared notebook). Planner (plans)
OneNote (shared notebook). Planner (plans).
Connectors
Connectors, Bots, Apps, and Tabs.

Clients Outlook 2016 Yammer (browser) Teams (browser)

Outlook for Mac Yammer (Windows and Mac desktop) Teams (Windows and Mac desktop)

Outlook Web App Mobile Mobile

Outlook mobile apps

Scalability Maximum of 2,500 members (dynamic groups are The membership limit is > 50,000 (unless synchronized with an Maximum of 2,500 members (dynamic groups are also
also supported). on-premises directory) supported)

Access Default private but can be set to public. Default public (any user can join), can be set to private. Default private but can be set to public.

Communication Choice of direct access to conversations or by Threaded discussions that occur over wide-ranging topics. Support for personal and shared (channel) messages
style sending email to the group. Contributions can be Contributions are usually short and entered directly through the organized in threaded conversations, which are usually
short or long. A group can host thousands of browser. Interaction through email is possible. very short. The conversations are persistent. Email
conversations. support is limited to acceptance of inbound mail.

Search Search constrained to conversations within a single Search across all Yammer groups to which a user has access. Search within all teams, a single team, and a channel
group. You need to run a separate search using (including associated files).
SharePoint or Delve to search files in document
library. Outlook 2016 supports search across
pinned groups.

Group purpose Team-based collaboration centered on email and Wider (up to entire company) collaboration and sharing of ideas. Small teams working on projects. A good choice for
shared documents. Good choice for teams that A good choice to host open discussion forums, groups dedicated teams that need to work on a specific problem or issue.
 work together on an ongoing basis for a prolonged to sharing knowledge and information to broader community, A team can have up to 200 channels to use to segment
period. Replacement for traditional email and when a “community of interest” form of collaboration is discussions.
distribution lists. required.

Discoverability Office Graph used to recommend suitable groups to Yammer recommends suitable groups to users Office Graph used to recommend suitable teams to
users users.

Management Office 365 Admin Center and EAC. Broad range of Yammer administration console. No PowerShell support, apart Teams and Skype for Business Online Admin Center.
PowerShell cmdlets to interact with mailbox and from being able to use some of the Groups cmdlets to manage Separate PowerShell module.
group. group properties.

Platform Support for Office 365 tenant and guest users with Office 365 users only. Support for Office 365 tenant and guest users with
some limitations for users with on-premises some limitations for users with on-premises mailboxes.
mailboxes.

Support for Support for content search, eDiscovery cases, No support for discovering or searching conversations. Files Support limited to copies of conversations captured in
Office 365 Office 365 labels, retention policies, and the Office stored in SharePoint are indexed and discoverable (see section Exchange mailboxes. These are indexed and
compliance 365 audit log. above about migration of Yammer storage to SharePoint). discoverable and can be included in content searches
features and eDiscovery cases and come within the scope of
retention policies.

Data Supported in all Office 365 datacenter regions. Hosted in U.S. datacenters. Hosted in most Office 365 datacenter regions. Not
sovereignty available to sovereign clouds.

Table 11-3: Comparing different Office 365 collaborative applications

Some guidelines are necessary to help administrators and users select the best form of collaboration technology for a specific task. The following might serve as a basis for discussion:

An Outlook group is an excellent choice when a team has a need for a collective repository where they can share documents, notebook, and calendar, and discuss topics through email. It
is likely that the group has a well-defined purpose and might be part of the organizational structure, like a department or location. Because these groups often serve as a direct
replacement for traditional distribution lists, the email traffic can be relatively high compared to the number of files generated in the document library. Together, the files and
conversations held in the group represent its collective history and are valuable in terms of bringing new members up to speed with the group’s activities. Office 365 Groups and the
Outlook apps offer the most comprehensive support for the Office 365 data governance framework.
A team is a good choice for small to medium projects where sets of people come together to execute specific tasks. Deadlines are likely to be in place for the team to achieve and this
encourage a certain kind of focused and rapid conversation between team members that Microsoft refers to as “high-velocity chats”. The chats tend to be short and informal. The
intention behind a team is to get work done rather than to accumulate knowledge. When the project or work item completes, the team membership might disband and reform in other
teams.
Yammer groups are an effective way to share knowledge broadly within large communities. They work well as public forums within companies, where Yammer groups often serve as the
basis for communities who share a common interest. Unlike Teams, and Office 365 Groups, which can be private or public, Yammer Groups often host unstructured audiences that flex
over time. The information held in Yammer groups is often not as specific as found in Office 365 Groups or Teams but is no less valuable. Indeed, the knowledge developed through the
contributions of the higher number of users supported by Yammer groups often has long-lasting value for the entire company.

It is obvious that considerable overlap exists between the various methods. An Outlook group might be as good a choice as a Team in some circumstances. Some companies use nothing but
Yammer and are quite happy with their choice. Some will find that Teams revolutionize the way that small groups work. All of which goes to prove that collaboration is uniquely individual to a
company or organization. No one size fits all, which is why choice exists within Office 365.

Power BI and Groups

The integration between Power BI and Office 365 allows groups to control access to Power BI workspaces. You need a Power BI Pro license before you can use Office 365 Groups in this
manner. The Power BI integration also supports the ability to create new Office 365 Groups. The experience is like the OWA creation process as similar UI collects information about the
properties of the new group, and so on. Power BI populates only a small number of group properties and you might want to update the properties afterwards to comply with naming policy
(for example, to add a prefix to the group’s display name), subscribe members for updates, and so on. For example, to update group properties so that members receive conversations via
email, run the following command:

[PS] C:\> Set-UnifiedGroup –Identity PowerBIAficionados –AutoSubscribeNewMembers:$True

-RequireAuthenticationEnabled $False

When you add members to a new Office 365 Group through Power BI, you input their email addresses. However, Power BI does not access AAD to confirm that these users exist. Like the
other group-enabled applications, it takes longer for a group created with Power BI to synchronize and show up in the directory than for groups created with Outlook or the Office 365
administrative interfaces. Guest user access is not supported to Power BI.

Stream and Groups

Microsoft Stream uses groups to organize videos and to control access to content. Each group gets its own video portal that you can restrict to the members of the group. Videos can exist in
multiple groups or even within multiple channels within a group. Stream is available to non-Office 365 customers as well as Office 365 tenants. When Office 365 tenants create groups inside
Stream, they create Office 365 Groups. Figure 11-23 shows the creation of a new group.

Figure 11-23: Creating an Office 365 Group from Stream

Although many of the settings used when creating a new group are familiar, because Stream is available to customers without Office 365, some of the nomenclature is different.

Make this group company-wide is the privacy setting and means that the group is public, and any video owned by the group is available to anyone in the tenant. If the group is not
 company-wide, it is private and only members of the group can access the videos owned by the group. In addition, like other private groups, an administrator or group owner must add
new members.
Allow members to contribute controls who can upload videos to the group and create or remove channels. This is a setting specific to Stream and does not affect how the Office 365
Group works. If users cannot upload content, they are restricted to view-only access to videos.

Channels are how Stream organizes content. You can have company-wide channels and group channels. Company-wide channels do not belong to any specific group. These channels hold
content that you want to make available to everyone within the tenant. The videos in these channels must allow company-wide viewing. A group channel belongs to a specific group and
inherits the membership of that group. For more information about Stream, see Chapter 7 in the companion volume.

More Groups
It is obvious that we need to learn more about how to manage the possibilities that exist within Office 365 Groups. Chapter 12 is our next port of call and it is where we delve much deeper
into the dark art of managing Office 365 Groups.
Chapter 12: Managing Office 365
Groups
Tony Redmond

In the last chapter, we covered the architecture and use of Office 365 Groups. This chapter takes a different tack and
discusses the management of groups, including dynamic groups. We look at the Azure Active Directory policy used by
Office 365 to store settings for groups. As ever, there is lots to cover.

Groups Policy
Office 365 Groups offer an identity and membership service to multiple applications. Those applications must know
how a tenant wishes groups to function. To give guidance to group-enabled applications, Office 365 stores settings in
an Azure Active Directory policy. Applications then retrieve settings from the policy to know how they should interact
with groups. Table 12-1 lists the policy settings, including those used to control who can create new groups (and
therefore teams) within the tenant.

Policy setting Use

AllowToAddGuests Controls whether users can invite guest to become members of Office 365 Groups.
By default, this setting is True , meaning that group owners can add guests.

AllowGuestsToAccessGroups Controls whether guests can access content held in the SharePoint site belonging to
Office 365 Groups. By default, the value is True .

AllowGuestsToBeGroupOwner Controls whether guests can be group owners.

ClassificationDescriptions Comma-separated descriptive pairs for the classifications available in the tenant.

ClassificationList Stores a comma-separated list of valid classification values to give users a visual
indication of what is stored in different Office 365 Groups.

CustomBlockedWordsList A list of words that the tenant does not wish group owners to use in the names of
new groups.

DefaultClassification Sets the default classification for Office 365 to apply to new groups.

EnableGroupCreation Controls whether users can create groups. By default, this value is True and any
user can create a new Office 365 Group.

GroupCreationAllowedGroupId Points to the object identifier (GUID) of a security, distribution, or Office 365 group
whose members can create new Office 365 Groups. This group is used if the
EnableGroupCreation setting is False .

GuestUsageGuidelinesUrl Defines a URL displayed in client user interfaces to remind users about guidelines
for how to share company information with guests. For example:
http://mydomain.com/GuestUserAccessO365Groups.html


PrefixSuffixNamingRequirement Used by the group naming policy to define the pattern of prefix and suffix to apply
to the display names of new groups.

UsageGuidelinesUrl Defines a URL displayed in client user interfaces to give guidelines about the
creation and usage of Office 365 Groups. For example:

http://mydomain.com/HowtoUseO365Groups.html

Table 12-1: Settings in the Office 365 Groups policy

While you can configure the AllowGuestsToBeGroupOwner setting to True to allow guests to become owners of
groups, this capability is not exposed in the Groups or Teams user interfaces.

Licensing Groups Functionality

If you use Office 365 Groups or Teams without changing any setting in the groups policy, the only license you need are
those included in your Office 365 plan. Things become more complex if you decide to utilize some of the more
advanced features enabled and controlled through policy settings because Microsoft requires users to have Azure
Active Directory premium P1 licenses to benefit from the added functionality. Microsoft’s view is that these features
reduce administrative effort or make groups more productive for organizations, so they price the features accordingly.
The reason why premium Azure Active Directory licenses are needed is that Microsoft views the features as
extensions of the basic Azure Active Directory plan bundled with every Office 365 tenant.

The general rule is that any account that gains benefit from a feature must be licensed. Sometimes this rule is easy to
understand, as in the case of dynamic Office 365 Groups where you need to license the accounts found by the queries
used to populate group membership. For instance, if you create a dynamic group based on a query that finds all users
with mailboxes in the tenant, it means that you need licenses for all those accounts. In other cases, the logic is harder
to follow. For example, if you implement a policy to control the creation of new groups, you must license every account
in the tenant because every account comes under the control of the policy and is either allowed or denied the ability
to create new groups. On the other hand, if you use the expiration policy to control how long groups exist before their
owners must reconfirm that the groups are needed, you only need licenses for the members belonging to the groups
to which you apply the expiration policy.

Table 12-2 summarizes the licensing requirement for the features you can activate or control through the groups
policy. See this support article for more information.

Feature controlled by a group policy setting Requires Azure Active Directory Premium P1

Allow non-admin users to create groups No

Restrict ability to create new groups to defined set of accounts Yes

Use expiration policy to force renewal of groups after set period Yes

Create and use dynamic Office 365 Groups Yes

Apply group naming policy Yes

Apply default classification to new groups Yes

Provide URL with usage guidelines to internal users Yes

Provide URL with usage guidelines to guests Yes

Allow guests to access groups No

Force new groups to have classification selected from set list Yes

Allow guests to participate in groups No

Table 12-2: License requirements for features controlled by group policy settings

Obviously, the need for Azure Active Directory premium licenses to use certain features make them less attractive
than if Microsoft bundled the functionality into the standard Office 365 plans. However, you might have other reasons
to acquire Azure Active Directory premium licenses, such as buying the Enterprise Security and Mobility suite, so the
problem might not be as big as it seems.

Guest Access Licenses

Guest access to Office 365 applications is included in all Office 365 Business Premium and Enterprise subscriptions,
which means that you do not need to assign licenses to guest users to access applications like Groups, Teams, and
Planner. However, Microsoft’s guidance for B2B collaboration licensing sets out some rules for guest access to “paid”
(premium) Azure Active Directory features like conditional access policies. Briefly, for every paid license you buy, you
secure B2B collaboration use rights for five guest users invited to the tenant. You cannot assign licenses directly to
guest accounts, so the licensing rights depend on licenses held by tenant users.

Creating and Updating the Groups Policy

By default, a tenant does not have a Groups policy and applications use default settings. To change any settings, we
first create a directory settings object to hold the policy settings from the correct template. Azure Active Directory
loads default values into the new policy and you can then update those settings. Some of the examples described here
use the cmdlets in the preview release of Version 2 of the Azure Active Directory PowerShell module.

Several directory settings templates exist in the instance of Azure Active Directory used by an Office 365 tenant,
including one to hold group policy settings. A different template is available to control guest access for specific groups
(explained later in this chapter). To see the available templates in a tenant, use the Get-
AzureADDirectorySettingTemplate cmdlet.

[PS] C:\> Get-AzureADDirectorySettingTemplate

Id DisplayName Description

-- ----------- -----------

62375ab9-6b52-47ed-826b-58e47e0e304b Group.Unified ...

08d542b9-071f-4e16-94b0-74abb372e3d9 Group.Unified.Guest Settings for a specific Unified Group

4bc7f740-180e-4586-adb6-38b2e9024e6b Application ...

898f1161-d651-43d1-805c-3b0b388a9fc2 Custom Policy Settings ...

5cf42378-d67d-4f36-ba46-e8b86229381d Password Rule Settings ...

80661d51-be2f-4d46-9713-98a2fcaec5bc Prohibited Names Settings ...

aad3907d-1d1a-448b-b3ef-7bf7f63db63b Prohibited Names Restricted Settings ...

The template used to create the new Groups policy for the tenant is the one with the display name “Group.Unified. .”
The one used for a specific group has the display name “Group.Unified.Guest .” To create a new groups policy to hold
customized settings for the tenant, we run the New-AzureADDirectorySetting cmdlet to create the new policy from
the template.

[PS] C:\> Connect-AzureAD

[PS] C:\> $Policy = Get-AzureADDirectorySettingTemplate | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings = $Policy.CreateDirectorySetting()

[PS] C:\> New-AzureADDirectorySetting -DirectorySetting $Settings

When the New-AzureADDirectorySetting cmdlet runs, it creates a new directory settings object. Any application that
can access Azure Active Directory can then check the settings to know what it needs to do to respect the policy. For
example, when a user tries to create a new Office 365 Group, the client can verify that the user can create a new
group. To view the policy settings, we run Get-AzureADDirectorySetting cmdlet. In this case, many of the settings
have non-default values.

[PS] C:\> Get-AzureADDirectorySetting | ForEach Values

Name Value

---- -----

CustomBlockedWordsList

ClassificationDescriptions General Usage:Anyone can access,External Access:Available outside the company,Internal Only:Must not be
shared with external people, Confidential:Can only be disclosed with management permission

DefaultClassification General Usage

PrefixSuffixNamingRequirement

AllowG uestsToBeGroupOwner False

AllowGuestsToAccessGroups True

GuestUsageGuidelinesUrl

GroupCreationAllowedGroupId A3c13e4d-7083-4448-9224-287f10f23e10

AllowToAddGuests True

UsageGuidelinesUrl http://office365itpros.com/GroupGuidelines.html

ClassificationList General Usage,External Access,Internal Only, Confidential

EnableGroupCreation False

We will discuss how to update individual settings in the Groups policy using the Set-AzureADDirectorySetting cmdlet
over the next few pages. One thing to remember is that names of policy settings are case-sensitive, so be sure to type
the names exactly as shown here (for example, (“AllowToAddGuests ” rather than “Allowtoaddguests”). If you use the
wrong name, Azure Active Directory will not be able to update the policy. Another thing to consider is that Exchange
Online distributes mailboxes for a tenant across multiple servers that can run inside different Office 365 datacenters,
so it can take an hour or so before new or updated settings are active across a tenant. Finally, if you make a mistake
and want to start over with a new group policy, you can do so by removing the directory settings object:

[PS] C:\> $DeleteObjectId = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> Remove-AzureDirectorySetting -Id $DeleteObjectId.Id

In the rest of this section, we cover how to update the Groups policy with settings to control classifications, group
creation, naming, and group guidelines.

Classifications Policy Settings

Some groups host internal discussions, some hold confidential information that the tenant wants to keep restricted
within the group, and some exist to share information with guests. The ClassificationList setting in the Groups policy
holds a list of classifications that group owners can apply as a visual sign to tell users how they should deal with the
information held in a group. OWA and Outlook 2016 display the group classification in the conversation and files
views, SharePoint displays the classification in the header of the team site, while Teams displays the classification in
its header when working with conversations, files, and so on. (Figure 12-1 ).
Figure 12-1: Classifications for groups in Teams (top) and Outlook (bottom)

Defining Classifications
Classifications are defined in the Groups policy as a comma-separated list within quotes without spaces between the
values. For example:

“Confidential,External Access,Top Secret”

These PowerShell commands show how to update the classification settings for a tenant. We:

Fetch the existing settings.

Update the DefaultClassification setting to be “Confidential.” The default classification must exist in the
classification list. Office 365 applies the default classification to a new group if the owner does not select another
classification. Using a default classification is a feature needing Azure Active Directory Premium P1 licenses.
Update the ClassificationList setting with a comma-separated list of classifications. Some tenants use over 100
classifications, but it is often wiser to have a more restricted set so as not to confuse users.
Update the ClassificationDescriptions setting with more expanded explanations of what each classification
means.
Update the Groups policy with the new settings.

[PS] C:\> $Settings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings[“DefaultClassification”] = "Confidential"

[PS] C:\> $Settings[“ClassificationList”] = " General Usage,External Access,Internal Only,Confidential"

[PS] C:\> $Settings[ClassificationDescriptions"] = " General Usage:Anyone can access,External Access:Available outside the
company,Internal Only:Must not be shared with external people,Confidential:Can only be disclosed with management permission"

[PS] C:\> Set-AzureADDirectorySetting -Id $Settings.Id -DirectorySetting $Settings

Assigning and Changing Classifications

Usually when creating a group, the owner selects the value that best describes the information held in the group from
the list. Placing a classification on a group is strictly advisory. The classification gives users a visible clue as to the
sensitivity of the information discussed in the group, but does not affect how the group works, nor does it protect
content in any of the group resources or place any restriction on how that content is used. Adding or updating a new
classification or removing a classification from the list does not affect classifications placed on existing groups.

The ClassificationDescriptions setting is associated with the classification list. Although you do not need to provide
descriptions, it is usually a good idea to give people a hint about which classification is the most appropriate for a new
group or team. The classification might be enough, but if not, you can give some extra information for each
classification in the form of a short description. Azure Active Directory will not complain if you input long descriptions
(say, more than 100 characters), but the space reserved to display descriptions in the user interface of different
applications might, so it is best to keep descriptions short and snappy. Make sure that you give a description for each
classification in the ClassificationList setting and use the form classification:description as shown below.

You can discover the classifications assigned to groups with a simple PowerShell command:

[PS] C:\> Get-UnifiedGroup | ? {$_.Classification -ne $Null} | Format-Table DisplayName, Classification

To change a classification for a group after creation, edit the property through a client. For example, in Teams, use
Edit team and then change the setting while in SharePoint, the classification is available in Site information , while
in OWA and Outlook you can change the classification through Edit group . Alternatively, use the Set-UnifiedGroup
cmdlet. Make sure that you use a value from the list in the policy as otherwise the cmdlet will fail:

[PS] C:\> Set-UnifiedGroup -Identity Hydra -Classification "General Usage"

Groups created with PowerShell won’t have a classification unless one is specified in the New-UnifiedGroup cmdlet.
Also, groups created before Microsoft introduced the ability to assign classifications to groups in mid-2017 might still
not have classifications. To fix the problem, you can run this command to assign a default classification to all
unclassified groups:

[PS] C:\> Get-UnifiedGroup | ? {$_.Classification -eq $Null} | Set-UnifiedGroup -Classification "Internal Only"

Controlling Group Creation

If a tenant allows unrestricted creation of Office 365 Groups, each licensed user in the tenant can exploit twenty-two
separate ways available across a spectrum of clients and applications to create up to 250 groups. A thousand-user
tenant therefore might end up with 25,000 user-created groups to add to the groups created by administrators.
Allowing people to create the groups they think they need through a self-service model is in line with Microsoft’s
original view that control over groups should be user-led. Although the model gives users the power to decide what
groups they need to collaborate with their colleagues, it’s easy to see how chaos can result and why, in most cases, it
is better when administrators put some thought and planning into the process of group creation and maintenance.
This is especially true in large enterprises.

Previous experience with user-controlled creation of shared objects, such as the mayhem that often occurred with
public folders when anyone could create top-level folders in the early versions of Exchange, proves that if you allow
users free rein to create new objects, you can expect a rapid expansion of those objects, many of which duplicate the
purpose of other objects. The usual upshot of creating many groups for no good reason is user confusion. People do
not know which groups to use for what reason, especially if you do not implement a naming policy and groups with
duplicate names arise from Teams, Yammer, and Outlook. This can lead to chaos in the GAL with groups scattered
amongst other mail-enabled recipients. Creating groups often leads to groups used for a period and then left to decay.
It is to impose order on potential madness that tenants decide to exert control over who can create new groups.

Blocking creation of different group types within a tenant : In the past, Microsoft used the
EnableGroupCreation setting in the tenant’s Azure Active Directory configuration to control the creation of all types
of groups, including security groups and Office 365 Groups. From August 12, 2017, a separation exists between the
settings that apply to security groups and those that apply to Office 365 Groups. The rule is simple:

If you want to restrict the ability of users to create security groups, run the Set-MsolCompanySettings cmdlet to
set EnableGroupCreation to $False in the Active Directory configuration. You can check the current setting by
running the Get-MsolCompanyInformation cmdlet.
If you want to restrict the ability of users to create Office 365 Groups, you follow the advice given in this chapter
to create an Azure Active Directory policy for Office 365 Groups and set the EnableGroupCreation policy setting
to $False and the GroupCreationAllowedGroupId setting with the object identifier of the group holding the list of
accounts you allow to create groups.

The two policies do not depend on each other.

Administrative interfaces such as the Office 365 Admin Center, the Azure portal, or PowerShell are not subject to
policy control because end users cannot use these tools to create new groups. Users who hold administrative roles for
the tenant, including Exchange Administrator, can create new groups using the admin tools even when blocked by
policy. On the other hand, those holding administrative roles cannot create groups using apps like OWA or Teams
unless allowed by policy. The administrative roles that can always create groups are:

Company Administrator (global administrators).

Team Service Administrator.
User Account Administrator.
Mailbox Administrator.
Partner Tier1 Support.
Partner Tier2 Support.
Directory Writers.

In addition, groups created by these users do not come under the scope of the group naming policy

Policy-Based Group Creation

Follow these steps to create the settings to control the creation of new groups through the Groups policy.

Create a group for the set of authorized users : You need to know the set of users who can create new Office 365
Groups. To define the set of authorized users, you create a group to hold their names. This can be a distribution
group, a security group, or an Office 365 Group. A security group is the best choice for two reasons. First, if you
create new groups through SharePoint, you need a security group to control this process. Second, unless you mail-
enable the group, a security group does not appear in the GAL and is therefore less likely to be known to users. If you
restrict group creation and do not specify a group for authorized users, only administrators can create new Office 365
Groups.

Add the set of authorized users to the group : As explained in Chapter 11, each group type supports different
limits for the number of users who can be members. Because Office 365 Groups only exist in the cloud, the users who
create Office 365 Groups must have cloud accounts. Make sure that you add all the people you want to allow create
new groups as members of the security group as being an owner of the group is insufficient. Include those who hold
administrative roles so that they can create new Office 365 Groups from applications.

Prepare to edit the Azure Active Directory policy : Because no user interface exists for this purpose, you must
create the policy and populate its settings using cmdlets in the Azure Active Directory PowerShell module. The first
step is to retrieve the object identifier (ObjectId ) for the group that you just created to hold the set of authorized
users. For example:

[PS] C:\> $ObjectId = (Get-AzureADGroup -SearchString “GroupCreationControl”).ObjectId

You can see the same information through the properties of the group as exposed in the Azure Active Directory
console (Figure 12-2 ). The object identifier is among those shown for group properties. If you click the Properties link
in the action pane, the console reveals the general properties of the object, including its identifier. Copy the object
identifier to your clipboard, from where you can paste the value into PowerShell to use in commands.

Figure 12-2: Retrieving the object identifier for an Azure Active Directory group

Use PowerShell to update the Azure Active Directory policy : Two policy to control group creation.

Set the EnableGroupCreation setting to False to show that only authorized users can create groups.
Populate the GroupCreationAllowedGroupId setting with the object identifier of the group holding the list of
authorized users. This is the value copied from the group properties as shown in Figure 12-2 or retrieved using
the Get-AzureADGroup cmdlet. Office 365 does not check the object identifier when you use it in a policy setting,
so it is easy to make a mistake and end up in a situation where only an administrator can create a group. This is
the reason why you should either copy the object identifier from the Azure Active Directory console or use
PowerShell to get the value instead of typing it in manually.

Together, the two policy settings tell Office 365 applications that only the members of the authorized group can create
new groups. In this example, we fetch the current settings of the Groups policy into a variable and then update the
two settings with the necessary values. If you retrieve the object identifier of the security group with the Get-
AzureADGroup cmdlet as described previously, you can use it here, or you can paste the value copied from the Azure
Active Directory console. We then write the new values for the settings back into the policy.

[PS] C:\> $Settings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings[“EnableGroupCreation”] = "False"

[PS] C:\> $Settings[“GroupCreationAllowedGroupId”] = $ObjectId

[PS] C:\> Set-AzureADDirectorySetting -Id $Settings.Id -DirectorySetting $Settings

To check who can create new groups, we can read the membership list using the object identifier for the group stored
in the policy. The Get-AzureADGroupMember cmdlet can retrieve the membership of any type of group.

[PS] C:\> Get-AzureADGroupMember -ObjectID $Settings["GroupCreationAllowedGroupId"]

As noted later, Exchange Online stores copies of some group policy settings in its organization configuration, so the
following command also reveals who can create new groups:

[PS] C:\> Get-AzureADGroupMember -ObjectId (Get-OrganizationConfig).GroupsCreationWhiteListedId

Test that the new policy works : A user included in the membership of the authorized user group should be able to
create new Office 365 Groups from any application that supports the policy, including Planner, Teams, Dynamics,
Stream, SharePoint Online, and OneDrive for Business as well as the Outlook and Teams mobile apps. Applications
signal errors when a user who is not allowed to create a group tries to create a new group (something like “The group
couldn’t be created. Your admin hasn’t given you permission to create a new group ”).

Lack of Granularity in the Creation Policy : The use of a single policy to control group creation makes things
simple. The downside is that once you restrict creation, the decision applies to all applications that use Office 365
Groups for membership services. For instance, you might want to control the creation of new Plans but not Teams,
but the same policy applies to both applications, as it does to Stream, SharePoint, and the other group-enabled
applications. You should take this point into consideration when you decide to apply restrictions to group creation.

Changing the Authorized Group

It is possible that you want to swap the group specifying the set of users allowed to create groups for another group,
perhaps for testing purposes. You can change the group specified in the policy by retrieving the object identifier for
the group that you want to use and updating the GroupCreationAllowedGroupId setting in the policy. In this example,
we retrieve the object identifier for a security group called “Authorized Users” and use it to update the policy.

[PS] C:\> $ObjectId = (Get-AzureADGroup -SearchString “Authorized Users”).ObjectId

[PS] C:\> $Settings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings[“GroupCreationAllowedGroupId”] = $ObjectId

[PS] C:\> Set-AzureADDirectorySetting -Id $Settings.Id -DirectorySetting $Settings

Updating Membership of the Authorized Group with PowerShell

Using the Azure Active Directory console to add multiple users as members of the group allowed to create new Office
365 Groups can take a lot of time, especially in large tenants where the number of users who you might want to
authorize are in the hundreds or thousands. In these scenarios, some PowerShell scripting is probably the best way to
keep the group membership updated.

We must achieve two tasks. First, understand the users to add to the group membership. Second, add those users as
group members. We will get to the details of how to use PowerShell to manipulate group membership later. For now,
here is an example to find the users who can create groups by the presence of the string “Groups” in the
CustomAttribute7 attribute of their mailbox (the use of these custom attributes is explored in Chapter 6). Once we
know who they are, you can add the users to the group with the Add-UnifiedGroupLinks cmdlet.

[PS] C:\> $Users = (Get-Mailbox -Filter {CustomAttribute7 -eq 'Groups'})

| Add-UnifiedGroupLinks -Identity PermittedToAddOffice365Groups -LinkType Member -Links $Users.Alias

This code is a very simple example that lacks error handling or any of the other kind of processing that you might
want to include in a script intended to update the set of authorized users on a regular basis. However, it proves just
how quickly you can put something into place to automate the maintenance of this important group.

OWA Mailbox Policy for Group Creation in OWA and Outlook

A setting in the OWA mailbox policy was the original mechanism to control group creation. This was acceptable when
only Exchange Online clients could access Office 365 Groups. Although many other applications now use Groups, the
OWA mailbox policy is still in force today. The rule is as follows:

Users that the AAD Groups policy allows to create groups can always do so in non-Outlook apps (like Teams and
Planner).
Users that the AAD Groups policy allows to create groups and only do so in Outlook or OWA if the
GroupCreationEnabled setting in the OWA mailbox policy assigned to their mailbox is True .
Users can create groups in OWA or Outlook if the OWA mailbox policy assigned to their mailbox allows.

In other words, if you want to create groups through OWA or Outlook, the OWA mailbox policy assigned to your
mailbox must have GroupCreationEnabled set to True. Table 12-3 summarizes how the two policies control group
creation for different clients.

OWA Policy AAD Policy Can create in Outlook Can create in Groups-enabled apps

 Allowed Allowed Yes Yes

Blocked Blocked No No

Blocked Allowed No Yes

Allowed Blocked No No

Table 12-3: Control over group creation

Over time, the need for the OWA mailbox policy setting will reduce and will eventually reach the point where
Microsoft can remove it. In the interim, you can synchronize the two sets of permissions with PowerShell as a
workaround to ensure that the settings from the AAD policy apply everywhere. The code below first sets a restricted
OWA mailbox policy on all mailboxes. It then gets the membership of the group specified in the AAD policy who can
create groups and assigns another OWA mailbox policy that allows group creation to those mailboxes:

[PS] C:\> Get-Mailbox -RecipientTypeDetails Usermailbox | Set-Casmailbox -OwaMailboxPolicy OwaMailboxPolicy-Default

$Grp = (Get-AzureADDirectorySetting | ForEach Values | ? {$_.Name -eq "GroupCreationAllowedGroupId" } | Select Value)

$AuthorizedUsers = (Get-AzureADGroupMember -ObjectId $Grp.Value | Select UserPrincipalName)

ForEach ($Mbx in $AuthorizedUsers)

{ Set-CasMailbox -Identity $Mbx.UserPrincipalName -OwaMailboxPolicy OwaFullAccess }

Remember that synchronizing the two policies using PowerShell is a one-time operation that you need to repeat on a
regular basis to ensure that the policies stay synchronized.

Group Naming Policy

Exchange Online has a naming policy that applies to distribution groups. Office 365 Groups takes much the same
approach in its policy to control the display name generated for new groups. The feature is in “private preview” and
should reach general availability soon.

Two settings in the Groups policy control the form of display names users can give to groups when they create new
groups or update the display name of existing groups:

The PrefixSuffixNamingRequirement setting is a pattern controlling whether the group name has a prefix and
suffix. The pattern can include fixed strings or reference user attributes such as the department, company, or
office of the user who creates the group. Attribute values come from the information held in Azure Active
Directory for the user’s account and you can combine multiple attributes in a display name. Although you can
combine multiple attributes in a display name, it is best to favor simplicity over complexity when selecting what
Azure Active Directory attributes to use to construct a display name. The policy ignores values that are missing in
the directory. If the tenant uses a naming policy for Exchange distribution groups, it is wise to use different
prefixes for the two types of groups. Finally, the display name created for a group can be up to 256 characters. If
you tend to use very long names for groups, you must ensure that the application of prefixes and/or suffixes to
display names does not exceed the 256-character limit.

The CustomBlockedWordsList holds a set of words that users cannot include in group names. These might be
words that you consider offensive or have other reasons to exclude. Tenants often include business terms that
they do not want people to use in group names such as “CEO” to avoid the possibility that users send messages
to the wrong place. The check against blocked words is case-insensitive and you do not have to enter the blocked
words alphabetically. For practical purposes, you can enter as many words as you like in the list as Office 365
sets no upper limit.

The naming policy does not apply when administrators create groups. Office 365 assumes that these users know what
kind of display name to give to a group.

Here is an example of how to update the settings used for the group naming policy. In this case, we want to add the
prefix “O365Grp-“ to the names of newly-created groups (and existing groups when a group owner updates their
properties). We also specify a set of custom blocked words.

[PS] C:\> $Settings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings[“PrefixSuffixNamingRequirement”] = "O365Grp-[GroupName]"

[PS] C:\> $Settings[“CustomBlockedWordsList”] = "Sexy,Stupid,Giggles,Funny,CFO,CEO,Shit,Payroll"

[PS] C:\> Set-AzureADDirectorySetting -Id $Settings.Id -DirectorySetting $Settings

Microsoft has updated many of the applications that create Office 365 Groups, including Teams, Planner, Yammer, and
Stream to support the naming policy. All clients check the name entered by the user rather than the name as it will be
after application of the prefix and/or suffix dictated by the policy (what Microsoft refers to as the “decorated” name).
Another point to remember is that the check for blocked words is for complete words rather than substrings. This is to
avoid problems where a blocked word is part of another perfectly legitimate word (think “ass” as part of “class”).

Some differences exist in the detail of how a client applies the naming policy. In general, web-based clients like OWA
(Figure 12-3 ) and Teams include a preview of the group name after application of the policy together with warnings
when a user types in a blocked word for a group name. Other clients, like Outlook desktop and the Outlook mobile
clients, enforce the policy without giving so many visual clues as users type in names. Instead, these clients flag
errors when they check the policy before trying to create or edit a group.

Figure 12-3: OWA detects a blocked word in the name for a new group

Once a naming policy is in effect, clients apply the policy when users create new groups or edit the display name of an
existing group.

Updating Group Display Names with PowerShell

The naming policy is not retrospective and does not apply to existing groups, so if you want to bring those groups into
line with the policy, you must update the display name for older groups. The easiest solution is to use PowerShell to
make a one-time pass to find and update non-compliant group names. The example shown below is one way to
approach the problem. You can easily adapt the code to apply a naming standard to groups without using the AAD
policy.

[PS] C:\> Connect-AzureAD

$Policy = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

$NamingPolicy = $Policy["PrefixSuffixNamingRequirement"]


If (!($NamingPolicy))

{ Write-Host "No naming policy defined..."

EXIT }

Else

{ Write-Host "Office 365 Groups naming policy is" $NamingPolicy }

# Find the Prefix

$Prefix = $NamingPolicy.SubString(0,($NamingPolicy).IndexOf("[Group"))

$PrefixMatch = "*" + $Prefix + "*"

# Find Office 365 Groups that don't match the naming policy

$Groups = (Get-UnifiedGroup | ? {$_.DisplayName -NotLike $PrefixMatch})

If ($Groups.Count -gt 0)

{ $Prompt = "You have " + $Groups.Count + " groups to update. Proceed? [Y/N]"

$Answer = Read-host -Prompt $Prompt

If ($Answer.ToUpper() -ne "Y")

Write-Host "Exiting..."

EXIT }

# Update Groups

Write-Host "Updating Groups..."

ForEach ($G in $Groups)

{$NewDisplayName = $Prefix + $G.DisplayName

Write-Host $G.DisplayName "updated to" $NewDisplayName

#Set-UnifiedGroup -Identity $G.Alias -DisplayName $NewDisplayName

If you implement a group naming policy, you might end up with some groups having display names set by policy and
others named outside the policy guidelines. For example, if administrators create new groups or teams, the new
objects take the display name as entered by the administrator. All other groups or teams in the organization follow a
different convention, meaning that you do not achieve the goal of having a common naming scheme for groups. If this
happens, you can use code like that shown above to adjust the names of the non-compliant groups.

Usage Guidelines Policy Settings

Two settings in the Groups policy are available to help communicate the kind of information stored in groups and how
people should use groups.

The UsageGuidelinesUrl setting holds a pointer to a web page that users can consult to learn about
organizational guidelines governing Groups. For example, what content is allowed, who can create new groups,
and what to do when a group supports external access.
The GuestUsageGuidelinesUrl setting points to a web page holding information to guests about how they
should behave in Groups.
If you configure guideline settings, OWA displays links to the web pages in the Conversations and Files views. Teams
also shows the link to the usage guidelines page on the dialog used to create a new team. Here is an example of how
to update the usage guideline settings.

[PS] C:\> $Settings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq "Group.Unified"}

[PS] C:\> $Settings[“UsageGuidelinesUrl”] = "http://office365itpros.com/GroupGuidelines.html"

[PS] C:\> $Settings[“GuestUsageGuidelinesUrl”] =

"http://office365itpros.com/GuestUserGuidelines.html"

[PS] C:\> Set-AzureADDirectorySetting -Id $Settings.Id -DirectorySetting $Settings

Group Policy Settings in Exchange Organization Configuration

For convenience (mostly for OWA), Office 365 stores copies of some group policy settings in the Exchange Online
organization configuration. You can view these settings by running the Get-OrganizationConfig cmdlet:

[PS] C:\> Get-OrganizationConfig | Format-List Group*,Data*

GroupsCreationEnabled : False

GroupsCreationWhitelistedId : 12cb915b-2365-4bed-baf6-6257b3543273

GroupsUsageGuidelinesLink : Http://office365exchange.com/GroupGuidelines.html

GroupsNamingPolicy : O365Grp-[GroupName]

DataClassifications : <DataClassifications Default="Confidential"><Classifications><Classification Name="General Use"

Description="Anyone can access" /><Classification Name="External Access" Description="Available

outside the company" /><Classification Name="Internal Only" Description="Must not be shared with

external people" /><Classification Name="Confidential" Description="Can only be disclosed with

management permission" /></Classifications></DataClassifications>

The group settings in the organization configuration are read-only and cannot be updated with the Set-
OrganizationConfig cmdlet.

Guest Access for Office 365 Groups

The original version of Office 365 Groups did not support external access. Because many teams involve external
advisors, consultants, and experts, this limitation meant that Groups could not be an effective collaboration platform
in many scenarios. The solution adopted to permit external access to Office 365 Groups is based on Azure Active
Directory guest accounts . Generally, Azure Active Directory treats guests like tenant users and allows guests to join
the membership of groups. Office 365 Groups, Teams, SharePoint, and Planner support guest access but other group-
enabled applications like Yammer groups do not yet support the feature.

A guest is someone whose account and credentials exist outside the Office 365 tenant. Because these are Azure Active
Directory objects that share many of the characteristics of user objects, applications can use guests to control access
to content within a tenant. Azure Active Directory uses email addresses to create user principal names for guests. The
email address can point to another tenant domain within Office 365, an email domain for a company that doesn’t exist
within Office 365, or a consumer email service, like Gmail.com or Yahoo.com. You can view a list of guest accounts in
a tenant through the Users section of the Office 365 Admin Center, the Users section of the Azure Active Directory
portal, the Contacts section of the Recipients tab in EAC, or by running the command:

[PS] C:\> Get-Recipient -RecipientTypeDetails GuestMailUser | Format-Table DisplayName, Name

You can edit properties of guest users through the Office 365 Admin Center or Azure Active Directory portal. You can’t
through EAC because the list of guests displayed there is for informational purposes only.

The UserType property for guest accounts in Azure Active Directory is “Guest ,” so you can also use the Get-
AzureADUser cmdlet:

[PS] C:\> Get-AzureADUser -Filter "UserType eq 'Guest'" -All $True | Sort DisplayName | Format-Table DisplayName, UserPrincipalName

Any application that supports the Azure B2B Collaboration framework, including Office 365 Groups and Teams, can
use the guest accounts in the tenant directory to grant external access to their resources.

Azure B2B Collaboration

Office 365 uses Azure Active Directory as the foundation for user identity and authentication. External access is just
another way for a user to authenticate their right to use an application. Office 365 applications that support external
access leverage the B2B collaboration framework for Azure Active Directory to create and manage the guest
accounts. You can create guest accounts through the Azure portal with the Invite a Guest option, with PowerShell ,
or when an application issues an invitation to someone from outside the tenant to join an application, which what
happens when you add a guest to an Office 365 Group or team.

When Azure AD or an application adds a new guest, it calls the Invitation API to create and send an invitation via
email to the user’s address to tell them that they have been invited to start using an application in the tenant. In
addition to allowing applications to send customized invitations , the API supports the inclusion of a link (URL) for the
user to enter the application, like the access information needed for someone to join a group or team. Figure 12-4 is
an example of an invitation sent to a guest to invite them to join an Office 365 Group. The links in the invitation bring
the user to the group document library (Files) or create a new message addressed to the group (conversation). In fact,
the invitation message is more of a notification than anything else because the guest account already exists in the
group membership. The recipient can leave the group if they want or redeem the invitation to confirm that they wish
to continue.

Figure 12-4: An invitation sent by email to a guest to join a group

The guest account created when an invitation is generated allows the guest to join the group membership. At this
stage, the guest account is a placeholder. When the guest redeems the invitation, Azure Active Directory validates the
guest account and put it into a state where the account credentials can be used to access resources within the tenant.
It is important to realize that a guest account is tied to an email address or user principal name (ideally, the primary
email address and the UPN are the same). If a guest needs to change their email address, you must remove the guest
account tied to the old address and issue a new invitation to their new email address. Removing the old guest account
means that the guest loses access to all resources in the tenant, so you might have to add their new guest account to
multiple groups and teams afterwards.

The source for the guest is one of the account properties and tells Azure AD the source of the guest comes from and
how it will authenticate. You can see the source by looking at the profile of guest accounts in the Azure portal, where
accounts have one of three sources :

Users from other Office 365 domains have “External Azure Active Directory” as their source. These guests
sign into the Azure AD instance for their tenant and use those credentials access group resources.
Users of Microsoft consumer services such as Outlook.com have “Microsoft account” as their source. These
guests sign into their Microsoft (MSA) account and use those credentials to access group resources.
Users of other domains such as Gmail.com also have “Microsoft account” as the source. In this case, guests
sign into the Microsoft account created for them during the invitation redemption process and use those
credentials to access resources.
The source for the guest account for anyone who has not redeemed an invitation is “Invited User .” When the
redemption process completes, the source changes to one of the other sources listed above.
The tenant extending invitations to guests can require multi-factor authentication to force guests to use a mixture of
credentials: their own password and another authentication method (like a one-time code sent to the guest’s mobile
device).

All applications that use the Azure B2B collaboration framework generate and redeem invitations in the same manner.
The difference is the link contained in the invitation – it can point to an individual team, group, or other application
and the application referenced must be ready to allow access to that guest. SharePoint is an exception in that it
creates a guest user account in your tenant when an external user from another Office 365 tenant redeems a sharing
invitation to a document or folder with the usual one-time code mechanism. SharePoint does this to allow these users
to use their tenant credentials to open the shared documents with the Office desktop apps.

If you find that a guest cannot access a resource after redeeming their invitation, the easiest fix is usually to remove
their account from the group membership and then add them back. The user receives a new invitation and can
redeem it to complete the joining process.

DIY Invitations
You can reproduce what an application does when it generates an invitation to a guest with a series of PowerShell
commands. It is critical to understand that this is example code not intended for production use. Instead, it illustrates
some of the processing that occurs when a group owner invites a new guest to join an Office 365 Group.

First, we run the PowerShell New-AzureADMSInvitation cmdlet to generate and send an invitation to the email
address of the guest we want to add, in this case John.Doe@Outlook.com . If the user accepts the invitation, the
redirect brings them to the specified URL, which should point to the URL for the SharePoint Online team site
belonging to the group. You can get this URL by navigating to the Files section for the group from OWA. Copy the URL
from the browser and paste it into the command.

[PS] C:\> New-AzureADMSInvitation -InvitedUserDisplayName "John Doe" -InvitedUserEmailAddress John.Doe@Outlook.com -

SendInvitationMessage $True -InviteRedirectUrl https://office365itpros.sharepoint.com/sites/GroupName

When you generate an invitation to a guest, Azure AD checks whether an account exists for the user. If one exists, it is
used. If not, Azure AD creates a new guest account. You can check this by running the Get-AzureADUser cmdlet after
the invitation goes. The account has a UserType of “Guest” and the User Principal Name is based on the email
address used in the invitation. In this case, the UPN is
John.Doe_outlook.com#EXT#@office365itpros.onmicrosoft.com . The creation of a mail user is a side effect of
creating a new guest account. Because Exchange Online owns these objects, they exist in the Exchange directory
(EXODS). The new mail user object turns up in EXODS after a background synchronization process detects the new
guest account, between one and fifteen minutes after the creation of that account.

The mail user object and the email address for the guest allows Exchange Online to route email to the guest. It also
allows us to add the guest to the membership of the group with the Add-UnifiedGroupLinks cmdlet. Here we use the
alias assigned to the mail user object to point to the guest to add. As you can see, the alias is based on the UPN for
the guest account.

[PS] C:\> Add-UnifiedGroupLinks -Identity MyGroup -LinkType Member -Links John.Doe_outlook.com#EXT#

Guests need to receive copies of group conversations via email as they cannot access conversations in the group
mailbox. If the AutoSubscribe property for the group is $True , new members automatically join the subscriber list
when they become a member. If not, we need to add them as a subscriber.

[PS] C:\> Add-UnifiedGroupLinks -Identity MyGroup -LinkType Subscriber -Links John.Doe_outlook.com#EXT#

Alternatively, you can use the Add-AzureADGroupMember cmdlet to add the guest to the group. This approach has the
advantage that the write occurs direct to Azure AD and avoids the need to replicate from EXODS before SharePoint
learns about the new group member. On the other hand, a short interval elapses before EXODS learns about the new
member and adds them to its view of the group membership.

[PS] C:\> $NewUser = (Get-AzureADUser -ObjectId John.Doe_outlook.com#EXT#@office365itpros.onmicrosoft.com).ObjectId

[PS] C:\> $Grp = (Get-AzureADGroup -SearchString "MyGroup").id

[PS] C:\> Add-AzureADGroupMember -ObjectId $Grp -RefObjectId $NewUser

After they receive the invitation, the guest clicks the link to accept the invitation. If they do not already have a
validated guest account in the host tenant, the redemption process kicks in to complete the sign-up process. Once this
happens, the URL redirects them to the document library and because the guest account is a member of the group,
SharePoint grants access. It all sounds simple and these steps do work, but the dependency on directory replication
makes this approach to creating guests difficult to use in practice. For that reason, it is best to use the Office 365
applications to invite guests and have their accounts created through the redemption process.

Audit Records for Guest Additions

Office 365 captures details of the addition of a new guest account to a tenant in an “Add User ” audit record. Office
365 also captures audit records for “Add member to group ” events when a new member joins a group. Together,
these audit records give enough information to allow administrators to track and report when new guests join groups
in the tenant. To extract the information, use the Audit log search feature in the Security and Compliance Center, or
use the PowerShell Search-UnifiedAuditLog cmdlet. Several examples of how to use the cmdlet to extract, refine, and
analyze audit log data are included in Chapter 21.

Updating Properties for Guest Accounts

If you examine a new guest account, it is obvious that Azure AD populates few of its properties. It is a bare-bones
account that lacks information that might be useful to identify the guest, such as a display name that has a suffix with
the name of the guest’s company or contact information. You can add this information for guest by updating their
accounts through the Office 365 Admin Center. Go to the Users section of the Office 365 Admin Center and you will
find the guest included in the set of Active Users. You can then select and edit the Contact Information for a guest like
any other user. You can also edit guest user properties through the Users blade of the Azure AD portal.

What Guests Can Access in Office 365 Groups

Each application that supports the Azure B2B collaboration framework dictates the flow of processing that occurs
when someone accepts an invitation to access content managed by the application. Office 365 Groups leverage the
existing SharePoint external sharing mechanism for external access, which means that guests have good access to
group resources managed by SharePoint Online but only indirect access to those managed by Exchange Online.

Guests can share group resources in two ways. Like any other item stored in a SharePoint Online library, you can
grant a guest access to a specific file stored in a group document library. Alternatively, you can add the guest to the
membership of the group and so gain access to everything in the group document library. The dependency on
SharePoint Online means that guests do not have direct access to the Exchange Online resources and cannot open the
group shared mailbox to browse conversations or the group calendar. However, guests can take part in group
conversations via email by sending messages to the group and receiving copies of contributions made to
conversations by other members. Table 12-4 summarizes the functionality available to guests through external access
to Office 365 Groups.

Workload Feature Guests can access?

Exchange Contribute to conversations in group mailbox Yes (via email)

Online

Exchange View items in group shared calendar No direct access, but can
Online receive updates via email

Exchange Search group conversations Yes – but only copies held

Online in personal mailbox

Exchange Browse Global Address List for tenant No. (Note: Guests don’t
Online appear in any default
Exchange address list)

SharePoint Access a single document Yes – via specific

Online invitation

SharePoint Access a complete document library Yes

Online

SharePoint Search group documents Yes

Online

OneNote Access group shared notebook Yes

Azure Members can post encrypted conversations to group conversations but No (apart from
Information cannot access messages protected with other IRM templates. Members encryption)
Protection cannot read IRM-protected files in the document library.

Planner Access plan associated with group Yes

Teams Access team chats associated with group No (unless the user is
granted separate access
by Teams)

Power BI Access Power BI workspace associated with group No

Dynamics Access customer data associated with group No

365

Groups Browse directory for groups to join No

Groups Perform group management No

Table 12-4: Group functionality available to guests

Apart from the 2,500-member limitation for a group, there is no specific limit on the number of guests who can join a
group (or team). You can have a group with 1 local member (the owner) and 2,499 guests.

Restricting Guest Access

The basic principle of the Groups membership model is that every member enjoys the same access rights to resources
belonging to the group such as the document library. Owners exist to maintain group membership and settings, but
they still have the same access to group resources as other members. The model is simple and easy to understand and
manage, but some potential downsides exist that tenant administrators need to understand.

Consider files stored in the SharePoint document library provisioned for every Office 365 group. When a guest joins
the membership of a group, they receive the same access rights to documents and other files held in the group
document library. They can view, print, add, edit, check-out, and perform all the file management actions available to
other group members, including the ability to remove items. Two issues might occur:

1. Guests will remove information from libraries (the deleted items will end up in the site recycle bin and an
administrator can recover the file if necessary).
2. Guests will be able to access confidential material.

One solution is to create special groups (or teams) to share information with guests and keep all confidential
information separately. For instance, let’s assume a project team needs some advice about a complex contract
document that will be provided by an external legal advisor. Is it better to allow the advisor full access to the group
where the contract is stored along with other project files or to create a specially-defined group that holds just the
contract document? An argument can be made that the latter is the right approach in the circumstances, but an
equally valid case exists that the advisor can give better advice if all the supporting information is made available to
them.

If you store confidential material alongside other files in the document library, can you make sure that guests cannot
access this content? The answer is yes, but you must apply rights management through Azure Information Protection
templates to the files you want to protect from guests. As explained in Chapter 24, a protection template specifies the
rights that users have over content. Users must be able to authenticate their access before Azure Information
Protection will decrypt and expose the content. Authors have full rights while other users might only be able to read
the material. If guest users are not included in the set of users and groups assigned permission in a template, they
cannot access content protected by that template.

Metadata are not encrypted for protected documents, so guest users will know that these files are in a document
library. However, unless they are included in a template, guest users will not have the rights to open files, so they will
not be able to access the content, even if they download files to their PC. Even better, if a protected document is
inadvertently circulated outside the tenant, anyone who receives a copy will be unable to open the file.

In all cases, the key thing is to understand why guests need access to a group or team and what resources they will
have once they join the group before issuing any invitations.
Adding Guests to an Office 365 Group
You can use OWA, Outlook 2016, or the Outlook and Teams mobile apps to add guest to a group. A group owner or a
tenant administrator can add a guest while other group members can request an owner to add a guest. In either case,
when you add a guest to a group, Office 365 generates an invitation to the guest to join the group. To start, open the
group and click the ellipsis menu to the right and select Members to display the complete group membership,
including any guests. Now click the Add Members icon and enter the email address of the guest you want to add. If
the group settings allow guest access, Office 365 checks the email address as you enter it against the directory and
your Outlook autocomplete list (Figure 12-5 ).

Figure 12-5: Adding a guest to an Office 365 group with OWA

Office 365 does not restrict you to using addresses already known to the directory or Outlook as any valid email
address is acceptable. If you input the email address of an account belonging to the tenant, they join the group as a
regular member instead of a guest. Office 365 adds the guest to the group’s membership and generates the email
invitation when you click Save .

If group settings (for the tenant or for the specific group) block guest access, OWA displays an error to say that you
must contact an administrator to add a guest. OWA also flags an error if you try to add an email address for a guest
that already exists for an unsupported object (like an email-enabled public folder).

Guests and the Office 365 Admin Center : You can edit the membership of a group via the Office 365 Admin
Center and can add a guest to a group’s membership there, providing that the guest account already exists in the
directory. You can create new guest accounts in the Azure portal or with an application that supports external access,
such as Office 365 Groups or Microsoft Teams.

How Guests Join an Office 365 Group

When a guest joins the membership of a group, Groups checks Azure Active Directory to confirm whether a guest
account for the email address exists in the tenant. If not, AAD creates a new guest account. An invitation then goes to
the guest to let them know that they are now a member of the group. Should the guest wish to decline the invitation,
the message has a link to allow the guest to leave the group. If not, they can click the link to access the group shared
document library and confirm their membership.

If a guest does not receive (or loses) the email notification to tell them about their new membership status, the group
owner can send them the URL for the document library. The guest can input the URL into a private browser session
and go through the sign-in process to connect to the group. The private session makes sure that existing cached
credentials don’t interfere with the process. The URL for a group document library is in this form:

https://mytenant.sharepoint.com/sites/groupalias/

Another approach is to remove the user from the group and then add them again to force Office 365 regenerate the
invitation. This is perhaps the easiest way to solve the problem of lost invitations.

Some guests who receive invitations to join a group find that they cannot log into Office 365 to access the group using
the email address specified in the invitation. This might be because they have the same address registered in Office
365 (as a user in another tenant) and for a Microsoft consumer service, like OneDrive personal. In this case, the
solution is to either use a different address to invite the guest or ask them to rename their personal Microsoft
consumer account .
A guest can also receive an invitation to a file held in a group document library through the action of sharing a
document belonging to the group as a “cloudy attachment” (one where the attachment is a link to the content held in
a OneDrive for Business or SharePoint Online library rather than a complete copy of the document). Clicking the
attachments invokes the invitation to access the document.

If the guest chooses to accept the invitation, two conditions can occur. First, they might already have a Microsoft
(MSA) or Office 365 account linked to the email address used for the invitation (including a consumer account for a
service like Outlook.com). If so, they can use this account to authenticate with Office 365 and access the group files.

If they do not have a Microsoft account, a redirect occurs to invitations.microsoft.com and the user can sign-up for a
Microsoft account, including creating a password, which in turn becomes the guest account in the tenant. Creating a
fully-functional guest account needs the guest to prove that they own the email address associated with the account.
Office 365 sends a verification code to the email address specified by the person extending the invitation. When
prompted, the user gives the code to authenticate their address and complete the activation process, the guest
account can then access the group.

Finding group files : Guests who have accounts in other Office 365 tenants cannot add external groups to their set
of favorite groups. Instead, to ensure fast access, they should bookmark the link contained in the invitation message
as a favorite in their browser. Clicking the bookmark will then bring the guest to the Files section of the group.

Guest Access to Group Resources

When guests access a group document library, they see a modified version of Files (for instance, OWA shows no other
groups in the navigation pane, even if the guest has access to other groups in the tenant). All the files and folders in
the document library and the shared notebook are accessible. In addition, if group members circulate “cloudy
attachments” to the group, guests can use their credentials to see those files.

OWA also limits guests to the amount of information they can see about the tenant and other group members. For
instance, modified Exchange address lists restrict their access to the GAL and the contact cards revealed for group
members don’t hold organizational information (like job titles). In addition, guests do not appear in other Exchange
address lists. Depending on customer feedback as to whether this feature is desirable, Microsoft might add the ability
for an administrator to configure address lists to include guests in the future.

Figure 12-6: OWA lists a guest among other members of an Office 365 group

Visual hints show group members that guests are present. When viewing group membership or conversations, the
group type appears as “Public group with guests” (Figure 12-6 ) or “Private group with guests”. OWA also displays
MailTips to warn users whenever they post new conversations. These visual indications serve as a warning that
members should not share or discuss confidential material within the group. Unsurprisingly, you cannot promote a
guest to become the owner of a group.

Apart from email interaction with group conversations, guest can only use browsers to access group resources as
Outlook 2016 does support guest access to groups.

Guests and Group Conversations

Guest cannot browse the conversations held in the group mailbox because they do not have access to the mailbox. For
the same reason, they can’t access the group calendar. Instead, guests take part in conversations via email by being
auto-subscribed to the group, much in the same way as other external users can take part in conversations if the
RequireSenderAuthenticationEnabled property for the group is False . The RequireSenderAuthenticationEnabled
setting does not govern guest members because they are full members of the group.

The exception to this rule is where the primary SMTP address of a guest is different to the email address used for
their invitation. When this is true, any attempt to respond to a conversation will fail because the email address on the
inbound item does not match any the address of a group member. An easy workaround is to set
RequireSenderAuthenticationEnabled to False , which allows any external user to email the group, Alternatively, you
can create a guest to allow the person to log in using their normal UPN and send messages to the group using their
primary SMTP address.

Although they can’t access the group calendar, guests receive calendar invitations sent to the group and can add
these invitations to their personal calendar.

Controlling Guest Access to Protected Information

You can protect confidential information held in group document libraries by applying a protection template to the
items. Guests are not able to access protected items because they cannot get a license to access the content. In fact,
this can be an advantage because you can then protect ultra-confidential information with Azure Information
Protection and allow guests to access the other files in the document library. Although they won’t be able to see the
content of protected documents, guests can see the metadata like title, author and last modified date.

Updating Guest Accounts

New guest accounts are now supported by many Office 365 applications. After an application creates a new guest
account in Azure Active Directory, AAD synchronizes it to the workload directories. A guest account can be a member
of multiple groups.

Like tenant accounts created through the Office 365 Admin Center, the creation process makes sure that guest
accounts get a skeleton set of attributes necessary for the accounts to work. If you want your directory to be well-
populated with job titles, locations, and phone numbers, you must add this information to accounts. Often
administrators forget to update guest accounts, and the accounts stay in a usable but threadbare state. For example,
no matter what application invites a guest, their default display name is their email address. This can make it hard to
understand who a guest is, especially when they use accounts from consumer email systems with addresses like
LazyBoy5014@gmail.com . The display name and other details like the job title, phone numbers, and address can be
changed through the Office 365 Admin Center (edit the account’s Contact Information ) or the Azure Active
Directory portal.

Guest Photos
Given the collaborative nature of Office 365 Groups and Teams, it makes sense to update guest accounts with suitable
photos. It’s much better when people get a visual reminder of whom they work with. Planner does not currently
display photos for guest accounts. OWA, SharePoint, Teams, and OneDrive do. For example, the left-hand screen in
Figure 12-7 shows a conversation within Teams where some guest accounts have photos and one does not. The
anonymous “VM” circle beside Vasil’s contribution does not negate the value of his comments, but it’s just nicer to get
a visual reminder of who’s saying what.

Figure 12-7: Adding photos for guest accounts

Tenant users can upload photos to their own accounts through Office 365 settings or Delve profile and the picture
data is then stored in their mailboxes. Guests don’t have mailboxes and administrators can’t upload a photo to a user
or guest account through the Office 365 Admin Center, but you can through the Azure Active Directory portal. Go to
the Users section, select the guest account, and upload a JPEG or PNG file. The file must be less than 100 KB. If users
can’t give you a suitable photo, you might be able to use their profile picture from their LinkedIn.com account, as
these photos are correctly sized and well under the 100 KB limit. If you must downsize a large picture, a 100 x 80-
pixel size usually works. To have a better-populated directory, it is a good idea to update other information about the
guest into Azure Active Directory at the same time, such as their first name, last name, display name, job title,
company name, and contact details. Office 365 apps can then show this information in contact cards.

After uploading, the background jobs responsible for synchronizing Azure Active Directory with the workload-specific
directories will make the changes available to applications. This process might take up to a day to complete. To force
Teams to synchronize, always update another attribute of the account, like the display name. This “tickles” Teams to
let the app know that it should synchronize the account. Browser clients pick up newly-available photos first. Mobile
and desktop clients must synchronize their local caches with the workload directory. This can add another day or so
before photos show up.

Managing Guest Account Properties with PowerShell

You can view details of guest accounts through PowerShell by running the Get-AzureADUser cmdlet. Here are three
examples:

1. You can pass the user principal name for the guest account. The format of the user principal name assigned to a
guest account is username_domainname#EXT#@servicedomain .
2. You can pass a partial name of a guest (in the example below, their email account) and search Azure Active
Directory.
3. You can look for all guest accounts known in Azure Active Directory for the tenant by looking for accounts with
the UserType property set to “Guest”.

[PS] C:\> Get-AzureADUser -ObjectId flayosc_yandex.com#EXT#@office365itpros.onmicrosoft.com | Format-Table UserPrincipalName,

DisplayName

UserPrincipalName DisplayName

----------------- -----------

flayosc_yandex.com#EXT#@office365itpros.onmicrosoft.com Flayosc Worker

[PS] C:\> Get-AzureADUser -SearchString Flowers@Outlook.com | Format-Table UserPrincipalName, DisplayName

UserPrincipalName DisplayName

----------------- -----------

Flowers_outlook.com#EXT#@office365itpros.onmicrosoft.com Flower Guru

[PS] C:\> Get-AzureADUser -Filter "Usertype eq 'Guest'” -All $True | Format-Table DisplayName, ProxyAddresses

You can use the Set-AzureADUser cmdlet to set properties of a guest account. For example, here is how to change the
display name:

[PS] C:\> $User = Get-AzureADUser -SearchString "Van20143"

[PS] C:\> Set-AzureADUser -ObjectId $User.ObjectId -DisplayName "Michael Van Hybrid"

Other attributes of the Azure Active Directory object for a guest account include:

ObjectType : User.
AccountEnabled : True.
CreationType : Invitation. You cannot change this.
Mail : The SMTP email address for the guest.
MailNickName : A modified version of the email address to show that this is a guest. For example,
Flowers_outlook_com#EXT# . This attribute is a unique alias for the account.
OtherMails : A multi-valued attribute used for different purposes for different sorts of accounts (for instance,
accounts enabled for MFA store their “rescue” email address in this attribute). Guest accounts hold their SMTP
email address in this attribute. SharePoint Online uses the attribute to control external file sharing, so if you
remove this attribute for an account, you remove the ability for the user to access any files shared with them.
UserPrincipalName: Used in the same manner as the UPN for other Azure AD accounts.
UserType: Guest.

If you use the Get-UnifiedGroupLinks cmdlet to examine the membership list for a group that has guests, you’ll see
that the name assigned to guests is based on the first part of their user principal name.

[PS] C:\> Get-UnifiedGroupLinks -Identity BRK3001 -LinkType Members

Name RecipientType

---- -------------

TRedmond UserMailbox

amitgutta.microsoft.com#EXT# MailUser
chris.fiessinger_microsoft.com#EXT# MailUser

e.zenz_microsoft.com#EXT# MailUser

benny.niaulin_share-gate.com#EXT# MailUser

t.redmond_live.com#EXT# MailUser

flowers_outlook.com#EXT# MailUser

In a hybrid environment where group membership synchronizes with an on-premises Active Directory via
AADConnect, the synchronization process excludes guests because guest accounts do not exist in on-premises
directories.

Resetting Membership for Guests

Occasionally the creation process for a guest account does not complete as planned. The guest account exists in Azure
Active Directory and the guest shows up in a group’s membership list, but whenever the guest attempts to access the
group files, they receive an error telling them that their account is not in the tenant directory.

The quickest and simplest solution is to recreate the guest account. To do this, you must remove the account through
the Office 365 Admin Center or with PowerShell and then erase the deleted object from the Azure Active Directory
recycle bin. If you leave the deleted object in the recycle bin, it lingers there for 30 days and any attempt in that time
to reuse the account will restore the old flawed object. Here’s how to do the job with PowerShell. First, remove the
account (you can also remove the account using the Office 365 Admin Center):

[ PS] C:\> Remove-AzureADUser -ObjectId t.redmond_live.com#EXT#@Office365ITPros.onmicrosoft.com

The object is now in the AAD recycle bin. Now, remove the deleted object. You can’t do this through the Office 365
Admin Center, but you can through the Users section of the Azure Active Directory portal. Select the Deleted users
view, then the specific object, and then click the Delete permanently button. Alternatively, run this PowerShell
command.

[PS] C:\> Remove-MsolUser -UserPrincipalName t.redmond_live.com#EXT#@Office365ITPros.onmicrosoft.com -RemoveFromRecycleBin

Finally, go back to a client and re-add the guest to force the creation of a new guest account. Everything should now
work! The big downside with this method is that permanently removing a guest account also removes the membership
for that user to all Office 365 Groups and Teams and removes sharing permissions that the user might have in
SharePoint and OneDrive for Business sites. For this reason, you should not remove a guest account without first
thinking about the potential consequences.

Mail Contacts and Guest Accounts

A basic rule of email transport is that an addressable object must have a unique email address. In other words, you
cannot have two objects in a directory that share the same email address. Guest accounts are an exception because
they can share an email address with a mail contact. If you add a new guest account and give the same email address
as a mail contact, Office 365 creates the guest account based on the properties of the mail contact. You cannot do the
reverse and create a mail contact using the email address of an existing guest account.

The need to support two objects with the same email address is explained because each serves a different purpose. A
mail contact exists to allow user to send email to external contacts, individually or through a distribution group. Guest
accounts are also addressable for email, can appear in address lists (see below), and have the added ability to access
application resources in a tenant.

Guests in Exchange Address Lists

By default, the guest accounts created by applications like Office 365 Groups and Teams do not appear in Exchange
Online and SharePoint Online. Guests are registered as valid email recipients and show up as guest mail users. You
can see the list of guest accounts known to Exchange Online with the command:

[PS] C:\> Get-Recipient -RecipientTypeDetails GuestMailUser | Format-Table DisplayName, HiddenFromAddressListsEnabled,

PrimarySmtpAddress

DisplayName HiddenFromAddressListsEnabled PrimarySmtpAddress

----------- ----------------------------- ------------------

Chris Burger True Chris@burger.com

Jon Vickers False flayosc32@outlook.com

Benjamin N Smith True benjamin.n.smith@contoso.com

James Tracey True

We can see the guests that appear in Exchange address lists (HiddenFromAddressListsEnabled is $True) and guests
who have not yet accepted an invitation to join the tenant (its PrimarySmtpAddress is blank). If you want an account
to show up in Exchange address lists (to allow users to email the guest), you update its ShowInAddressList property.
For example:

[PS] C:\> Set-AzureADUser -ObjectId stale.hansen_cloudway.no#EXT#@Office365itpros.onmicrosoft.com

-ShowInAddressList $True

The guest account will appear in Exchange address lists after the account update synchronizes from Azure Active
Directory to Exchange Online. This process usually takes no more than a couple of minutes. The guest account will
appear in the offline GAL the next time Exchange Online generates OAB updates and Outlook clients download and
process those updates.

Guests in SharePoint and OneDrive for Business Pickers

If you want guests to show up in the people pickers used by SharePoint Online and OneDrive for Business, you can
run the Set-SPOTenant cmdlet to update behavior for the tenant or Set-SPOSite for a specific site. For example:

[PS] C:\> Set-SPOTenant -ShowPeoplePickerSuggestionsForGuestUsers $True

EAC Support for Guest Accounts

Some parts of Exchange Online support guest users better than others. For example, EAC lists guest accounts in its
contact section with the restriction that you can’t edit a guest account through EAC. Another example is that you
cannot add a guest account as a member of a distribution group through the GUI because the people pickers do not
recognize guest account, but you can add a guest to a distribution group with PowerShell. The command shown below
adds a guest account to the “External Suppliers:” distribution group. Exchange recognizes the form of the address
used here because it is the alias (MailNickname ) of the guest, which has a recipient type of “MailUser” and is
therefore able to receive email. You can also pass the user principal name or the display name of the guest to add
them to a distribution group.

[PS] C:\> Add-DistributionGroupMember -Identity “External Suppliers List”

-Member ExternalPerson_outlook.com#EXT#

Taking everything into account, you can replace mail contacts with guest accounts if you accept that you’ll meet some
limitations in EAC.

Controlling Guest Access to Groups

Four distinct pieces come together to control guest access to Office 365 Groups:

1. Azure Active Directory must allow group owners to generate invitations to external users to join groups.
2. The settings for Office 365 Groups in the Office 365 Admin Center must allow people outside the organization to
access group content (see below).
3. SharePoint Online must allow sharing for content stored in SharePoint and OneDrive for Business sites with
external users.
4. The AAD policy for Office 365 Groups must allow guest access to groups.

As discussed in Chapter 13, the first three settings must be in place before Teams support external access.

Because guest access for Office 365 Groups primarily focuses on group document libraries, it follows that a
prerequisite for sharing to occur is that SharePoint external sharing must be enabled for the tenant. This setting is
available in the Sharing section of the SharePoint Admin Centre (Chapter 8). If a tenant does not allow sharing, guest
access to Office 365 Groups cannot work. SharePoint Online allows you to restrict sharing with users in specific
domains (a whitelist). However, Office 365 ignores this whitelist when adding guests to groups. If you want to prevent
guests from domains outside the whitelist becoming members of groups, you should implement the block policy
available for groups and conduct a periodic check of group membership. We will get to these topics shortly.

The Azure Active Directory settings for the tenant must also allow invitations to go to guests. For instance, if you
disable the “Members can Invite” setting in the User Settings section for Azure Active Directory in the Azure portal,
group owners cannot send invitations to people outside the organization, meaning that guests will never be able to
redeem the invitations and join groups.

Tenants control guest access to Office 365 Groups through the Settings section of the Office 365 Admin Center.
Select Services & Add-ins and then Office 365 Groups to view the current settings ( Figure 12-8 ). You can turn off
the ability for guests to access Office 365 Groups, in which case existing guests will lose any access they might have
to groups. You can also control whether group owners can add guests to groups. This setting only affects group
owners as administrators will still be able to add guests to groups.
Figure 12-8: Office 365 Admin Center settings for guest access to Groups

Optionally, you can use PowerShell to update the settings for a group to allow or deny guest access to that group. We
discuss how to do this later in this chapter.

Using the Groups policy to Control Guest Access

The AAD policy for Office 365 Groups includes two settings to control guest access to groups. The policy applies to all
applications that support guest access to groups. The policy settings are independent to the sharing controls for
SharePoint Online, which means that you can disable guest access to groups while keeping the ability for users to
issue invitations to external users to share specific items in SharePoint libraries and lists.

The AllowToAddGuests setting in the Groups policy controls whether group owners can add guest user accounts to the
membership of Office 365 Groups. By default, the value of the setting is True , meaning that group owners can add
guests to any group in the tenant that is not blocked at a group level (we’ll discuss how to block guest access to
individual groups shortly). You can use the Get-AzureADDirectorySetting cmdlet to examine the relevant policy
settings:

[PS] C:\> (Get-AzureADDirectorySetting).Values | ?{$_.Name -Like "*Guest*"}

Name Value

---- -----

AllowGuestsToAccessGroups True

AllowToAddGuests True

We can see that the AllowToAddGuests setting is True . We can also see that AllowGueststoAccessGroups is True ,
meaning that external users with guest accounts in the tenant can access resources through their membership of
Office 365 Groups.

Stopping Group Owners Adding Guests to Groups

To change one of these settings, you update the AAD directory setting object. For example, these commands update
the AllowToAddGuests setting in the Groups policy to False to prevent group owners adding any more guests. The
block on adding guest users applies even to groups whose individual AllowToAddGuests policy setting is $True . In
other words, in this instance, the organization-wide setting trumps an individual group setting.

[PS] C:\> $PolicySettings = Get-AzureADDirectorySetting | ? {$_.DisplayName -eq “Group.Unified”}

[PS] C:\> $PolicySettings["AllowToAddGuests"] = "False"

[PS] C:\> Set-AzureADDirectorySetting -Id $PolicySettings.Id -DirectorySetting $PolicySettings

Administrators can still add guests to groups after updating the policy to block group owners. However, guests can
only be added using an administrative interface:
 PowerShell: By running Add-UnifiedGroupLinks , Add-TeamUser , or Add-AzureADGroupMember .
Consoles: Through the Office 365 Admin Center, the Azure Active Directory Portal, or the Teams and Skype for
Business Online Admin Center.

Administrators can add existing guest accounts to groups without any further step. However, if you need to add a new
guest user to a group, you must first create the guest account through the Azure Active Directory portal before you
can add it to a group. The new guest account can then be added to a group, but the guest won’t be able to access any
resources until they redeem their invitation to join the tenant.

Turning Off Guest Access

If you update the AllowGueststoAccessGroups setting in the groups policy to False , all guests lose access to the
groups to which they belong after a brief period to allow cached directory settings to refresh. Think of this as “flipping
a switch” to turn access on or off if the need suddenly arises to limit sharing for a tenant. You can restore access later
by updating the Groups policy. Guests will continue to have access to individual files for which they have received
sharing invitations.

Blocking Guest Access for Selected Groups

When a group owner tries to add a new guest, Groups checks the settings of the target group to find out whether
guests are allowed. If the target group does not have a specific setting to control guest access, Groups checks the
general policy. By changing the AllowToAddGuests setting for specific groups, you control whether guests are allowed
or blocked for these groups. However, note that if a tenant blocks guest access to groups in the general Groups policy,
you cannot allow group access to individual groups because the general tenant-wide prohibition has priority.

Unlike other group properties, you do not use the Set-UnifiedGroup cmdlet to block external access to a group.
Instead, the Get-AzureADDirectorySetting and Set-AzureADDirectorySetting cmdlets retrieve and update group
settings. The process is very like the one used to manipulate a policy setting in the general policy. The first step is to
retrieve the object identifier for a group and store it in a variable. We can do this by running the Get-UnifiedGroup
cmdlet to retrieve the ExternalDirectoryObjectId property for the group. The object identifier is a GUID like 84fe8ec1-
d85f-4564-a786-1f7bedd9862c used to find and update the group object within Azure Active Directory. For example:

[PS] C:\> $ObjectId = (Get-UnifiedGroup -Identity “Sales Professionals”).ExternalDirectoryObjectId

Now, check that a settings object exists for the group:

[PS] C:> $Settings = Get-AzureADObjectSetting -TargetType Groups -TargetObjectId $ObjectId

If the $Settings variable does not hold a value after this command completes, you know that the group does not yet
have a settings object. This is fine because it simply means that the group uses the default policy settings for the
tenant. Before we can set a block at group level, we must create a settings object for the group by running the
commands shown below. In this instance, the value passed to the TemplateId parameter points to the AAD
“Group.Unified.Guest” template, which is used for settings applied to a specific group (the GUID is different to the
template used for the AAD group policy). The $ObjectId variable points to the GUID for the target group, which we
retrieve using the Get-UnifiedGroup cmdlet as shown above.

[PS] C:\> $Template = Get-AzureADDirectorySettingTemplate | ? {$_.DisplayName -eq "Group.Unified.Guest"}

[PS] C:\> $Setting = $Template.CreateDirectorySetting()

[PS] C:\> $Setting["AllowToAddGuests"] = $False

[PS] C:\> New-AzureADObjectSetting -TargetType Groups -TargetObjectId $Objectid -DirectorySetting

$Setting

Once the block is in place, any attempt to add a guest via OWA or Outlook results in the message:

“Before you can add guests as members of the group, you need to contact your administrator. ”

Or:

“You can only add individual people who are inside your organization”

Existing guests remain in place after you block adding new guests. If you want to purge all external access to the
group, you must remove existing guests from the group membership after you set the block.

To reset a group to allow group owners to add guests, you reverse the process and either set the value for the
AllowToAddGuests setting to $True and then update the settings object again or remove the policy from the group.
This code snipper uses the Get-AzureADObjectSetting cmdlet to read the values from the group and the Set-
AzureADObjectSetting cmdlet to update the settings.

[PS] C:\> $GroupSettings = Get-AzureADObjectSetting -TargetType Groups -TargetObjectId $ObjectId

[PS] C:\> $GroupSettings["AllowToAddGuests"] = $True

[PS] C:\> Set-AzureADObjectSetting -Id $GroupSettings.Id -DirectorySetting $GroupSettings

-TargetObjectId $ObjectId -TargetType Groups

If necessary, an administrator can always add a guest to a group with PowerShell or the Office 365 Admin Center,
even if the group is blocked for external access. The exception is for groups that have dynamic membership where
Azure Active Directory calculates the group membership by running a query against the directory. You must adjust the
query to make changes to the membership of a dynamic group as you cannot add an individual member. Here is an
example of a command to add a guest to a group with PowerShell. For this command to work, the guest account must
already exist in the tenant directory.

[PS] C:\> Add-UnifiedGroupLinks -Identity "Confidential Group" -LinkType Member -Links

SomeUser_outlook.com#EXT#

See Chapter 14 for information about how to use a script to find groups matching a certain classification and block
guest access to those groups

Azure B2B Collaboration Policy

Office 365 tenants can deploy an Azure Active Directory policy called B2BManagementPolicy to manage the domains
that guest users can come from. To access the policy, go to the Organizational Relationships section in the Azure
Active Directory portal, select Settings , and then to the section called Collaboration restrictions . Three values
are available:

Allow invitations to be sent to any domain (most inclusive): This is the default setting and it means that
invitations to join Office 365 Groups and Teams can go to users in any other domain.
Deny invitations to the specified domains : You can create a policy to block invitations going to specific
domains. Microsoft believes that using a deny list is the most common scenario as most organizations know the
domains that they do not wish to share information with. For example, you might decide to block guest users with
consumer email addresses, and block domains like Gmail.com, Yandex.com, Outlook.com, Yahoo.com, and so on.
Including the domains of competitors in a deny list is also sensible. When a deny list is in place, Office 365 blocks
any attempt to invite someone to join a group or team if the invitee has an email address in one of the blocked
domains.
Allow invitations only to the specified domains (most restrictive): You can create a policy with an allow
list, meaning that invitations can only go to domains in the list. Office 365 blocks any attempt to generate
invitations to any other domain not on the list. This choice is greyed out because Microsoft has not completed
development.

To create a policy with a deny list, click the deny list button and enter the domains to include and then Save (Figure
12-9 ). You can add up to 60 domains to a block or allow list. After you populate a list, the policy applies to all
applications that use Azure B2B collaboration for external sharing, including Office 365 Groups and Teams.

Figure 12-9: Adding domains to the deny list in the Azure B2B collaboration policy

A tenant can only have a deny or an allow list. If you think that an allow list will work better than a deny list that is
already in place, you must remove the current deny list and then create the allow list. The external collaboration list is
separate to a similar list to restrict sharing for OneDrive for Business and SharePoint Online. One list blocks
invitations to join groups or teams, the other blocks invitation to share documents or folders.

If you create a deny or allow list to control access to certain domains, you should synchronize the domains with the
SharePoint Online restricted domains setting . Failure to do this can lead to inconsistencies. For example, a guest user
might discover that they can view conversations in a team but cannot access the SharePoint document library.

Using PowerShell to Manage Deny or Allow Lists

The PowerShell module for Azure Active Directory (version 2.0.0.98 or later) includes cmdlets to create and manage
an AAD policy to implement block and allow lists to control the domains to which guests belong. For example, to see
details of the policy created above, use the Get-AzureADPolicy cmdlet:

[PS] C:\> Get-AzureADPolicy | ? {$_.DisplayName -eq "B2BManagementPolicy" } | Format-List

Id : 14c3fc40-25fd-4f32-a0f0-c3fdd22f7de8

OdataType :

AlternativeIdentifier :

Definition : {{"B2BManagementPolicy":{"InvitationsAllowedAndBlockedDomainsPolicy":

{"BlockedDomains" :["Yahoo.com","Gmail.com","Google.com"]}}}}

DisplayName : B2BManagementPolicy

IsOrganizationDefault : True

KeyCredentials : {}

Type : B2BManagementPolicy

You can also manipulate policy settings with PowerShell, as in this example of using the Set-AzureADPolicy cmdlet.
However, it can be tricky to construct the “ stringfied JSON ” required to pass the new policy rules, so it is usually
easier to work with deny or allow lists through the Azure Active Directory portal.

[PS] C:\> $Policy = (Get-AzureADPolicy | ? {$_.DisplayName -eq "B2BManagementPolicy" })

If ($Policy.Id -ne $Null) {

$PolicySettings = @("{`"B2BManagementPolicy`":{`"InvitationsAllowedAndBlockedDomainsPolicy`":{`"AllowedDomains`":
[],`"BlockedDomains`": [`"live.com`",`"yandex.com`"]}}}")

Set-AzureADPolicy -Id $Policy.id -Definition $PolicySetting

Adding Guests Programmatically

Tenant administrators can create guest accounts through Azure B2B connections , which is a good way to set up
sharing between two organizations that work together on a regular basis. Tenant administrators can also create guest
accounts through the Azure Active Directory portal. Finally, group owners can add a guest account by adding a guest
as a member of a group. Once a guest account exists in the tenant, the administrator can add that guest to any group
via the Office 365 Admin Center, EAC, or by running the Add-UnifiedGroupLinks cmdlet. For example:

[PS] C:\> Add-UnifiedGroupLinks -Identity ‘Engineering Testers’ -LinkType Members -Links flowers_outlook.com#EXT#

If you have mail contacts, you can also use these objects to create a guest account and add that account to the
membership of a group. To allow mail contacts (which appear in address lists) and guests (which do not) share the
same email address, Groups supports an exception to the rule that usually ensures that every mail-enabled object in a
tenant must have a unique email address. To add a mail contact as a guest to an Office 365 group, we run the Add-
UnifiedGroupLinks cmdlet and pass the email address of the contact. For example:

[PS] C:\> Add-UnifiedGroupLinks -Identity GroupName -LinkType Members -Links

(Get-MailContact -Identity MailContact).WindowsEmailAddress

You can also use the Add-TeamUser and Add-AzureADGroupMember cmdlets to add guests to Office 365 Groups. For
example, here’s how to add a guest user to an Office 365 Group with Add-AzureADGroupMember . The need to pass
object identifiers for the target group and the source guest user account makes the code a little more complex than
running Add-UnifiedGroupLinks .

[PS] C:\> Add-AzureADGroupMember -ObjectId (Get-UnifiedGroup -Identity "GDPR Planning Mark II").ExternalDirectoryObjectId -
RefObjectId (Get-AzureADUser -ObjectId JackSmith_outlook.com#EXT#office365foritpros.onmicrosoft.com).ObjectId

Cleaning up Guest Accounts

People are often added as guests to a tenant for a specific reason that expires in time. For instance, you might ask
someone to join a team or group to give expert external advice for a project. After the project completes, no good
reason exists for that person to continue having a guest account in your tenant. You can, of course, leave the guest
account in place on the basis that it might be needed in the future, but usually it is a better idea to clean things up
and remove these accounts when they are no longer used, especially because these accounts have access to tenant
resources.

Several ways exist to control the longevity of guest accounts in a tenant:

You can use the PowerShell script in Chapter 14 (see “Removing Unwanted Guests”) to find groups with external
members and then check the individual members of each group.
You can use Azure AD Access Reviews to force group owners to check the memberships of the groups they
manage and attest that each member should keep or lose their status. Azure AD Access Reviews is a feature that
requires Azure Active Directory P2 premium or Enterprise Security and Mobility E5 licenses.
You can check guest accounts based on their activity level. Chapter 21 describes how to use the Office 365 audit
log and email tracking logs to calculate the last activity for guest users. If a guest hasn’t been active for months,
perhaps it is time to remove their account.

After target accounts are identified for removal, you can ask group owners to remove them or do so programmatically
tor through an administrative portal. For example, here’s an example of using the Remove-UnifiedGroupLinks cmdlet
to remove a member, If an account is a group owner, remember to remove it from the owner list first.

[PS] C:\> Remove-UnifiedGroupLinks -Identity "GDPR Planning Mark II" -LinkType Member -Links jacksmith_outlook.com#EXT#

To remove a guest account completely, run the Remove-AzureADUser cmdlet:

[PS] C:\> Remove-AzureADUser -ObjectId JackSmith_hotmail.com#EXT#@office365itpros.onmicrosoft.com

Alternatively, you can leave it to individual guests to decide when it is time to remove their account from your tenant.
A user can belong to up to 23 Azure Active Directory tenant directories and can choose to leave a tenant and remove
their guest account at any time. To leave a tenant, the user goes to the Access panel , and clicks their name in the top
right-hand corner to expose their profile. Click the gear icon in the Organizations section, which lists the tenants in
which they have accounts. Azure Active Directory displays the user’s profile, with the tenants listed towards the
bottom. To leave a tenant, select Leave organization , and then confirm with Leave (the user might have to sign-into
the tenant first). Azure Active Directory then removes the guest account from the chosen tenant directory, signs the
user out of the tenant, and sends email to confirm that they have left the organization and can no longer access
applications in that tenant.

The guest account stays in a soft-deleted state for 30 days following removal. During this time, the tenant
administrator can restore the guest account, or complete the process by removing the account permanently. When a
guest account is removed, it loses all access to SharePoint and OneDrive documents, libraries, and lists that it has
been granted access to on an individual basis or through a group. It also loses access to all Teams it was a member of
in the tenant.

Office 365 Groups and Compliance

As explored elsewhere in this book, compliance technology exists to help companies follow the regulatory and legal
frameworks that apply to their business activities. Because Office 365 Groups function across the service, it makes
sense that they use the service-wide compliance features rather than workload-specific features. For example, you can
include group mailboxes and the group document libraries in simple content search or the searches associated with
eDiscovery cases (Chapter 20). Office 365 retention policies can apply to content held in group mailboxes and
document libraries and group owners can apply retention labels to conversations to keep those items for compliance
purposes. Table 12-5 outlines the functionality supported by Office 365 Groups in different search and hold scenarios.

Include group mailbox Include group document Create in-place hold on

library data

Exchange Online Yes No Yes

eDiscovery search

SharePoint Online No Yes Yes

eDiscovery search


Security and Yes Yes Yes

Compliance Center
eDiscovery case

Office 365 retention Can keep or remove selected Can keep or remove selected Can impose in-place hold to
policies group conversations or Team files in group document keep items for defined
chats. libraries. periods.

Security and Yes Yes Not applicable

Compliance Center
Content search

Security and Yes Yes Yes

Compliance Center
eDiscovery cases

Table 12-5: Office 365 compliance features applied to Office 365 Groups

Yammer-based Office 365 Groups hold their discussions in the Yammer data store. These discussions are not subject
to Office 365 compliance features at present. The same is true for the plan metadata created by Planner and the chats
created by Teams.

Controlling Group Creation and Compliance : One aspect of compliance sometimes overlooked is the desirability
of controlling group creation. If you allow everyone to create groups, you might end up with an unmanageable mess.
Not only will obsolete and unwanted groups clutter up the corporate, but it will be hard to figure out what groups
hold information needed for compliance purposes. If compliance is of concern to your tenant, consider applying a
policy to control group creation so that you know what groups exist, their purpose, and whether they should come
under the control of compliance policies.

Dynamic Office 365 Groups

Office 365 Groups support both static and dynamic membership. Static membership is where you add and remove
people from membership via a client or programmatically (for example, with PowerShell). Dynamic is where the
membership of the group is computed by running a query against Azure Active Directory. Dynamic membership is
useful when you have groups that change often and whose membership can be determined by reference to one or
more attributes of user accounts. For example, the people who work in the Madrid office, or everyone in the
Engineering department. You can create a new dynamic Office 365 Group by:

Creating a new group from the Azure portal, setting its type to be “Office 365”, and then setting its membership
type to be “Dynamic User.”
Creating the new group through PowerShell by running the New-AzureADMSGroup cmdlet. Due to the
difficulties of making sure to build correctly-formed complex queries, this method is only recommended for
dynamic groups that use simple queries.

In all cases, when a group has dynamic membership, you cannot update group membership through OWA, Outlook, or
the PowerShell *-UnifiedGroupLinks cmdlet and the only methods available to change membership are:

Alter the Azure Active Directory query to find another set of users.
Update the properties of individual accounts in Azure Active Directory so that they come under the scope of the
query used to calculate group membership.

Once you decide that a group should have dynamic membership, the focus for its management usually shifts onto the
Azure Active Directory portal, unless you decide to use the Set-AzureADMSGroup cmdlet to create the new group
through PowerShell. Go to the All Groups section of the portal and click the New Group button. Make sure that you
select Dynamic User for the membership type and enter a name and description for the new group. Move the Enable
Office features slider to Yes to make the new group an Office 365 Group. Now click Add dynamic query to create
the query defining the group membership.
Figure 12-10: Enabling dynamic membership for an Office 365 Group

You now have a choice of using a simple rule or an advanced rule to form the query. A simple rule is one that has a
simple check against an attribute like Department, City, or Country. An advanced rule combines checks against
multiple attributes to find the accounts to include in the group. Figure 12-10 shows an example based on the
Department attribute. Keeping track of the members of a department is often a challenge and using a query against
the directory is an effective way of avoiding the need to constantly update group membership.

If you need to combine attributes in a query, click the Advanced rule checkbox. If a simple query already exists, the
portal copies the query code into the textbox used to create the advanced query. The query input is free-text and no
context-sensitive intelligence checks that the query entered is valid. You will only discover that an error exists in the
syntax entered when you try to save the query, which is when the Azure portal will refuse to save the query. Figure 12-
11 shows how to refine the original query to find users who belong to the Sales department by adding an extra clause
to the query so that it only selects users from Ireland.

As explained in the Azure Active Directory documentation for advanced rules , you can also use the -in and

-notin operators to compare the value of user attributes against a list of values (such as countries or department
codes) to include or exclude users from the results that a query returns.

When the query is complete, add it to the group and then click Create to create the group. To check the effectiveness
of the query, view group properties and click Members . Sometimes it takes a little while for Azure Active Directory
to refresh the list of members after you update a query, but eventually you should see the membership that you
expect.

Figure 12-11: Creating an advanced rule for dynamic membership

It can take a little time before the new group shows up as an Office 365 group. This is because of the need to
synchronize information between Azure Active Directory to Exchange Online and SharePoint Online. If you examine
group properties, you will see that the portal assigns a GUID as the alias for the group. For later convenience when
using PowerShell to work with the group, you should assign a more user-friendly alias. In this example we see the
type of alias for a newly-created dynamic Office 365 group and then change the alias to make it easier to work with.
PS C:\> Get-UnifiedGroup -Identity "Dynamic MVPs"

Name Alias ServerName AccessType

---- ----- ---------- ----------

02044c64-32d4-479f-9c5... 02044c64-32d4-479... am0pr0402mb3715 Public

PS C:\> Get-UnifiedGroup "Dynamic MVPs" | Set-UnifiedGroup -Alias DynamicMVPs

Also, it can take a little time before a change made to the query for a dynamic group results in an updated group
membership. This is because a background process performs the computation and updates Azure Active Directory,
which then needs to synchronize the new membership to workload directories such as Exchange Online. The exact
time needed for this process to complete depends on the current load on the service.

Advanced Queries : The maximum number of characters that can be in an advanced rule for a dynamic group is
2048. That is more than enough to come up with some interesting queries. A quick read of some good advice from
Microsoft should help you to understand exactly what you can and cannot do with queries for these groups.

Checking Dynamic Membership

If you are uncertain about the query you use or want to confirm that the expected results appear in the portal, you can
use the Get-AzureADUser cmdlet to execute a similar query against Azure Active Directory and check the results that
it reports. If what you see from both queries match, you know that everything is working as expected. Here is an
example of how to query Azure Active Directory with the advanced query shown above

[PS] C:\> Get-AzureADUser -All $True -Filter "Department eq 'Sales' and Country eq 'Ireland'" | Format-Table DisplayName, Department,
Country

DisplayName Department Country

----------- ---------- -------

Andy Ruth (Director) Sales Ireland

David Pelton (Sales) Sales Ireland

Luka Abrus (Sales) Sales Ireland

You can then compare the membership calculated with the filter with the data reported by running the Get-
UnifiedGroupLinks cmdlet against the group.

[PS] C:\> Get-UnifiedGroupLinks -Identity “Marketing Department” -LinkType Member

Or, if you want to go back to Azure Active Directory, you can read the membership from the group. This is a two-step
process for dynamic groups because the normal Get-AzureADGroupMember cmdlet returns a list of object identifiers
that need to be translated.

[PS] C:\> $GroupId = (Get-UnifiedGroup -Identity "Marketing Department").ExternalDirectoryObjectId

[PS] C:\> $Members = (Get-AzureADGroupMember -ObjectId $GroupId | Select ObjectId)

ForEach ($M in $Members) {

Get-AzureADUser -ObjectId $M.ObjectId | Select DisplayName }

You can also check the membership of a dynamic group with OWA or Outlook. If you do, note the warning displayed
when you access membership information to say that only an admin can add or remove members from the group. And
of course, an admin must do this through the Azure Active Directory portal.

Dynamic Group Owners

Like any other group, a dynamic group must have at least one owner. The ownership of a dynamic group is static in
that these members are not affected by the query used to compute the normal group members. Instead, group owners
are added and removed individually to the group using the Azure Active Directory portal or with PowerShell. As
owners, these accounts are automatically considered part of the group membership.

User Access to Dynamic Groups

User access to dynamic Office 365 Groups works identically to groups that have static membership. Every time
someone tries to access a group resource, Azure Active Directory checks whether they have the necessary permission
and if so, the user gains access. Like regular Office 365 Groups, messages sent to a dynamic group exist as
conversations in the group mailbox. Groups automatically adds members as subscribers so that they receive copies of
all contributions to group conversations via email. One difference between a static and dynamic group is that Groups
does not add the user who creates the group to the member list. This is because the query for the group defines the
membership. Unless the account who creates the group comes within the scope of the query, the group owner is not a
member and therefore cannot access group resources. Likewise, the group owner is not in the subscribers list.

Groups does not notify members when they lose access to dynamic groups due to a change in the query that populates
the group membership or because their account details have changed. The first time someone is likely to discover that
they no longer are a member is when they try to access some group resources.

Experience of Exchange dynamic distribution groups suggests that dynamic group membership is a popular feature.
Some obvious differences exist in the implementation of the two types of dynamic group. Office 365 Groups only
support Office 365 accounts while the Exchange equivalent supports any mail-enabled objects. In a hybrid
environment, this means that an Exchange dynamic group can include both cloud and on-premises objects. Another
difference is that a wider range of tools exists that can manipulate or update Exchange dynamic distribution groups.
For instance, it is often convenient to be able to work with the queries used for dynamic groups through PowerShell,
albeit perhaps only for those who like to work out the details of how to create effective OPATH queries.

Hybrid Dynamic Office 365 Groups : Office 365 Groups are unique to Office 365. It should therefore come as no
surprise to learn that only Office 365 can run the query against Azure Active Directory to retrieve the membership of
a dynamic Office 365 Group. For this reason, hybrid deployments always route messages addressed to these groups
to Exchange Online. If you enable group writeback with the AAD Connect tool, the directory synchronization process
ensures that the value of the targetAddress attribute for dynamic Office 365 Groups uses the service domain (i.e.
onmicrosoft.com) rather than any vanity domain belonging to the tenant. By routing messages sent to these groups
via onmicrosoft.com, Exchange Online processes the message through its transport service and queries Azure Active
Directory to discover the group membership, and then delivers copies of the message for individual group members,
More information about using the AAD Connect writeback feature in hybrid deployments is available in the
companion volume and online .

The Cost of Dynamic Office 365 Groups

Using dynamic Office 365 Groups means that you need to buy Azure Active Directory Premium P1 licenses for
administrators who create and update dynamic groups plus any user whose account falls under the scope of a filter
used by a dynamic group. In other words, if you create a dynamic group that has a filter that resolves to every person
in the organization, you need to buy an Azure Active Directory Premium P1 license for all those users. In most cases,
it is better to use a standard dynamic distribution group to communicate with large sets of employees because you
avoid any licensing issue and will not exceed the supported limit for group membership. If you want to use something
like an “All Employees” group to foster a sense of collaboration rather than simply a means of sending out information,
a Yammer-based Office 365 group might be a better choice.

Microsoft does not enforce the licensing requirement for dynamic groups. Those who create or update the queries for
dynamic groups need a license before the Azure Active Directory portal will allow them to change queries, but the
query used in groups return all matching accounts found in the directory even if some do not have the necessary
license. It is entirely possible that Microsoft will enforce the licensing restriction in the future. When that happens,
queries will return licensed accounts only.

Because dynamic groups come at a cost, you should be careful to decide which groups should be dynamic and which
can stay with static membership. Before creating a new dynamic group, ask whether the membership is likely to
change over time. If you expect a low turnover in group membership, then static membership is a better choice as it
will not incur an extra cost and the membership is not dependent on the directory. On the other hand, organizations
that manage many groups whose membership is volatile (such as the students who sign up for a class) and can
calculate group membership by querying one or more of the supported directory attributes for dynamic membership
might be able to justify the cost.

Using Dynamic Groups with Teams and Planner : You can use a dynamic group with Teams but not with Planner,
which depends on fixed team membership. Dynamic groups are intended for groups whose membership is highly
volatile and whose communication is email-centric.

Group Expiration Policy

Office 365 Groups have been available to tenants since November 2014. Although it is possible to restrict the ability
to create new groups to a select set of users, but even in the most tightly-managed tenant, it is likely that some groups
eventually reach their best-by date and become disused. If this happens, it is good to track down those groups,
recover anything valuable stored in the group resources, and then remove the groups to reduce GAL debris. Groups
are not unusual in this respect. Experience shows that the same falloff in usage over time happens for shared
mailboxes, distribution lists, public folders and other objects shared by teams of people.

To make things easier for administrators to manage potentially-obsolete groups, Azure Active Directory supports an
automated expiration (or lifecycle) policy for Office 365 Groups to control how long groups can exist within a tenant
before a group owner must renew the group. As groups expire, Office 365 can automatically remove them from the
tenant. The expiration policy applies to all Office 365 Groups, no matter how they are used or whatever application
someone uses to create groups.

Figure 12-12: Updating the settings of a group expiration policy

A disabled group expiration policy exists for every tenant and is different to the other Azure Active Directory policy
used to control other group settings like limiting creation or naming rules. The expiration policy is an Azure Active
Directory premium feature that requires licenses for every member of a group coming within the scope of the policy.
Administrators also need an Azure Active Directory Premium P1 license to configure the policy, as if you access the
Azure Portal without this license, you will not see the UI controls for the policy. However, you can manipulate the
policy settings with PowerShell even if your account does not have a license. To work with the expiration policy, go to
the Azure Active Directory portal and navigate to Expiration under Group Settings to see the current policy settings.
You can now enable the policy, define settings and the groups that come under the scope of the policy, and then click
Save to make the policy effective.

The settings are:

The group lifetime : The default is 365 days, but you can select a higher value. For example, you could use 730
days to make groups expire after two years. Consider setting a higher value if you decide to enable the policy so
that older groups do not expire when you enable the policy and to allow group owners to become used to the idea
of expiring groups.
The default notification address . This handles the situation where a group has no owner. You can specify one
address (usually an administrator) to receive these notifications, or the address of a distribution group or Office
365 group, or the SMTP addresses for multiple recipients (separated by semi-colons).
The groups that come under the scope of the policy . Initially, the value is None , meaning that the policy is
in the default disabled state and groups do not expire. Selecting All means that every Office 365 Group in the
tenant comes within its scope. The Selected button allows you to apply the policy to one or more groups. You
pick the target groups for the policy from a list of all groups in the tenant (for example, you might decide to
exclude all the groups used by Teams from the policy). This is easy for a small tenant but can become very
tiresome for larger tenants that support thousands of groups. In this case, you can use PowerShell to script to
find and bring the right groups to bring under the scope of the policy. We explore how to manage the expiration
policy through PowerShell later in this chapter. Figure 12-12 illustrates the settings of a group expiration policy
 that applies to the selected groups.

If you change the scope of the policy from All to Selected , the policy ceases to apply to the groups outside the
selected set and stops those groups expiring.

Expiration Policy Timeline

No one wants Office 365 to remove data without warning. When an expiration policy is active, Office 365 checks the
last renewed date for every group and begins to send warning notifications to group owners to tell them when their
groups need renewal. The renewedDateTime property of a group holds the last renewal time for a group (or the time
of creation). Office 365 adds the expiry period to this date to know when a group expires. After you renew or restore a
group. Office 365 updates its renewedDateTime property with the current date and time (you can see this information
with the Graph Explorer but not with the Get-UnifiedGroup cmdlet) as a new starting point for the expiry countdown.
A crude method is to look at the WhenCreated property. For example, to find the groups that are older than 1 year and
report their creation date and the last changed date, we can do this:

[PS] C:\> Get-UnifiedGroup | ? {$_.WhenCreated -le (Get-Date).AddDays(-365)} | Format-Table DisplayName, WhenCreated, WhenChanged

The last changed date reflects the last time that someone updated a property of the group. We will discuss how to
track user activity within a group shortly.

Another way of finding out when groups are likely to expire is to consult the Office 365 Groups usage report in the
Office 365 Admin Center. The report lists the group name and last activity date along with the number of mailbox
items, SharePoint files, and Yammer messages belonging to the group. If you see no date listed for last activity, it
means that Office 365 found no trace of any activity since January 1, 2017.

Office 365 sends three warning notifications before it removes a group:

30 days before the group expires.

15 days before the group expires.
One day before the group expires.

For example, if the expiry interval is 365 days (one year), Office 365 uses the timeline in Table 12-6 . (You cannot
change these intervals as they are hardcoded).

Days Action

1 Group created (or renewed).

335 First expiry notification sent to group owners.

350 Second expiry notification sent to group owners

364 Final warning sent to group owners.

365 Expiry period reached. Office 365 soft-deletes the group.

395 30-day soft-delete retention period expires. Office 365 removes the group permanently.

Table 12-6: Timeline for Group Expiry

Within minutes of enabling the policy, owners of expired groups begin to receive notifications. Notifications for
expiring groups that do not have an owner go to the recipient nominated in the policy. After enabling group
expiration, it is possible that some groups will expire immediately because their creation date is older than permitted
by the expiry period. For instance, if you set an expiry period of one year, any group older than a year expires.

Normally, Office 365 soft-deletes expired groups. However, special processing occurs if groups expire at the point
when a tenant enables the policy. In this case, Office 365 sends a special form of notification to the owners and treats
these notifications as second reminders. In effect, even though their groups are already technically expired, the
owners have 15 days to renew these expired groups.
The owners of all groups that come within the scope of the policy must renew them, even if a group is very active. The
sole criterion for renewal is the original creation date of the group and then, following a renewal, the date of the
group’s last renewal. An example of where this might cause a problem is when you use a policy covering all groups in
the tenant, including a group used to limit the users who can create new groups. Eventually, that group will expire
like every other group. If you ignore the notification, Office 365 removes the group when its expiration period elapses,
which means that no one except an administrator will then be able to create a group.

Expiry Notifications
Group owners receive expiry notifications (Figure 12-13 ) through email irrespective of how they create groups. For
instance, you can create groups through many different applications without going anywhere near an email client.
Obviously, this means that group owners need to check their mailbox regularly for expiry notifications. If not, they
might overlook a notification and Office 365 will remove the group unexpectedly. Teams is a notable exception in that
this application exposes the expiration date for a team in its settings (Manage team > Settings > Team expiry) and
allows team owners to renew the underlying group for whatever lifetime is defined in the policy settings. When they
receive notifications, group owners decide whether to renew the group or let it expire. To help owners decide whether
they wish to renew the group, notifications include links to:

Outlook : Open the group and view the conversations that are taking place to see the last time that anyone was
active. Obviously, if the group is used for Teams, you won’t see any activity here.
SharePoint : Open the document library in a browser to reveal the documents stored there. Because Teams uses
the SharePoint library to hold its files, including a folder for each channel in a team, this is a valuable indicator of
activity in a team.
Group details : Open Azure Active Directory in a browser to reveal information about group members and
owners.

To renew a group, an owner can click the Renew group button in the notification message. This brings them to the
Azure portal to renew the group. Three things can happen:

1. If the group still exists and has not expired, Office 365 renews the group and signals success. Office 365 also
records an “Updated Group” audit record because the renewal updates the group’s RenewedDateTime property.
The same action occurs when a group is renewed from Teams.
2. If the group is soft-deleted, Office 365 restores it and sets a new expiration date.
3. If the group is hard-deleted (permanently removed from the tenant), the owner sees an error message. The group
is no longer recoverable.

If owners choose not to renew a group, Office 365 will eventually expire and soft-delete the group. At this point, Office
365 removes all the group resources – the group mailbox, Yammer group, Stream group, team, plan, and SharePoint
site. If you decide to let a group go through to final deletion, you should make sure to recover and preserve any
valuable information that exists in these resources. This is not an automatic process and it will take time and effort to
ensure the retrieval of any valuable information.
Figure 12-13: An email to tell a group owner that their group has expired

Restoring Expired Groups

When an Office 365 group expires, the expiration process moves the group into a soft-deleted state and its owners get
a confirmation that Office 365 has removed the group. The notification includes a Restore group button that the
owner can click to restore the group and bring it back from the dead. However, like any other soft-deleted group,
Office 365 permanently removes the group after 30 days, so you have limited time to restore a group. Alternatively,
you can use the steps described in Chapter 11 to recover a group.

Case Study – Microsoft IT

Those given the job of planning the deployment of a new technology usually like to know how other companies
approach the same task. At the Ignite 2018 conference, Microsoft IT described the way they manage Office 365
Groups (the slide shown in Figure 12-14 is the summary). Microsoft is different from most companies: they don’t have
to worry about the cost of licensing advanced features and their user community is more technically-savvy than the
norm. However, there’s still value in understanding their perspective towards groups.

First, Microsoft uses a dynamic group for all full-time employees (“blue badges”) and allows members of this group to
create new groups. While allowing all full-time employees to create new groups might lead to a lot of groups that
don’t get much usage, Microsoft uses an aggressive 180-day expiration policy to age out groups that no one needs.
Microsoft doesn’t use a naming policy, possibly because they never used a naming policy for distribution lists. They
have custom jobs to scan for groups with no owner (important when you have an aggressive expiration policy), to
ensure that groups have at least two owners, and to make sure that groups that have certain classifications are
disabled for guest membership. They also use Azure Active Directory group reviews to make sure that guest members
only keep access to groups for as long as they need to.
Figure 12-14: Microsoft IT guidelines for Office 365 Groups

Microsoft also uses the Office 365 multi-geo capabilities for SharePoint Online and Office 365 Groups (in preview) to
provision the team sites according to users’ preferred data locations (the Office 365 datacenter region they are
deployed in).

Documenting a management framework for Office 365 Groups within an organization is a good idea because it brings
clarity to the deployment and lays out how the groups policy and other associated policies (like the Azure B2B
collaboration policy and expiration policy) fit into the framework.

Migration from Email Distribution

Groups
Microsoft is keen that tenants should replace distribution groups with Office 365 Groups. To make the move easy for
tenants, five methods are available:

Administrators can migrate individual distribution groups using the function available in the EAC.
Group owners can migrate individual distribution groups using the function available in OWA.
Administrators can run the PowerShell scripts created by Microsoft to migrate batches of distribution groups at
one time.
Administrators can run the New-UnifiedGroup cmdlet to convert an individual distribution group to an Office 365
Group.
Tenants can create their own versions of PowerShell scripts to perform the migration in their own way.

None of these methods work with distribution groups belonging to on-premises organizations. To convert these
groups, you must first move them to Office 365 and then, when the groups are “cloud-owned”, you can go ahead and
convert them. Of course, it might be simpler to recreate the on-premises distribution groups as Office 365 Groups and
then remove them from the on-premises organization.

Email distribution groups have been around for a long time. Different forms of distribution groups exist, some of
which are easy to convert while others are very difficult to move to Office 365 Groups today. It is safe to say that it is
relatively easy to code the steps to migrate a simple distribution group managed by Exchange Online that only has
cloud mailboxes in its membership. Things become more challenging when the need arises to process distribution
groups with the following characteristics:

Dynamic distribution groups : Office 365 supports Dynamic Office 365 Groups but no plans exist to offer
conversion facilities to transform dynamic email distribution groups in dynamic Office 365 Groups. This is likely
to be due to some of the difficulties created by the different objects supported by the two types of group. For
example, you can create a dynamic distribution group that includes several types of mail-enabled objects that are
not all supported by Office 365 Groups.
Nested distribution group : It is common to meet distribution groups composed of other (nested) distribution
groups. For example, an “All Company” group might include several other groups, each of which might include
people who work in a business unit or other division of the company. In turn, each divisional group might include
 other distribution groups for its operating units that hold the actual mail-enabled recipients. Nested distribution
groups are not supported by Microsoft’s migration toolset.
Mail-enabled universal security groups : These groups are not migrated because Office 365 Groups cannot
act as a security principal.
Distribution groups that include more than mailboxes : Office 365 Groups only support user accounts that
have mailboxes as members. They do not support some of the other kinds of mail-enabled recipients often found
in email distribution groups, such as public folders.
Moderated distribution groups : These are groups where a moderator must approve messages sent to the
group before delivery to group members.
Distribution groups with advanced settings : Groups that have send on behalf of settings or are hidden from
address lists, or that have restrictions on who can join a group (see note below).

Microsoft summarizes the situation by saying that their toolset can deal with “cloud-managed, simple, non-nested
distribution lists .” Email distribution groups that Microsoft’s migration toolset cannot process must stay in place until
they are either not needed and you can remove them safely or Office 365 Groups evolve to a point where they can
serve as a replacement. In some cases, as in the example of nested or dynamic distribution groups used by
organizations to send messages to very large number of users, that point might never come. Remember too that some
clients still in use do not support Office 365 Groups, so do not plan to move away from distribution groups until all
clients are prepared.

In this context, migrating an email distribution group to an Outlook group is only a question of moving members from
one group to another. Any other assets used with an email distribution group, such as an associated public folder, are
unaffected by the migration.

Migration from the Exchange Administration Center

The EAC supports the bulk migration of one or more distribution groups to Office 365 Groups. The process is simple.
If EAC detects that the tenant has some distribution groups that it considers eligible for migration, you can invoke a
wizard to do the work. Eligibility means that the distribution groups belong to the cloud rather than on-premises, do
not have nested groups, the membership is composed of mailboxes only, and meet other criteria (see above).

Figure 12-15: Bulk conversion of distribution groups in EAC

The conversion process is painless. Select the groups that you want to migrate (Figure 12-14 ) and click Start
Upgrade . The conversion wizard processes each group in turn in the background, running a command like that
shown below to convert each distribution group in turn. To be sure that the wizard converts the right distribution
group, Office 365 references the group with its unique identifier instead of the alias, display name, or distinguished
name.

[PS] C:\> New-UnifiedGroup -DlIdentity 'bf6aba7a-6dbc-4f2f-8e94-ae97289945d7'

-ConvertClosedDlToPrivateGroup:$true -DeleteDlAfterMigration:$true

If successful, the wizard removes the old distribution group and a brand-new Office 365 Group exists, complete with
the membership and other properties of the old group including its email addresses. The fact that the new group has
the email addresses previously assigned to the old group means that users can reply to messages sent to the old
group with a guarantee that Exchange can route those replies to the correct destination.

To check the progress of the conversion process, select “upgraded DLs” from the drop-down list and then Refresh .
You might need to do this several times as EAC does not signal the successful conversion of a distribution group. The
appearance of the new Office 365 Group in the list is the only way to know when the new group is ready to go.

When you convert distribution group, you might consider changing the display name of the group to reflect that it now
offers added resources to its membership. This step is manual and is not performed by the conversion wizard.

Upgrade Failure with Email Address Policies : If you use email address policies (see the companion volume) to
assign SMTP email addresses from a specific domain to newly created groups, you cannot use EAC or PowerShell to
convert distribution groups to Office 365 Groups. EAC will accept distribution lists for conversion and never do
anything thereafter and any attempt through PowerShell throws the error that the distribution group cannot be
migrated “due to an email policy set by your IT administrator .” The only solution is to convert the distribution group
through a multi-step manual process as described later in this chapter.

User-Controlled Conversion
Many users manage distribution groups, which they might control on behalf of a business unit or just to help do their
job better. If they are owners of Office 365 Groups, they can transfer the members of a distribution group to an Office
365 Group by using OWA to edit the membership of the Office 365 Group and including the distribution group as a
member. When OWA saves the group, it expands the membership of the distribution group and adds the individual
members to the group. Although this is an effective way of moving members from email distribution groups to Office
365 Groups, the obvious downside is that the old distribution group stays behind, leading to “GAL clutter”.

OWA allows users who own distribution groups to convert their groups. Behind the scenes, a background job
periodically scans for distribution groups that are cloud-managed and hold objects that can be members of Office 365
Groups. The job uses other factors to find suitable groups for conversion, such as those that have fewer than a
hundred owners. When the processing completes, a list of suitable groups owned (or managed by) the user is
available for display by OWA. However, the choice to migrate groups only appears when the user can create new
Office 365 Groups as it obviously would not make sense to allow people who cannot create new groups to attempt a
migration.

The set of eligible distribution groups appears at the top of the Groups section in OWA. The “DL” prefix appears before
the name of the distribution groups to show that they are not Office 365 Groups. To migrate a distribution group, click
on it. OWA then displays a screen with some details of the selected group and a big “Upgrade” button. Click to go
ahead with the migration. A new Office 365 Group replaces the old distribution group and takes on all the properties
of the source group.

Microsoft’s PowerShell Scripts

PowerShell is a terrific way to automate the process of converting distribution groups. Microsoft has scripts available
in the download center to migrate all eligible distribution groups found in a tenant:

Get-DlEligibilityList.ps1 : Scans the tenant to find distribution groups to create a report detailing the
distribution groups that can be migrated to Office 365 Groups along with those that cannot (for one of the
reasons given above). The output of the script is a text file (DlElugibility.txt) that you can review.
Convert-DistributionGroupToUnifiedGroup.ps1 : Converts all eligible distribution groups in a tenant to
Office 365 Groups.

The scripts are effective but can only deal with distribution groups that have the same characteristics discussed
above.

The DIY Approach to Converting Distribution Groups

Another example of how to approach the task of converting an email distribution group to an Outlook group is posted
in the TechNet gallery , but like all PowerShell scripts, the code is easily altered to meet the needs of a certain
scenario. The basic set of tasks that a conversion script needs to perform include:

Check whether an Office 365 Group with the same alias exists. The script could exit if this is the case or continue
by migrating the distribution group to a new Office 365 Group with a different alias.
Check whether the distribution group being migrated is a simple distribution group (type equals
MailUniversalDistributionGroup ). You could migrate the membership of a universal mail-enabled security group
to an Office 365 Group but not the security principal. There is probably some good reason why the security
principal exists, so it is probably best not to migrate these groups unless you are sure that the security element is
resolved.
Gather information about the owners (managers) and members of the input distribution group.
Update the alias of the input distribution group with a new value to allow the reuse of the existing alias for the
new group.
Check whether the new group should be public or private. One way to do this is to look at the
MemberJoinRestriction property of the input group. If this is “Closed” or “ApprovalRequired”, then it is
 reasonable to assume that a private group should be the result.
Create the new group by running the New-UnifiedGroup cmdlet.
Migrate the properties of the input distribution group to the new Outlook group. A direct match of all properties
does not exist, but a reasonable number of properties are supported by both types of group.
Add the members of the new Outlook group by reading the membership information from the input distribution
group and using the Add-UnifiedGroupLinks cmdlet to update the membership of the Outlook group. Some
checking is necessary to ensure that only mailboxes are added to the Outlook group. Mail-enabled public folders
and shared mailboxes are unsupported. In addition, you must expand the membership of nested distribution
groups to figure out the set of valid members and then add them to the group.
Because distribution groups act as a single address to distribute email to members, it is reasonable to expect that
the substitute Outlook group would behave in a comparable manner. Accordingly, you can also add the members
of the group as subscribers so that they receive new copies of conversations via email.
Add the owners of the new group by running the Add-UnifiedGroupLinks cmdlet. Mailboxes must be members of
the group before they can be added as owners.
Hide the old distribution group from the GAL so that users pick up and start using the replacement Outlook
group.

The solution gathers the necessary PowerShell commands together into a script, like the example available in the
TechNet Gallery . Since that script was written, Microsoft has updated the New-UnifiedGroup cmdlet so that it can
convert a distribution group to an Outlook group if it is an “eligible” group. For example, this command migrates the
settings from the “Department Managers” distribution group and creates an Outlook group of the same name. The
original distribution group is kept but is renamed as “MigratedDL-Department Managers” and is assigned a new alias
and email address.

[PS] C:\> New-UnifiedGroup -DLIdentity "Department Managers"

Closed distribution groups can be migrated to private Office 365 Groups. However, you must specify the
ConvertClosedDLToPrivateGroup parameter as otherwise the New-UnifiedGroup cmdlet cannot process a closed
group. For instance:

PS] C:\> New-UnifiedGroup -DLIdentity "Senior Managers" -ConvertClosedDLToPrivateGroup

PowerShell removes the original distribution group if you specify the DeleteDLAfterMigration parameter. However,
you might feel that it is safer to hide old groups from address lists and keep them around or a week or two, just to
make sure that everything is OK. This is not the approach taken in the EAC migration option, which deletes the source
distribution group after conversion.

A script built to migrate groups can be as complex as you think necessary to handle the kind of distribution groups
found in your tenant. The caveat is that you should not migrate every distribution group without thinking. In some
cases, a simple distribution group is the right tool for the job and you do not need to create the added overhead and
complexity that goes along with a group (document library, group mailbox, and so on.).

Migration from Site Mailboxes

A site mailbox allows users to share email information along with pointers (“stubs”) to files held in a document library.
Administrators create a site mailbox by adding the mailbox app to a site, which causes SharePoint to create a new
Exchange mailbox. Users can then create and send messages from the site mailbox or move items from their
mailboxes to the site mailbox to share with other members of the site.

Two major features distinguish site mailboxes from Office 365 Groups. First, the integration of site mailboxes into
Outlook functions more like a shared mailbox. For instance, users can drag and drop items from their personal
mailbox or a shared mailbox into the site mailbox. Unlike the conversations stored in group mailboxes, Exchange
treats items stored in the site mailbox as messages and keep all the characteristics of the messages. Second, site
mailboxes use pointers as placeholders for files in document libraries. The pointers hold some attributes, such as the
author, file size, document name, and so on, and a synchronization process keeps the pointers updated. Office 365
Groups always defer to the browser interfere when users want to open files in the group document library.

Beginning in March 2017, SharePoint site owners cannot create new site mailboxes. Users can continue to access
existing site mailboxes until the information they hold transition to another repository. Microsoft has committed to
deliver a migration process for site mailboxes to Office 365 Groups, but it is not yet available. In the interim, if you
want to move content from a site mailbox to an Office 365 group, you can do it manually.

Three sets of data need to be moved:

1. Users who have access to the site mailbox. These users become members of the target Office 365 group.
2. Messages stored in the site mailbox.
3. Documents stored in the SharePoint site belonging to the site mailbox.

To discover a list of site mailboxes in the tenant, use the command:

[PS] C:\> Get-Mailbox -RecipientTypeDetails TeamMailbox | Format-Table DisplayName, UserPrincipalName

DisplayName UserPrincipalName

----------- -----------------

Exchange 2013 Academy SMO-Academy@Office365itpros.onmicrosoft.com

Office 365 for Exchange Pros SMO-Office365forExchangePros@Office365itpros.onmicrosoft.com

Exchange Connections 2015 O365-ExchangeConnections2015@Office365itpros.onmicrosoft.com

Projects SMO-Projects@office365itpros.onmicrosoft.com

Alternatively, a user can see the list of up to 10 site mailboxes they have access to by selecting Manage All Site
Mailboxes from the menu exposed when they right-click on their mailbox root in Outlook desktop. The information
displayed in the list includes the email address of the site mailbox, the URL for the SharePoint site, the owners of the
site, and the number of members.

To migrate content from a site mailbox, do the following:

1. Create a target Office 365 group to receive the content from the site mailbox. Add the people who have access to
the site mailbox as members of the group.
2. The owner of the site mailbox can access the new Office 365 group with OWA. They can also add the site mailbox
as a shared mailbox to OWA by:

Expand the Folders section of OWA to expose your mailbox name.

Select the mailbox name and select “Add shared folder” from the right-click menu.
Input the name (or SMTP address) of the site mailbox you want to access.
OWA adds the site mailbox to its resource list.
Click the site mailbox to open it and expose its folder structure.
You can now drag and drop or copy items from the site mailbox to the group mailbox. Do this for all the folders in
the site mailbox, except the folders that hold document items synchronized with SharePoint. All the copied items
go into the inbox folder of the group mailbox and show up as conversations when viewed by an Office 365 Groups
client.

3. Open the SharePoint library belonging to the site mailbox and synchronize it to your workstation using the
OneDrive sync client. Do the same for the SharePoint library belonging to the target Office 365 group.
4. After the OneDrive sync client downloads the files and folders from the site document library to your
workstation, you can use Windows Explorer to move whatever files and folders you want to keep to the local copy
of the group document library (this is a great opportunity to dump unwanted information). The OneDrive sync
client then synchronizes the copied files to the SharePoint document library.
5. When the synchronization completes, the target Office 365 should have all the information you want to keep from
the site mailbox in its document library and conversations. You can now go ahead and clean up by removes all
traces of the site mailbox. The usual approach is to hide the site mailbox from address lists for a short period
(just in case) and then remove it. To hide the site mailbox, run the Set-Mailbox cmdlet. If the mailbox is under
litigation hold or an in-place hold, you can cancel the hold too (as otherwise you cannot remove the mailbox). To
remove the site mailbox, run the Remove-Mailbox cmdlet.

[PS] C:\> Set-Mailbox -Identity SMO-Academy@Office365itpros.onmicrosoft.com

-HiddenFromAddressListsEnabled $True -LitigationHoldEnabled $False

[PS] C:\> Remove-Mailbox -Identity SMO-Academy@Office365itpros.com

6. Remove the shared folder link for the site mailbox from OWA.
7. Delete the SharePoint site belonging to the site mailbox.

This process is manual and can take a long time to complete, especially when synchronizing files to and from
document libraries with the OneDrive sync client. However, it does get the job done and move all the content from the
site mailbox into an Office 365 group. An alternative approach to synchronizing libraries is to download the complete
copy of the document library belonging to a site mailbox and then uploading the files to another document library. The
downside of this approach is that SharePoint creates new files in the target library and does not keep any metadata.

Comparing Office 365 Groups,

Distribution Groups, and Shared
Mailboxes
Although Microsoft dedicates an impressive amount of development effort to support and enhance the capabilities of
Office 365 Groups, situations do exist when you should use a distribution group, a dynamic distribution group, or a
shared mailbox rather than an Outlook group. You can use the following questions to help guide your choice:

Is the intended use of the group to distribute email to a set of people? If so, then a standard distribution group is
 probably the best choice. Remember, if you need to, you can convert a standard distribution group to an Outlook
group.
Does the group span more than Exchange Online mailboxes? Office 365 Groups support a limited set of recipient
types (Exchange Online user mailboxes and guest users) while standard distribution groups support all mail-
enabled recipient types – mailboxes, mail users, mail contacts, public folders, resource mailboxes, and other
groups.
Do you want to moderate submissions to the group? Largely because they are designed for a relaxed form of
interaction, Office 365 Groups do not support the moderation feature that allows designated users to approve all
contributions before they can be distributed to group members. This is a feature supported by both distribution
groups and shared mailboxes.
Do you want to have access to more than the Inbox and Calendar folders and allow group members to move
items between mailbox folders? If so, a shared mailbox is a better solution. The same is true if you want to use
other mailbox features, such as assigning categories to messages.
Do you want to preserve earlier contributions and conversations shared between members of the group? You can
do this with the old-fashioned combination of a public folder (to hold the content) and distribution group or with
a shared mailbox, but the added functionality available to an Outlook group means that it is a better long-term
solution.
Do you want dynamic membership of groups based on Azure Active Directory attributes? You can have dynamic
membership of Office 365 Groups, but only with extra cost. In addition, if you want to have groups whose
membership has more than Office 365 users, then dynamic distribution groups are the right choice.
Do you want to limit who can send messages to a group? Office 365 Groups allow you to define whether a group
is public or private and whether a group can receive messages from external users. They also allow you to be
precise about what users can send messages to the group, just like distribution groups and shared mailboxes do.
Is the communication flow within the workgroup purely internal and the ability to send outbound email is
unnecessary? If so, Teams (Chapter 13) might be a better choice.

The arguments for Office 365 Groups are often based on the use of a common identity to access resources drawn
across from the service. They are much less email-centric than standard distribution groups, so the question often
comes down to the fundamental choice whether you just need email or to exploit that common identity across
Exchange Online, SharePoint Online, Teams, Planner, Power BI, and so on.

APIs for Office 365 Groups

The Microsoft Graph is an intelligent fabric that connects various sources drawn from across Microsoft’s cloud
services and presents them through a unified REST-based API . More information about the API, including a reference
and access to a test application, is available online and in Chapter 4.

Office 365 Groups are among the endpoints that are accessible through the Microsoft Graph. This creates possibilities
such as exposing group conversations in other applications. For instance, you might have a corporate dashboard
application to give managers and executives with highlights of company operational data. Integrating groups into the
application might allow users to contribute ideas, questions, or comments based on the data revealed in the
dashboard and follow the conversation through while keeping context. If a meeting is needed to discuss an issue, it
could be scheduled directly from within the application, and so on. A good example of how to use the API to explore
Groups, conversations, and other data is available here .

On to Teams
Groups have had a significant impact on Office 365 since their introduction in 2014. The best example is how a set of
new cloud-only applications developed by Microsoft for Office 365 use Groups as an identity and membership service.
Microsoft Teams is a great example of an application that makes extensive use of Groups. On to Chapter 13.
Chapter 13: Teams
Tony Redmond

Chat-Based Collaboration for Office 365

Teams is the Office 365 chat-based workspace application available to all Office 365 business and education customers. The central concept behind
Teams is that it delivers workspaces that bring people together to exchange information in the form of chats and associated content. Chats, or
conversations, can be private (often 1:1 but can involve more users) or public, in which case they occur in channels (categories to organize discussions)
within a team. A chat is usually a more dynamic exchange of views than is the norm with email. Microsoft calls this “high-velocity” communication
because many interactions might occur between the participants over a brief period. In addition, the chats tend to be shorter than email, with many
contributions consisting of just a few words. Applications like Facebook, Twitter, and WhatsApp have popularized interactive chat in the consumer
space, and companies like Bloomberg and Reuters have long offered real-time chat applications for the financial world, so it is unsurprising that this
modality is gaining traction in the world of general communications.

Microsoft’s Teams Adoption Guide sets out the vision and intent for Teams. We’ll explore the concepts and ideas described in the guide throughout this
chapter. We also describe the Teams architecture and how different Office 365 components are integrated into Teams to deliver its functionality along
with lots of practical information about how to plan, deploy, and manage Teams in an Office 365 tenant.

The Competition for Teams

Slack is the biggest and most obvious competitor for Teams. The major advantages for Teams over Slack are:

Integration with other Office 365 applications like SharePoint Online, OneDrive for Business, Exchange Online, Office 365 Groups, Azure Active
Directory, and Planner.
Integration with Office 365 administration, security, compliance, and auditing systems. Deploying Slack means that an extra, non-integrated
application needs support and management.
Low cost to adopt because Teams are part of the Office 365 business plans. A free version of Slack is available that offers reduced functionality
over the paid-for version. Microsoft competes head-on with the free version of Teams (see below), which has a smooth path to the full enterprise
version within Office 365.

In January 2017, Spiceworks predicted that Teams will be used by more people than Slack by 2019. At that time, it seemed like a stretch goal, but
Teams maintained a steady and climbing trajectory from its introduction to arrive at a point where Microsoft could claim leadership in 2018. Measured
by customer adoption, Microsoft reported:

125,000 organizations in September 2017.

200,000 organizations in March 2018.
329,000 organizations in September 2018.

We assume that each organization is an Office 365 tenant. Some organizations run multiple tenants, but this is not the norm.

Teams is now available in 181 markets and in 44 languages, including Hebrew and Arabic (introduced in August 2018). At the Ignite 2018 conference,
Microsoft said that Teams is the fastest-growing business application in the company’s history, with 87 companies from the Fortune 100 using Teams.
Furthermore, they emphasize the enterprise credentials of Teams by noting that 60 customers have more than 10,000 people using Teams and that the
largest customer (Accenture) has over 108,000 users . We assume that the users reported here are active Office 365 users with a Teams license who
open and use the application at least once a month.

The data for Slack are less clear, but one May 2018 estimate puts it at eight million daily active users (but only three million of which are paid
subscriptions). Taking Microsoft’s figure of 329,000 organizations, if 100 people on average actively use Teams in each organization, the user total for
Teams approaches 33 million. That figure might seem high, but six million are accounted for by the 60 organizations with more than 10,000 users and
the overall number should be viewed in the context of the 135 million active users of Office 365. Whatever the numbers are, the competition between
Teams and Slack will drive the development of new functionality.

Teams and U.S. Government Cloud : Due to the need to achieve compliance with several government security standards, Teams will not be
available to tenants in the U.S. Government Cloud (GCC) until July 17, 2018. Even then, several limitations in Teams functionality exist in the GCC
deployment, again because of the need to follow government standards. For example, Stream is not yet qualified for GCC, so call and meeting
recording is unsupported. See this page for more information.

Teams as the Modern Outlook

When Microsoft launched Outlook 97, they delivered an application that brought together the tools an office worker needed to get their work done. The
notion of velocity was somewhat different then as it might take several hours to send and receive email, especially over dial-up connections. While
products like IRC and VAX Notes existed, the idea of unstructured, fluid, rapid, dynamic chats between team members was chats was rare in an era
when network bandwidth was scarce. Real-time video calling to the desktop or to a smartphone was many years in the future. Even with its now
ancient roots, since its introduction, Outlook has been the standard yardstick for measuring the effectiveness of office productivity tools. Microsoft
sometimes refers to Teams as “Modern Outlook.” This doesn’t mean that Teams replaces Outlook. Rather, Teams is a new take on an integrated
application that reflects the way that a growing part of the workforce work today. Teams lacks the same email and personal calendar capabilities as
Outlook has and is an application that focuses on internal communications rather than external. Instead, Teams includes other features to attract
people to consider making this application the place where they spend a lot of their working time:

Conversations, including the ability to communicate with individuals, small groups, and defined teams of people. The conversations are persistent,
meaning that they endure over time. Members who join a team can reference earlier conversations because they persist within the team.
Instant messaging, including interoperability with Skype for Business.
Video and audio meetings.
Access to SharePoint Online document libraries and OneDrive for Business to store files.
Extensibility through connectors, bots, apps, actionable messages, and tabs.

Table 13-1 compares Outlook and Teams under several headings. This is not an attempt to say that one application is better than the other because
both applications are of their time. In some ways, the difference between the two reflects how people work differently now than when Microsoft
created the first Outlook client. Another difference between the two is that Outlook is a personal tool that extends into some areas of sharing while
Teams has sharing at its core. Still another influence is that Outlook must support both on-premises and cloud platforms while Teams weaves together
Office 365 components in a manner that is impossible to deliver on-premises. Finally, extensibility and adaptability are core to the Teams paradigm,
which results in a far broader range of options to make Teams work in the way that individual teams need the platform to behave. Through extensions
built by partners and with automatic updating of clients, Teams adapts to changing circumstances in a way that Outlook has never could. Perhaps this
is the major difference in the two applications.

 Outlook Teams

Communications Email for both internal and external Conversations organized in personal chats and channels. Support for inbound email only.
communications. No external email.

Collaboration/sharing Public Folders Files (SharePoint)/OneDrive for Business

Calendar Personal and shared Team calendar

Sharing Shared mailboxes Channels

Integrated None SharePoint, Planner, OneNote, Power BI, and other group-connected applications.
applications

Video/Instant None Video and audio meetings (replacement for the Skype for Business Online client), and
Messaging persistent chat.

Extensibility Outlook add-ins, connectors, and Bots, connectors, tabs, apps, and actionable messages
actionable messages.

Clients Outlook desktop (Windows and Mac), Desktop (Windows and Mac), browser, and mobile platforms. Desktop clients can update
OWA, Outlook mobile apps automatically to make sure that users always run the latest code.

Table 13-1: Comparing Outlook and Teams

Even with all its quirks and flaws, after twenty years of polishing, Outlook still fits well into the work habits of hundreds of millions of people (perhaps
because they have used Outlook for years) while Teams is attractive to others. Given some email functionality to enable external communications
outside the boundaries of an Office 365 tenant, who is to say that Teams cannot enjoy the same long-term success as Outlook has had. Microsoft
sometimes refers to Outlook and Teams as the two hubs for teamwork within Office 365, and that’s about the best way to look at the two. Intelligently
used, Teams focuses on rapid, brief, informal conversations while Outlook continues as the go-to client when people need to communicate outside the
tenant or communicate in a more formal and structured manner inside the tenant. Some refer to this mix-and-match approach as “right-sizing” email,
meaning that email continues for all manner of communications while Teams takes on some of the less formal interaction that typically happens within
tight groups focused on a common purpose. Companies can choose to continue focusing on email as their primary method for collaboration, to move to
a chat-based platform, or to mix and match the two styles to meet the needs of different sets of employees within the organization.

Versions
Teams is available in three versions:

Enterprise : This is the version that’s part of Office 365 enterprise (and equivalent academic and government) plans and is the focus of the
discussion in this chapter. Teams is also included in Office 365 Business Premium and Office 365 Business Essentials and is available as an add-on
for Office 365 Business. The enterprise version of Teams is the only one that can be used with the Microsoft Phone system.
Trial : If you have Office 365 but are not licensed for Teams, Microsoft offers a 1-year free trial of the enterprise version.
Free : Available to organizations with 300 or fewer users who do not have an Office 365 or Azure Active Directory tenant, the free version of
Teams uses the same architecture and runs on the same infrastructure as its enterprise counterpart. The user interface is mostly the same, with
Meetings and Calls removed from the navigation bar because these features aren’t part of the free version. Users of the free version use accounts
from vanity domains that are not associated with Azure Active Directory domains or consumer domains such as Outlook.com or Gmail. The free
version supports chats, audio calling, guest user access, the Office Online apps, and includes 10 GB of storage for files within channels
(SharePoint Online) and 2 GB per user (OneDrive for Business) for personal storage. Over 140 apps can be installed into the free version, and the
full range of Teams clients is supported. The target markets for the free version include SMBs, non-profit organizations, and schools.

The big differences between the Enterprise and Free versions are:

Scalability . The enterprise version is limited to 2,500 users per team, but scales to deal with the largest enterprise on a tenant level. The storage
available to the free version tops out at 610 GB (for 300 users), after which extra storage needs to be bought from Microsoft.
Access to Phone System features such as PSTN calling and scheduled meetings (because the free version lacks an Exchange Online mailbox to use
to schedule meetings).
Compliance . The enterprise version captures compliance records and supports retention policies and the capture of auditing records.
Office Collaboration : The desktop versions of the Office applications include added collaboration and security features (like support for protected
documents).
IT Controls : The enterprise version includes tools for the administration of Teams, including automation through PowerShell and Microsoft Graph.
Extensions : The enterprise version of Teams supports extra first- and third-party apps, like Stream, Planner, and Dynamics 365.

One of the interesting aspects of the free version of Teams is that it uses a hidden Office 365 tenant and Azure Active Directory instance. In effect,
when you sign into a free version of Teams, you connect to a tenant that hides all the enterprise features in the GUI. This means that if an organization
wishes to upgrade from the free version, it is relatively straightforward for Microsoft to switch them over to a full-blown Office 365 tenant. As you’d
expect, because the free and enterprise versions share a common platform, Microsoft updates their code concurrently. This doesn’t mean that the free
version will gain new features over time as it’s quite likely that Microsoft will only expose new features in the free version if a competitive need exists.

Industry-Specific Versions
Teams for Education already delivers a special classroom-oriented experience tailored to meet the needs of teachers and students. Building off that
experience, Microsoft is preparing more industry-specific versions of Teams to address the needs of other worker groups. Teams for Frontline workers
will replace StaffHub (due to be retired on October 1, 2019) and give workers and managers the ability to organize their shifts and other aspects of
scheduling such as time-off requests. Another version is being worked on for healthcare professionals in hospital-like environments to organize and
streamline patient care. This version includes an integration between Teams and Electronic Health Records and uses capabilities such as image
annotation and priority notifications that will be available to commercial customers by the end of 2018. Due to the wide difference in legislation
governing healthcare, this version might not be available in all countries
Teams Architecture
The Office 365 fabric provides the foundation for Teams. Figure 13-1 shows the major components in the Teams architecture, divided into the
presentation layer (clients) and the back-end services, which run inside Office 365. Some of the terms used might be unfamiliar to you:

• Electron is used by Microsoft to build the desktop, web, and mobile versions of Teams. Electron is an open-source
framework based on Chromium and Node.js that is developed through GitHub. Electron allows developers to reuse web
components to create a desktop GUI. Although much of Teams is written in NodeJS and C++, the desktop version of
Teams is a wrapper (Electron) around the web site.

• The Next Gen CallingNotification is the audio/video stack used by Teams for functionality such as video
conferencing. Microsoft refers to the stack as “next generation” because this is a brand-new iteration of earlier stacks
created for on-premises products like Skype for Business (and its OCS and Lync predecessors). Because it runs in the
cloud, changes and evolution occurs at a more rapid pace.

• Experimentation is a configuration service used for Skype components.

• MRU means “Most Recently Used,”, a service used to keep track of files that users access through Teams. The “WS”
suffix means “Windows Service”.

• WAC means “Web App Companion,” more commonly known as the Office Web Apps Server .

• The email service used by Teams allows channels to accept inbound messages. Teams does not support outbound email.

• AAD is Azure Active Directory, used by Teams to authenticate tenant and guest users, manage group membership, and
for extended services such as group settings and expiration policies, conditional access policies, and so on.

• The Notification hub generates notifications for users that appear in their Activity Feed.

• The Firehouse listener component gives a single point where all chats pass through, somewhat like the way messages
must pass through the Exchange transport service. Teams uses the listener to populate the activity feed for users and to
generate email reminders to users to make sure that they do not miss out seeing conversations.

Figure 13-1: Teams Architecture (source: Microsoft)

Teams also includes an experimentation layer to support the deployment of different features to targeted tenants. This is typically used to deploy new
features for testing at different stages of the development process, so that Microsoft sees a very early version of the feature, customers who sign up for
testing see a more developed version, and normal tenants only see the feature when it is completely developed and ready for release.

Architects might love the flow of connections in Figure 13-1 , but Figure 13-2 is easier to understand because it boils Teams down into five major
building blocks.

Teams has one client that is available in desktop, browser, and mobile versions. This means that features are easy to roll out concurrently and that
you have access to the same functionality in all clients.
Teams consumes a range of services drawn from other Office 365 applications like Exchange Online and SharePoint Online to avoid recreating any
wheels. See the section covering dependencies on other Office 365 services later in this chapter.
Teams has its own set of microservices to manage different elements within its infrastructure. We discuss these microservices in the next section.
The voice, audio, and telephony components used by Teams come from the infrastructure built for the Skype consumer service.
Teams consumes many Azure services. For performance reasons, the persistent chat service stores recent chats in-memory while the complete set
of chats is committed to Azure storage (using blobs, tables, and queues). Teams stores images inserted in conversations in an Azure media store.
Another example is the use of Azure B2B collaboration to support external access to Teams.
Figure 13-2: Building blocks for Teams

Location of Teams Data

Like other non-core Office 365 workloads, Microsoft does not store the Teams-specific data in every Office 365 datacenter region. This data includes
personal chats and channel conversations, teams and channel metadata for a tenant, and media inserted into chats (data inserted by Teams into
SharePoint and Exchange exists in the datacenter region to which a tenant belongs). Chats are not indexed by the Search Foundation and are therefore
not available to applications like Delve, but because Office 365 captures copies of chats in group and user mailboxes, they are available for eDiscovery.
The storage for images and media used in chats (except for Giphy GIFs, which Teams stores as a URL to the original file) runs in an Azure-based Media
service deployed alongside the chat service.

Although Teams is not yet available in every Office 365 datacenter region, Microsoft plans to deploy the necessary Azure services to support Teams
more broadly to satisfy customer data sovereignty concerns. As of August 28, 2018, the Teams services run in the following Azure datacenters:

Asia-Pacific: Singapore and Hong Kong.

EMEA: Dublin and Amsterdam.
Americas: Bay (California) and Boydton (Virginia).
United Kingdom: Cardiff and London.
India: Chennai and Pune.
Canada: Quebec City and Toronto.
U.S. Government Cloud.
Australia: New South Wales and Victoria.
Japan: Tokyo, Saitama, and Osaka.

Some data accessed through Teams might be in other datacenter regions. For instance, in a multi-geo situation, the SharePoint team site (holding
documents, the shared notebook, and wiki) used by a team runs in the region of the person who created the Office 365 Group for the team. The user’s
personal data used with Teams such as the compliance records logged for personal chats and any OneDrive for Business documents shared in chats are
stored in the satellite region. Remember that Microsoft’s commitment to data sovereignty only extends to the data that they manage and does not
include third-party data accessible through Teams. For instance, if you create a tab for Trello, the data manipulated through that tab are in Trello’s
cloud and might be outside the region.

Microsoft also plans to offer a facility to allow tenants to move Teams data to in-country locations in certain circumstances (for example, from EMEA to
U.K., or U.S. to Canada). This facility is not yet available but is expected in early 2019.

Use of Exchange Online

Teams makes heavy use of other Office 365 applications. Ideally, for maximum functionality, Teams users should have their account in the cloud with
access to an Exchange Online mailbox. In hybrid configurations, users with an Exchange on-premises mailbox can still use most Teams features if
Azure AD Connect synchronizes their account with Office 365.

The big issue for on-premises users is usually that they cannot create or join meetings. For more information about functionality available to hybrid
Exchange users, see this document . An issue invisible to users is that Office 365 cannot create compliance records for personal conversations in on-
premises mailboxes belonging to participants. The compliance records for contributions from on-premises users are therefore unavailable to
eDiscovery searches. However, the issue is partially mitigated if the chats involve at least one cloud user and Office 365 can create compliance records
in their mailboxes.

Microservices
Figure 13-3 shows the set of microservices Teams uses to run various parts of its infrastructure. The purpose of most of the microservices is clear. For
example, the compliance microservice generates audit records about activities that occur in Teams, such as the creation or deletion of a channel. It also
captures copies of personal and channel conversations for compliance purposes. Like all applications, Teams has some settings that are individual to it,
like the settings that control whether users can remove or edit messages. The configuration microservice manages these settings and works with the
AAD synchronization service to manage other settings that affect Teams, like whether users can create new teams.

AAD synchronization also makes sure that changes to the groups that underpin Teams flow from Groups to Teams and Teams to Groups. It is obviously
important to ensure that membership and ownership changes are effective, user properties (like photos, job titles, and reporting relationships) become
known to Teams, and that Teams executes its side of management actions like the deletion of a group (and its associated team). The expected
synchronization interval is less than 15 minutes, but the defined SLA (internal to Microsoft) is 24 hours, so it might take some time before changes
made in AAD appear in Teams or vice versa. The AAD synchronization microservice is also responsible for the recovery of Teams components when an
administrator recovers a soft-deleted group (see Chapter 12 for details).
Figure 13-3: Teams micro-services

Dependencies on Other Office 365 Components

Along with its microservices, Teams consumes a range of services drawn from other parts of Office 365. These dependencies are:

Office 365 Groups . A group exists for each team. The group object is in Azure Active Directory and the members of the group are the members
of the team.
Exchange Online . The Meetings functionality within Teams uses the calendar in the group mailbox to hold details of meetings. The profile
picture (avatar) for a user comes from their mailbox. Compliance records for channel chats are in the group mailbox while those for personal chats
are in the mailboxes of chat participants. Users with on-premises mailboxes can use Teams, with some limitations , notably that on-premises users
must have mailboxes on Exchange 2016 CU3 (or above) servers to create and view meetings. This requirement is because Exchange 2016 CU3
and above support a newer version of the Autodiscover service which the Teams middle layer uses to retrieve mailbox and calendar settings.
Skype for Business Online : While the long-term plan is to phase out Skype for Business Online and replace it with Teams, a dependency exists
on Skype for Business Online for interoperability (chat and presence) between the two applications and for hybrid interconnectivity. In addition,
tenants might need Skype for Business Online for enterprise phone system features that Microsoft has not yet made available in Teams.
SharePoint Online : The Files functionality within Teams stores documents in a SharePoint team site created when Office 365 provisions a new
team. Teams users do not need a SharePoint license (in many cases, users have a client through an Office 365 plan) to access Files, but SharePoint
Online must be available within the tenant. Teams does not support SharePoint on-premises.
OneDrive for Business : Users need a OneDrive for Business license to be able to share files in personal chats.

In addition to these base services, Teams takes advantages of other Office 365 components to extend its functionality as-needed within an individual
channel. Examples of Office 365 components that Teams uses include Planner, Stream, Azure Information Protection, OneNote, the Office Online apps,
Office 365 connectors, Power BI, Flow, Forms, and actionable messages.

Teams Limits
Because of its dependencies on other Office 365 components, Teams shares the limits applying to those components. The free version of Azure Active
Directory , used by Office 365 tenants, supports up to 500,000 objects, so technically speaking, you could create 500,000 teams in a tenant.
Realistically, not every object in Azure Active Directory will be a group, so the upper limit for the number of Teams in an organization is less than this.
Teams uses Exchange Online and SharePoint Online for calendar, compliance, and document management, but it is unlikely that a group mailbox used
by Teams will exceed the 100 GB quota. On the other hand, a busy SharePoint document library might need some extra storage, which you can assign
using the SharePoint management tools. Microsoft has not yet documented any restrictions on the number of conversations, messages, or graphics that
Teams can hold for a tenant in its chat or media services.

The Transition from Skype for Business

In September 2017, Microsoft announced that Teams would replace Skype for Business Online. This will be a gradual evolution rather than a one-time
switchover. The Teams client will slowly accumulate functionality so that it matches features available in the Skype for Business client to allow tenants
to replace it with Teams. At the back end, the Teams telephony stack will evolve to bring in the enterprise features available in Skype for Business
Online like audio conferencing, cloud PBX (the “phone system”), and calling plans. As laid out in Microsoft’s roadmap published in October 2017 , the
functionality divides into three areas:

Messaging: For example, federated chat between Teams and Skype for Business.
Meetings: For example, support to hold participants in a lobby before a meeting begins.
Calling: For example, voicemail and transfer to PSTN calls.

In July 2018, Microsoft announced that Teams had reached feature parity with Skype for Business Online. New voice and audio functionality roll out at
a steady rate to allow organizations pick the best point for them to move over from Skype for Business Online. Microsoft then asserted in August 2018
that they had completed the roadmap set out in September 2017 to bring Skype for Business Online functionality into Teams. Of course, a lot of work is
needed within an organization to move communications off one platform to another, so the fact that Microsoft has reached a point in its development
plans is not the same as widespread adoption. The transition is likely to take several years and during this period, organizations can choose to run
Teams and Skype for Business Online alongside in several modes to a point where Teams finally takes over:

1. Separate Islands : Teams and Skype for Business co-exist but run separately.
2. Teams runs in collaboration mode : Skype for Business continues to handle meetings, instant messaging (private chats), and voice and video
calls. In many cases, this mode is needed to allow the capture of content from voice and video calls for compliance or regulatory purposes (using a
third-party product). In this mode, people access Teams to conduct conversations in channels and use apps.
3. Teams runs in collaboration and meeting mode : the role for Skype for Business declines as meetings move to Teams. However, Skype for
Business continues to handle private chats, and voice and video calls.
4. Teams Only .

At some point during the transition, you will want to move from two clients on user desktops to just Teams. Apart from anything else, this step
simplifies training and reduces the need for support. When this happens, you’ll rely on the interoperability features built into the Teams and Skype for
Business clients. These include:

Private chats (instant messaging): Simple text messages pass between clients. No emojis are supported, no file transfer is available, and screen
sharing is not possible.
Teams users can join Skype for Business users through the Skype for Business web plug-in or using the Skype for Business client configured in
meeting-only mode.
Skype for Business users can join Teams meetings using the Teams browser client.

Eventually, you will reach a point when all users connect to Teams and no further need exists for anyone to have the Skype for Business client running
on their desktop. For more information, see Microsoft's guidelines for interoperability using Teams and Skype for Business Online. Chapter 16 includes
more information about using Teams with the Microsoft Phone System.

Over the long term, Microsoft will deprecate Skype for Business and Teams will be the sole “hub for teamwork in Office 365,” a single client combining
business voice, video, meeting, and collaboration functionality. Because of its dependencies on Office 365, Teams is very much a child of the cloud and
there is no prospect of ever seeing this application show up in an on-premises version. This is one of the reasons why the Skype for Business server
continues to be available for on-premises deployments.

The Structure of Teams

The basic structure of a team is a set of channels that give users a framework to organize conversations (discussions composed of “persistent” chats)
and other relevant information, activities, and applications. A channel is a dedicated section within a team used by the team members to communicate
about a topic, like a space to organize a project that the team must manage. All team members can see everything in a channel, so if you want to keep
something private, you must do so in a personal chat. Only the people involved in a personal chat can see what goes on there.

No-one but team members can see the content within a team. When someone joins a team, they gain access to all the resources available to the team.
When they leave a team, they lose access to those resources. Although some restrictions do exist for guest members, the principle that all members
share equal access to team resources holds true.

Owners and Members

Teams use Office 365 Groups to manage membership and adopts the same basic structure of owners who manage the team and members who take
part in the team. An owner is also a member and there should be at least two owners for each team to make sure that someone is always available to
handle basic administration such as adding new members. If all the owners leave a team (perhaps because they leave the organization), a tenant
administrator can promote a member to become an owner or add a new owner to the team. Unlike Office 365 Groups, Teams does not support the
concept of subscribers who receive copies of any new posts for a conversation via email. If a member wants to see what is happening inside a team,
they must open it to join in the conversation.

Microsoft recommends that you should allow Teams users to create new Office 365 Groups. If limited by policy, users cannot create new Teams because
of the dependency on Groups. Although such an approach is liberating for users because they can create new teams as and when they like, it creates
some management challenges that we will get to later in this chapter. Information about how to control group creation by policy is in Chapter 12.

A team owner can promote any team member except a guest to owner status. To do this, use Manage team to display the current membership and
then click the down arrow in the role column beside the member’s name to select Owner. When someone is a group owner, they automatically become
an owner (or administrator) of the associated team and can perform actions such as adding or removing users, adding new channels and connectors,
and so on. To remove owner status, reverse the process and change Owner to Member. If you want to remove an owner from a team, you must first
demote them to Member status and then go ahead and remove them by clicking the X alongside their name (Teams displays the X when you move the
screen cursor over the role column). If an owner removes a user from a team, they also lose their membership of the underlying group. You cannot
have a situation where a subset of the members of a group can access the associated team and other members are not.

Like an Office 365 group, a team can have up to a hundred owners. An individual user can create up to 250 teams. However, a global administrator can
create as many teams as they need to within a tenant.

Channels
Every team has a default channel called “General.” The General channel is often where team owners post information about the team’s goals and
charter and discuss topics such as what those goals should be or how to reach them. It is also where the Teams service posts system messages about
the team, such as the announcement that someone has joined the membership, someone is now a team owner, or that someone created a new channel.
The other channels in a team are dedicated to different categories of information or topics. In this respect, channels segment the conversations that
occur within a team.

You should encourage users to post in channels relating to the activity they want to discuss instead of the General channel, where everything is
intermingled and can quickly become a confused mess. This is one of the reasons why you can change team settings to stop people posting in the
General channel. However, before you do this, the team owners should think about how they want to organize the work of the team into channels.
When the overall structure of the team is known, you can create the channels and encourage users to think about which channel is best for their posts.

A team can include up to 200 channels. For instance, a team might have a channel for their weekly update calls and others for specific topics, like ideas
for a marketing campaign or plans for a trade show. Although 200 channels give a wide degree of latitude in the topics supported by a team, the trick is
to create just the right number of channels. Too many channels can confuse users (the “where do I start this conversation” syndrome), while too few
can result in channels holding wildly different topics and lead to people losing sight of important conversations. When users find channels with content
that interests them, they can follow those channels so that notifications for items posted to the channel appear in their activity feed. Unless you like
dealing with hundreds of notifications daily, it can be a bad idea to follow a busy channel. Instead, users should follow channels where they absolutely
want to see notifications of new activity.

Apart from hosting conversations, channels are hosts for tabs, apps, connectors, and bots to allow owners to build out their teams with the necessary
functionality needed by members. You can also mail-enable a channel by asking Teams to generate an SMTP email address (see description later in this
chapter). When a channel is email-enabled, authorized users can send email to the channel to start conversations. The email capability only extends to
inbound communications as Teams does not support the ability to send email as a channel or on behalf of a channel.

Naming Channels
Ideally, each channel should have a name and description that tells members what they should expect to discuss in the channel. When composing the
name of a channel, you cannot use the special characters (~#%&*{}+/\:<>?|'"), but you can include the @ sign. You can include a full stop in a
channel name if it is not at the end. You do not have to enter a description for a channel, but it is good practice to do so.

Except for the General channel, you can edit channel settings to rename a channel if necessary. Members continue to have access to the content
because Teams uses GUIDs internally when it accesses channels. Teams displays the name given to a channel no matter what language someone uses.
For this reason, in multinational organizations, it is wise to seek channel names that make sense in as many languages as possible.

No Way to Reuse Channels : If a channel becomes disused, you can remove it to clean up the team. However, you cannot then reuse the same name
for a new channel in that team. Once a channel name exists within a team, Teams does not allow you to reuse it. Microsoft says that the restriction
exists “for information protection scenarios.”
Favoring and Following Channels
Some teams have more channels than others and the discussions in some channels are more interesting than others. Users can mark the channels that
they find most interesting as favorites, in which case Teams always lists these channels when you access the team. Channels not marked as favorites
show up under “more channels” and you must select a channel from this list to access it. If you then do something in the channel, like contributing to a
conversation, Teams adds the channel to the favorites list on a temporary basis. You can then decide to follow that channel if you wish. Note that if you
create a channel, Teams assumes that you want the channel to be one of your favorites and marks it as such.

To help new members get to know the ebb and flow of what happens inside a team, when someone joins a team, the application automatically adds the
five most popular channels from the team as favorites, meaning that the channels appear under the team name when you open it. Popular means the
busiest channel in terms of conversations. The user can then accept the recommended set of favorites or remove some of the automatically marked
channels from their favorites list. If a team has fewer than five channels, Teams automatically adds all existing channels to the favorite list for new
members. When they create a new channel, owners can mark the channel as an automatic favorite in the channel list of team members, which is a good
way to make members aware of the existence of an important new channel. If an owner forgets to highlight an important channel, they can do this later
through the Channels section of team settings by checking the auto-favorite box. Users can always remove the channel from their favorites list if they
decide that they don’t consider the content to be important.

Users can mark a channel as being important to them by “following” it. Following a channel has a different and more active meaning to favoring a
channel. When you follow a channel, you show Teams that you want to know when some activity occurs in the channel. Teams responds by including
what happens in the channel in your activity feed. If someone then posts to the channel, you will see it in the activity feed and receive a “toast” pop-up
notification. Posts from a channel continue to appear in the activity feed until you unfollow the channel. It is sensible to restrict followed channels to a
select set as otherwise it is all too easy to lose sight of important information in the flow of updates from busy channels.

Tabs
The content owned by each channel is structed into a set of tabs. A tab is analogous to a tab in a browser and serves as a link to a resource that is
available within the channel. Conversations, Files, and Wiki are default tabs that appear in all channels. Unlike the Files and Conversations tabs, you
can remove the Wiki tab from a channel. Examples of other tabs that you can add to a channel include:

Planner : Link to plans created with Microsoft Planner, including the base plan created for the underlying Office 365 Group or specific plans
created for use within the channel.
SharePoint : Link to a SharePoint site or folder to which team members have the necessary access rights.
OneNote : Link to sections of the shared OneNote notebook for the underlying group.
Power BI : Link to a Power BI report or workspace.
Office documents : Link to specific Excel, Word, or PowerPoint documents in the SharePoint document library for the team. You can also add a
link to a PDF document in the same manner.
Stream : Link (URL) to a video file or channel stored in Microsoft Stream.
Website : Link to the URL for any web site. For example, you can create a very easy integration with Yammer by creating a link to a Yammer
group.
Visual Studio : Link to a Visual Studio Team Services board to allow the team to work on code projects.
Third-party apps such as YouTube, HootSuite, Smartsheet, Polyscribe, Trello, Intercom, Polly, Zendesk, and Asana. You can browse the complete
set of available apps through the Teams Store.

Where it makes sense (as in the case of links to individual documents or web sites), you can assign your own name to the tab. When someone adds a
new tab to a channel, Teams highlights the tab with a “New” icon. The icon stays for seven days after the creation of the tab and then disappears. It
also disappears after you access the tab for the first time.

Access to Office 365 and other Web Apps : Because of the way it integrates different functionality into a single client, some people compare Teams
to Outlook. You can make Teams a launching point for many Office 365 applications by creating website tabs to link to those apps. For example, you
could create a tab for OWA by pointing it to https://outlook.office365.com, for the user’s OneDrive for Business site by linking a tab to https://tenant-
my.sharepoint.com/, or for To-Do by linking to https://to-do.microsoft.com/?app. The general rule is that if users can authenticate themselves with a
web site, they can access that site through Teams. This approach allows users to work in Teams while having access to all the applications they need
to use.

The Teams Wiki and OneNote

Microsoft’s definition for the Teams wiki is that it is “ a smart text editor that doubles as a communication machine, because you can draft, edit, and
chat all in one place .” A tab for the wiki is created for each channel, but you can rename or remove the tab as needed . Wiki content is stored in a
document library called Teams Wiki Data in the SharePoint team site belonging to the team. The first time someone accesses the wiki in a channel,
Teams creates a folder in the document library named after the channel and stores the wiki pages in MHT (web archive file) format. It is possible to
create multiple wiki tabs for a channel. In this case, the pages for all the wikis go into the channel folder. You can only edit the MHT pages through
Teams and any attempt to use another HTML-based editor generates an error message.

The wiki editor ( Figure 13-4 ) is available in the desktop and browser clients and is like that used to compose Teams messages. The editor includes the
ability to insert formatted text using a small number of predefined styles (paragraph, heading 1, and so on), hyperlinks, graphics, pages, and tables.
One obvious deficiency is the lack of support for search as you can’t use Teams search (command box), Delve, or SharePoint search to find content
stored in wiki pages.
Figure 13-4: Editing content in a Teams wiki (the channel tab is renamed)

Wiki versus OneNote

The way that wiki works, especially in terms of notetaking and the page/section structure, often causes customers to ask why Teams includes a
component that seems to compete with OneNote. OneNote is Microsoft’s flagship notetaking application which supports many high-end features such
as inking. You can integrate OneNote into Teams by creating a tab to point to a new or existing notebook. Many experienced deployment consultants
recommend that if a tenant is heavily invested in OneNote, you should not try to force the use of yet another notetaking application on users as it will
force the users to change the way they work. It’s also the case that people will lose functionality if they change from OneNote as the Teams wiki misses
some of the features supported by OneNote such as offline capability. In these circumstances, it is best to remove the wiki tab from channels after
creation and replace it with a tab linking to the most appropriate OneNote notebook.

Compared to OneNote, the Teams wiki is not as good for dynamic notetaking; it is better suited for capturing and sharing relatively static text, such as
the rules of engagement about the topics discussed in a channel. One way of thinking about the two tools is that a small team working in a channel
might use the wiki to sketch out some ideas that they later publish to a wider audience. A shared OneNote notebook might be the final publication
vehicle, but it is more likely that a Word document or web page will be.

Because it is part of the Teams product, the wiki is more integrated with Teams than OneNote is. For example, you can already start off a conversation
within a wiki page and use @ mentions to refer to the team, channel, or team members within wiki content if you need to draw their attention to
something. Microsoft says that the wiki will pick up other features over time, such as the ability to send information direct from a channel conversation
into the wiki. Whether tight integration is enough for Wiki to be preferred to OneNote is still to be seen.

Pinned Apps

Figure 13-5: Pinned apps

The ellipsis menu […] in the navigation bar reveals a set of pinned apps (Figure 13-5 ) with the idea being to give users quick access to the apps. The
OneNote link brings the user to their personal notebook while Planner gives views into tasks for plans the user belongs to, including the set of tasks
that they own.

In addition to the default apps selected by Microsoft, any apps installed for the user are in the list. In this case, among the installed apps, we see the
Hipmunk (travel planning) and Who bots (see section on Bots later in this chapter).

Teams Clients
Downloads are available for the Teams desktop app for Windows (7 or above) or Mac OSX (10.10 or above). If you prefer to use MSI packages to
control the roll-out of the desktop clients, you can download 32-bit and 64-bit versions . The browser client offers equivalent functionality to the
desktop client. As with other Office 365 web applications, you must use a recent version of Internet Explorer, Chrome, or Edge . Teams does not yet
support the Safari browser. If less than 600 pixels are available on a screen, access to Teams content is read-only. A special Teams app is available for
Microsoft Surface Hub devices.

The Teams desktop app does not support offline capability and needs an internet connection to work. From a practical perspective, the internet
connection needs to be reasonable in terms of both latency and bandwidth. Even though Teams clients cache some data for faster access, using the
Teams desktop or browser clients on an airplane Wi-Fi service can be an excruciating experience with noticeable delays in responsiveness. To help
assess the impact of Teams usage on a network, Microsoft has an online bandwidth calculator to help figure out what network capacity is needed. Your
mileage will vary depending on how people use Teams and the clients that they use. Remember that although Teams usage increases over time, the
usage of other applications like email and Skype for Business might decrease to offset the extra network demand. In terms of security, Teams supports
the same multi-factor authentication methods as other Office 365 applications do.

Mobile Teams apps are available for Apple iOS and Android . Platform-specific technology is used (Swift for iOS and Java for Android) to create the user
interfaces. Underneath, the mobile clients share the same basic functionality as their browser and desktop counterparts, including the ability to create
and manage teams (membership and channels). As you’d expect, the mobile clients are easier to use and more responsive than the other clients in
some areas and less functional in others. For example, the mobile client switches to a different tenant faster than the desktop or browser client do, but
it is easier to compose a complex post with the desktop or browser client.

Microsoft published a preview version of a Windows S client via the Windows Store. This client is now deprecated and will be retired on November 29,
2018.

Desktop Client Updates

The Teams desktop client is self-updating. The mechanism used (Squirrel, an open-source component that is the default auto-update method for
Electron-based apps) is different to the way that the other Office click-to-run desktop applications update as users do not have to exit the client before
an update proceeds. The fact that the Teams client self-updates is challenging for some organizations who like to control the software users have on
their workstations. However, Teams is more aligned with the app model found on mobile devices than traditional software distribution channels and the
aim is to deliver the best and most functional experience to users. For this reason, if you want to know what Teams delivers in client updates, you need
to keep a close eye on the release notes published by Microsoft . The release notes are also available by typing /whatsnew into the command box.

Sometimes it is useful to extract information about the version of the Teams client running on a workstation. The easiest way to do this is to click your
avatar and then choose About and then Version . The client then lists the current version number and the date the client last updated the software in
a strip at the top of the screen. If you need to retrieve the information programmatically, you can use PowerShell for the task on Windows to
interrogate the Teams executable and the log file that records updates. For example:

[PS] C:\> $TeamsExecutable = Get-Item("${Env:LocalAppData}" + "\Microsoft\Teams\Current\Teams.exe")

$TeamsInstallFile = $Env:AppData+"\Microsoft\Teams\InstallTime.txt"

$TeamsLogFile = $Env:AppData + "\Microsoft\Teams\Logs.txt"

$TeamsInstallDate = Get-Content $TeamsInstallFile

$TeamsVersion = [System.Diagnostics.FileVersionInfo]::GetVersionInfo($TeamsExecutable)

$R = Get-Content $TeamsLogFile | ? {$_.Contains("ring=") }

$LastRec = $R[-1]

$SplitRec = $($LastRec) -split "<"

$TeamsLastUpdated = $SplitRec[0]

Write-Host "Teams Executable:" $TeamsExecutable

Write-Host "Teams Version:" $TeamsVersion.ProductVersion

Write-Host "Installed on:" $TeamsInstallDate

Write-Host "Last Updated:" $TeamsLastUpdated

Teams Executable: C:\Users\tonyr\AppData\Local\Microsoft\Teams\Current\Teams.exe

Teams Version: 1.0.00.33658

Installed on: 12/10/2017

Last Updated: Wed Dec 13 2017 22:06:30 GMT+0000 (GMT Standard Time)

Teams Phone Application

The Teams Phone application is a special version of the Android mobile client built for device manufacturers such as Polycom, Audiocodes, Yealink, and
HP who wish to integrate Teams functionality into conference and desk phones. First shown at the Enterprise Connect conference in March 2018,
Teams-enabled devices allow users to call other Teams users or PSTN numbers, join a Teams meeting, and retrieve voicemail. In effect, the phone
application delivers the same functionality as other mobile clients with the chat (personal and channel), files, and extensibility features removed. Only
devices specifically designed to work with Teams can support the Teams Phone application.

User Settings for Clients

Users can control how Teams works for them. Clicking their avatar allows a user to:

Switch accounts, if they use Teams in multiple tenants.

Set their availability status to Busy, Do Not Disturb, Away, or Available.
Change their photo. Teams allows users to upload and publish photos even when other applications block this ability. Teams uploads the photo to
the user’s Exchange mailbox. Once uploaded to the mailbox, the photo becomes available to other Office 365 applications.
See a list of Saved conversations.
Check whether any software updates are available. Teams auto-updates, but you can check if you prefer.

Under Settings , the user can:

Change the theme used for Teams from the default to dark or high-contrast.
Define if Teams starts when Windows boots and if the application stays running when the user closes Teams.
Set the language used for Teams.
Select what Notifications to see when different events occur, such as the user being @mentioned in a conversation. If the user wants to receive
 notifications via email about events that occur when they are not logged into Teams, they can set the interval that elapses before Teams sends
email. The available intervals range from “as soon as possible” to “once a day.” You can disable notifications by setting the interval to “Never.” The
notifications Teams sends to users by email merely inform the recipient that they are missing something that is happening in a channel. Unlike
Yammer notifications, the email does not contain an extract of the conversation, so the recipient must go to Teams to see what provoked the
notification.

In addition, selecting Developer Preview from the About menu allows the user to decide if they want to run beta software for Teams.

Settings are specific to an account in a tenant. If someone is a guest in another tenant, they must configure settings for Teams in that tenant if they
want to see the same behavior everywhere.

Keyboard Accelerators
You can use different keyboard accelerators to get things done in the desktop client. For example:

Press R to reply to the current conversation or C to create a new conversation in the current channel.
ALT + X (Windows) or Option + X (Mac) expands the compose box to reveal extra formatting options.
ALT + Q (Windows) or Option+ Q (Mac) reveals the Emoji menu.
ALT +A (Windows) or Option + A (Mac) exposes the Attach file menu to allow the user to attach a file already in Teams or to upload from the
user’s computer or a connected cloud service.
ALT+E (Windows) or Option +E (Mac) goes to the command/search box where you can enter commands or search Teams for content. Press
ALT+K (Windows) or Option + K to see the available commands (as shown in Figure 13-6 ).
CTRL + K (Windows) or Command +K (Mac) inserts a link (URL) in a message.
CTRL – (minus) and CTRL + (plus) zoom out and zoom in on a Windows workstation. Command – and Command + do the same on a Mac.

Figure 13-6: Shortcut commands available in Teams

The command box at the top of the Teams window gives users fast access to common Teams operations. For instance, to call someone, type /call in the
command box and then select the person you want to talk to. The command box is also a great way to set your status with commands like /dnd (Do not
disturb) or /busy . Setting Do Not Disturb status stops Teams popup notifications when people reply to your messages or you receive an @mention.

Figure 13-7: Searching for News in the command box

See this page for the full set of keyboard shortcuts or type /keys into the command box.

In addition to Teams commands, if you install apps into Teams, they can become available in the command box. For example, if you install the weather
app, you can type @weather to find out the current weather in a location, or @News to use the News app to search for breaking stories about a topic
(Figure 13-7 ), or @YouTube to look for a video. Once you find what you need, you can copy it to the clipboard and then paste it into a conversation.

Debugging Teams Clients

If something goes wrong and you experience a problem with the Teams desktop client, the approach to get started again usually follows these steps:

1. For desktop and mobile clients, sign out of Teams and then sign in again. This removes any lurking problems with authentication and credentials
and solves many connectivity and cache synchronization issues. For browser clients, use a private session to connect to Teams. If this works, clear
any browser cookies associated with Teams and office.com and try with a normal session again.
2. If the desktop client is still failing, stop all the Teams processes. On Windows, use Task Manager to find the processes and end them. Restart the
client to see if the problem disappears.
3. If that doesn’t work and the desktop client continues to have problems, clear the Teams cache. On Windows workstations, to speed up
performance, Teams caches copies of server data in %\Appdata\Roaming\Microsoft\Teams folder. Occasionally, the cached data is out of sync with
the server or holds some corrupted data. Both conditions cause problems for the client. Before removing the cache, stop the Teams client (all
processes). Then remove the complete Teams folder. When you restart Teams, the client rebuilds the cache so that its content is fresh. Although
it’s not recommended to blow away the Teams cache as a first step in troubleshooting, it can solve problems where other steps fail.
4. If you cannot resolve the problem, you should file a service request and give the diagnostics and client logs (see below) to Microsoft to help them
understand where the root cause might lie.
5. Although not a formal service request (meaning that Microsoft won’t record the interaction as a support issue and follow it through to resolution),
you can also send feedback to Microsoft, including a screen capture by clicking Feedback in the bottom left-hand corner of the desktop or
browser client and then Report a problem . Teams automatically includes logs when you send Microsoft a problem report with the desktop client,
and you can choose to include a screenshot of what the client currently displays. You can send logs from the browser client, but you cannot
include a screenshot.

If you find that you can’t switch to another tenant, it’s likely that your credentials for that tenant have expired. To solve the problem, sign out of Teams
and sign back in again. After you sign in, if you then try to switch to another tenant, Teams will prompt for credentials to allow the client to connect to
that tenant.

Teams Client and Diagnostic Logs

The Teams client log holds information about operations such as bootstrapping the client, the download and application of code updates, plug-ins, and
sign-ins. To get the client log, click the system tray and find the Teams icon. Now right-click the icon and select Get Logs from the menu. Notepad then
opens the client log. Somewhat oddly, log data about the latest interaction between the client and the Teams service is at the bottom of the log. Some of
the information is in JSON format, which makes it a little harder to understand. Utilities such as Fiddler and Charles Proxy are useful tools to collect
information about the communications between a Teams client and the back-end services. Make sure that you install the root certificate for the utility
in the Trusted Root Certificate Store of your computer and enable HTTPS decryption before collecting anything.

The CTRL-Shift-Alt-1 key combination generates a diagnostic log file on Windows workstations, while Command + Option + Shift + 1 generates the file
on a Mac. Teams creates the diagnostics log in the Downloads folder on the workstation and creates a name by combining MSTeams Diagnostics Log
with the current timestamp. The diagnostics log is a text file that captures added information about client activity together with information about the
environment such as the client and version used. Its format is less approachable than that used for the client log.

Another more esoteric key sequence is to hold the CTRL key down and then left click four times on the Teams icon in the system tray followed by a
right click. This reveals a set of debugging options for developers, including access to cookies and other development assets. It’s a curiosity rather than
something you would ever use in anger, but it might be a handy thing to know when faced with a tricky support problem.

Teams Management

Figure 13-8: Teams and Skype for Business Online console

Originally, tenants managed the Teams application through settings available in the Services & add-in section of the Office 365 Admin Center. The
new Teams and Skype for Business Online Admin Center (Figure 13-8 ) brings together the settings for both Teams and Skype for Business Online. At
the time of writing, Apps is the only section of settings for Teams still in the Office 365 Admin Center. These settings define how apps are added to
Teams. See the section later in this chapter to understand how to control the set of default and external apps available within Teams.

Not only does the admin center allow tenants to manage the creation and maintenance of individual teams, it also supports the creation and application
of policies to control how different aspects of Teams work, such as messaging, meetings, and guest access. As explained later, while tenant
administrators have full access to all the settings and policies, a set of Teams administrative roles is available that can be assigned to users to give
responsibilities for the management of specific parts of Teams. See Chapter 4 for more information on these roles.

Teams and Skype for Business Admin Center

Teams has some org-wide settings, which apply to the entire tenant, and a set of policies governing how the product works. You can run Teams with
global or default policies that apply to all users or create policies with different settings and assign them to specific users. The global policies apply to a
user unless you define a new policy and assign it to their account, a feature that allows you to manage functionality on a user-by-user basis.
Org-wide Settings
Org-wide settings are divided into four sections:

External access defines how Teams and Skype for Business Online users communicate with people outside the tenant. The settings are:

External Access – Turn On or Off. You need to turn this setting on before users can chat with people in other Teams organizations.
Users can communicate with external Skype users – Turn On or Off.
In addition, you can define a list of domains to block or allow for chats. If you only want users to chat with people in specific domains, define those
domains here, or block the domains that you don’t want people to chat with. Leave the list blank if you are happy for people to chat with users in
any other Teams organization.

The Guest access settings control what a guest user can do within Teams. The settings include an On/Off switch to enable guest access for the tenant,
enable personal chat for guests, settings to control what guests can do with messages like edit or delete sent messages, and settings to control how
guests participate in meetings. These settings can be manipulated in PowerShell using the Set-CsTeamsGuestMessagingConfiguration cmdlet. For
example:

[PS] C:\> Set-CsTeamsGuestMessagingConfiguration -AllowUserChat $True

Teams settings control general aspects of how users work with Teams that are not covered by messaging or meeting policies. These settings are:

Email integration : Can users send email to channels (see section later in this chapter) using email addresses generated by Teams. If you enable
email integration, you can also confine the ability of people to send email to channels to an allow list of SMTP domains.
Files : Settings to control how users can share files, including whether they can share files using third-party cloud storage solutions (Box,
DropBox, and Google Drive).
Organization : Turn the organization tab on or off. If enabled, the tab allows Teams users (but not guests) to see details of the organization
hierarchy as stored in Azure Active Directory.
Skype for Business interop : An On/Off switch to control whether Teams users can chat with Skype for Business Online users.

The Devices settings control how Room Systems (like a Surface Hub) and IP phones work in meetings. The settings are:

Require a secondary form of authentication (the default is No access).

Set content PIN. The default is that a PIN is needed for outside scheduled meetings.
Resource accounts can send messages. Some devices such as Surface Hub use device accounts to interact with real users through email, IM, and
calls. By default, this setting is True, which allows devices to send IM messages.

Search defines whether directory searches performed by Teams users are limited by Exchange address book policies (ABPs). An address book policy
limits the visibility of a user to specific sections of the overall corporate address book, such as only the users in a certain department or country.
Policies are often used to limit communications between different groups like students and teachers or traders and financial advisors. The default for
the directory search setting is Off, meaning that no restrictions are placed on users. If set to On, the user is limited to whatever address book policy is
defined for their mailbox (if no policy is defined, they can see and communicate with everyone in the directory). For example, they can only start a
personal chat with another user who is visible to them according to the address book policy. See Chapter 3 in the Companion Volume for more
information about creating and using address book policies.

You can also use the PowerShell Set-CsTeamsClientConfiguration cmdlet to update the settings described above. For example:

[PS] C:\> Set-CsTeamsClientConfiguration -AllowGoogleDrive $False -Identity Global

The settings in Teams Upgrade define how Teams interacts with Skype for Business Online during the transition. See Chapter 16 for more
information.

Skype and Teams management cmdlets : The *-CsTeams* cmdlets are part of the Skype for Business Online PowerShell module. Like any of the
PowerShell modules used with Office 365, make sure that you download and use the latest version.

Meeting Policies
The meeting policy assigned to a user account controls what that user can do in meetings. The General policy is the default, but you can create as
many other policies as you like in the Meeting policies section under Meetings. The settings in a meeting policy are divided into the following groups:

General :

Allow Meet Now : Users can create ad-hoc private meetings within a channel.
Allow the Outlook add-in : Allow users to schedule Teams meetings via the Outlook add-in for Teams.
Allowing channel meeting scheduling : A channel meeting is available to anyone who can access the channel. For public teams, this means
that anyone in the tenant can join the meeting.
Allow scheduling private meetings : Users can create private meetings, meaning that the only invited attendees can access the meeting.

Audio and Video :

Allow transcription : Allow users to turn on automatic transcription generation for a meeting.
Allow cloud recording : Allow Teams to capture details of meetings in Microsoft Stream.
Allow IP video: The default is to allow video meetings. If you want to restrict meetings to audio, move this switch to off. Individual users have the
choice to restrict their participation to audio when they join a meeting and might choose to do so if the network connection is poor or they simply
don’t want other participants to see them in all their glory.
Media bit rate : The default is 50000 (50 MB). Do not change this value unless you have good reason to do so.

Content Sharing :

Screen sharing mode : Like Skype for Business meetings, Teams allows users to share content from their PC to meeting attendees as the entire
screen or as a single application. If you want to prevent this, disable the setting. The default is “Entire Screen.”
Allow participant to give or request control : Allow a meeting participant to give control of a meeting to another participant, or request
control if they don’t already have control.
Allow an external participant to give or request control : An external participant is a guest user or an anonymous user who joins a meeting.
The default is Off.
Allow PowerPoint sharing : Allow meeting participants to share PowerPoint presentations. Sharing a PowerPoint presentation is an alternative
to sharing a complete desktop and consumes less bandwidth when network resources are scarce. See this page for more information.
Allow whiteboard : Allow meeting participants to use the Office 365 whiteboard app.
Allow shared notes : Allow meeting participants to share notes during the meeting.

Participants and Guests :

Allow anonymous users to dial-out : This setting controls whether anonymous users can dial out of a meeting. The default is Off.
Allow anonymous users to start meetings : This setting allows anonymous users (people unknown to the organization) start a meeting if they
are the first to join. The default is Off.
Automatically admit users : This setting controls whether people can join meetings automatically. The default is Everyone in your organization ,
 which means that tenant users can join meetings without going through the meeting lobby. You can also set it to Everyone, which then allows
external users to join without having to wait for admittance in the lobby.

Teams meeting policies can also be created and managed using the New-CsTeamsMeetingPolicy and Set-CsTeamsMeetingPolicy cmdlets, while the
Grant-CsTeamsMeetingPolicy cmdlet is used to assign a policy to a user account.

Messaging Policies
The messaging policy assigned to a user account controls the actions a user can take when working with Teams messages. The Global policy is the
default. You can create new messaging policies through the Messaging policies section of the portal. A messaging policy has the following settings:

Owners can delete sent messages : Allow team owners to remove all messages from channel conversations.
Users can delete sent messages : Allow users to delete messages that they sent in channel conversations.
Users can edit sent messages : Allow users to edit the text of messages they sent to channel conversations.
Chat : If you disable this feature, Teams hides Chat icon from the navigation bar and limits users of the selected type to channel conversations.
This is a big piece of functionality to remove as it restricts the ability of those users to collaborate on an ad-hoc basis.
Use Giphy in conversations : Allow users can add animated GIF files to conversations. A connection is made to the Giphy service to fetch
available files. The choice to use GIFs links to the content rating, which can be set to “Strict” (nothing offensive to young audiences), “Moderate”
(parental guidance needed) and “Allow all content” (all bets are off). Adding gifs, memes, and stickers to conversations is acceptable in some
organizations and will be less acceptable in others. The culture, age, and location of the users all influence whether an organization wants to keep
conversations focused on strict business or is willing to let its hair down a little.
Use Memes in conversations : Allow users to add the memes available to Teams to personal or channel conversations.
Use Stickers in conversations : Allow users to add the stickers available to Teams to channel or private conversations.

No control is available over the ability of users to insert emojis into channel or personal conversations. Teams messaging policies can also be created
and managed using the New-CsTeamsMessagingPolicy and Set-CsTeamsMessagingPolicy cmdlets, while the Grant-CsTeamsMessagingPolicy cmdlet is
used to assign a policy to a user account.

Licensing Teams
Every account in your tenant needs a license to access Teams. In the past, Microsoft licensed Teams on a tenant-wide basis in two categories- Business
Users and Guests. Now, you can selectively enable or disable Teams on a per-user basis by selecting a user’s account and switching the Teams license
on or off through the Office 365 Admin Center. If you need to enable or disable Teams for many accounts, it’s easier to do this with PowerShell using a
script like the example described in Chapter 4.

To ensure that the widest range of functionality is available to users, make sure that you enable Exchange Online and SharePoint Online for anyone
using Teams. Users who do not have a license for SharePoint Online cannot use the OneDrive functionality in Teams to share files with users in chats.

Teams Licenses for Guest Users

Guest users do not need a license to access Teams. A set of controls applying to guest users are available under the Org-wide settings section of the
Teams and Skype for Business Admin Center, including their ability to join meetings, edit and remove messages, and participate in personal chats.
Blocking personal chats for guests is sometimes considered by organizations for compliance reasons. In the past, the fact that Teams did not capture
compliance records for guest chats was a factor in this decision, but that is no longer the case and any personal chat authored by a guest is captured in
a special mailbox created by Office 365 for this purpose (see the later section on Teams and compliance). If you block guests from using chat, you block
them from all personal chats, including guest to guest and guest to tenant user. There’s no way to differentiate between these types of conversations.

If the setting to control guest access to Teams doesn’t appear in the Teams and Skype for Business Admin Center, it means that Microsoft is still
upgrading the Teams administrative framework in the backend for your tenant. The workaround to control guest access is to run the Set-
CsTeamsClientConfiguration cmdlet to update the AllowGuestUser setting to $True or $False as needed. For example:

[PS] C:\> Set-CsTeamsClientConfiguration -AllowGuestUser $True -Identity Global

Because of caching, the new setting might take an hour or so to become effective.

If you don’t want a guest user to access Teams in your tenant, you should remove their account from Azure Active Directory. Removing the account has
the side-effect of removing their access from any other application that depends on Azure B2B Collaboration, such as Planner. It also removes any
sharing access the account has been granted to documents in SharePoint Online or OneDrive for Business libraries.

Release Notes
Like most of Office 365, Teams is developing rapidly. Microsoft publishes release notes for Teams online to give formal guidance about the introduction
of new features. Not everything released for Teams shows up in the release notes, but they are a useful resource.

Access for On-Premises Users

If a tenant uses Azure AD Connect to synchronize accounts with an on-premises deployment, users can create and access Teams even if their mailboxes
are hosted on Exchange on-premises servers. These users can take part in conversations, group chats, and calls and set up new tabs. However, they
cannot create or change the connectors associated with a tab, nor can they create or access meetings or even change their profile picture.

Creating Teams
Before you can create a new team, you must be able to create a new Office 365 Group. Some tenants allow any user to create a group while others
control group creation. The settings to control who can create new groups (and teams) are in the Groups policy in Azure Active Directory, and Teams
respects those settings. Chapter 12 has a full discussion of how to manage these settings. If the policy restricts the ability of users to create new
groups, it should also include a pointer to a group holding a list of people who can create new groups. If your account is in that list, you can go ahead
and create a new team. If not, you will see an error informing you that “Your IT department has disabled this Microsoft Teams feature for you ”
together with a suggestion to contact the IT department for help.
Figure 13-9: Creating a new team

Generally, the easiest and best way to create a new team is through the Teams app (desktop, web, or mobile). You can also create a new group and add
members to it through the Azure Active Directory portal, OWA, PowerShell, or any of the other methods available for group creation, and then team-
enable the group. However, this is a long-drawn-out process that involves extra synchronization (and possibility for delay), so it is best to go ahead and
use Teams.

To create a new team, click Teams in the navigation bar and then Join or Create a Team at the bottom of the screen. You can now select to create a
new team or join one of the teams suggested by Office 365. To go ahead and create a new team, click the Create team button in the Create a team
card to reveal the dialog shown in Figure 13-9 , completing the following properties:

Team Name : This is the display name for the team (and the underlying Office 365 Group) and it is what appears in the list of teams in the Teams
client and the Exchange GAL. It is always best when you give a team a meaningful name that conveys its purpose. If your organization uses a
group naming policy, Teams shows the effect of the policy by displaying the name of the team after it applies the policy when it creates the new
team. The group naming policy does not affect tenant administrators, so teams that they create keep the display name as entered.
Description : This is a free-form text description of why the team exists and what the members of the team use it to do. Users see the team
description when they browse teams, so it is important that some thought goes into the description to make it accurate, interesting, and easily
digestible. Get right to the point and put the essential information like the purpose of the group at the start and make sure that the first 80
characters convey the essence of the group because this is the amount of text displayed in search interfaces. Less essential information can be left
to the end of the description, such as who created the team, the contact name for the team, and so on.
Privacy : This is either “Public”, in which case anyone in the tenant can contribute to the team or join it as a member, or “Private”, meaning that
an owner or administrator must add a user to the team before they can access team resources. Office 365 Groups use the same differentiation. If
your tenant has less than 1,000 accounts, you’ll also have the choice to create an org-wide team (see below).
Classification : The classification helps to tell users the sensitivity level of the information shared within the team. You can select any of the
classifications defined in the AAD policy for Groups. Click the Change Setting link to expose a drop-down list of available values. In Figure 13-9 ,
you see the extra explanatory text for the classifications as defined in the ClassificationDescriptions setting of the Groups policy. If you need to
give guidelines about how to create new teams, you can set up a web page with detailed instructions and include the URL to that page in the
UsageGuidelinesURL setting in the Groups policy. If this setting is available, Teams includes the See your organization’s guidelines link in the
create dialog.

Teams clients do not check that a group, distribution list, or team of the same display name already exists in the tenant, so it is quite possible to create
multiple teams with the same name. This is one of the good reasons why some tenants restrict team creation to a small set of people, who have the
responsibility to check that the display name does not clash before they create a team.

After entering the properties, click Next to continue. Teams creates the Office 365 Group and provisions the resources for the team, such as the
SharePoint team site.

Using an Existing Team as a Template

You can create a new team using an existing team as a template. The idea is that if you are in the position where you need to create multiple teams
with approximately the same structure, it’s easier to create one team to use as a template and set the team up in a form that you want to replicate to
other teams. You can only use a team as a template when you have access to that team (you’re a member).

When you create a new team based on a template, you can replicate the membership (including guests), settings, channels, apps, and tabs from the
template team. No content is copied across in terms of conversations or files, and some tabs will need to complete a setup process before they can be
used. For instance, if the template team includes a tab pointing to Planner in a channel, the channel and the tab are copied to the new team, but
because the plan doesn’t exist for the channel, you must start Planner and create the plan.

After Teams creates the new team, you can add members (Figure 13-10 ) and specify whether the new member is an owner or just a normal member.
You can add individual users or use an existing distribution group or Office 365 group (or another team) as a source, in which case Teams expands the
membership of the selected group and adds each user individually to the team. Teams excludes unsupported recipient types found in these groups
(such as a mail-enabled public folder) and will not try to add them as members. If you use a distribution group or an Office 365 group to add members
to a team, remember that any later changes to the membership of the source group will not replicate to the team membership. Finally, as described
later in this chapter, if the tenant supports guest accounts, you can add external people as guest members to a team. To automate processing, any of
the supported member types can be added to (or removed from) a team with PowerShell.

Adding Team Members

Figure 13-10: Adding members to a new team

Up to 500,000 teams (or groups) can exist within a single Office 365 tenant. Each team can have up to 2.500 members. Organizations needing to
support larger groups should use Yammer Groups.

Teams with Duplicate Names : As noted above, Teams will let you create a team with a display name that duplicates another team. Obviously, this
will confuse users who belong to those teams as they will never be quite sure which team they should use. From a technical perspective, Teams is
quite happy to have duplicate display names because behind the scenes the teams have different aliases and names. If you get into this unhappy
circumstance, you can change the display name of one of the affected teams by editing the team properties (Edit team) or by running the Set-
UnifiedGroup cmdlet to update the DisplayName property for the underlying group. For example:

[PS] C:\> Set-UnifiedGroup -Identity BudgetPlanning -DisplayName "Budget Planning (2017)"

Renaming the team fixes the immediate issue of the duplicate display name. It will not rename the SharePoint team site for the team as SharePoint
does not support site renaming, possibly due to the knock-on effect that this might have on links pointing to objects in the site.

Creating an Org-Wide Team

If your tenant has fewer than 1,000 accounts, you can create an org-wide team. An org-wide team is designed to enable tenant-wide communications
for small to medium companies without the need for an administrator or team owner to manually add all the employees to the team membership,
including the need to check for new employees and add them periodically. As we explain in Chapter 14, the process of creating a team and populating
its membership with PowerShell is not difficult, but some work needs to be done to support the membership afterwards. This work is avoided if you use
an org-wide team.

Larger tenants who have more accounts than the limit can consider using:

Dynamic Teams to support discussions for different sections of the organization. For example, you might have a team for each department or
each country.
Yammer for company-wide communications and collaboration. Yammer can easily scale up to handle very large organizations with hundreds of
thousands of users.

If your tenant has between 1,000 and 2,500 active user accounts, another approach is to use scheduled PowerShell scripts to create and update the
membership and settings for that form of an org-wide team.

To create an org-wide team, choose Join or create a team as usual, and then select Org-wide from the Privacy drop-down list. The choice to create an
org-wide team is only exposed to tenant (global) administrators and when you create an org-wide team, Teams adds all the global admins as team
owners. It then adds all “active users” as members. You know the membership is managed by Teams because the Manage team screen has a banner
saying, “Team members will be automatically added and removed to reflect your Active Directory ”.

The theory is that automatic membership excludes accounts without valid Office 365 or Teams licenses as well as guest users. However, sometimes
accounts that have no place in an org-wide team turn up in the automatically-generated membership. Among accounts that should be checked against
the team membership are:

Shared mailboxes.
Room and resource mailboxes.
Service accounts (if they have an Office 365 license).
Mailboxes used for purposes such as DLP incident reports.
Accounts that don’t have a valid Office 365 license where the Teams license is disabled.

Because some odd or unwanted members might be added to the team, it is good practice to check the membership after creating an org-wide team to
remove anything that doesn’t belong. Once you remove someone from an org-wide team, the service remembers the deletion and won’t try to add that
account back to the membership. Likewise, if you find that an account that should be in the membership has been omitted for some reason, you can
add them manually. As with any team with a large membership, after you create an org-wide team, consider imposing order on discussions from the
start by updating team settings to restrict channel creation, the ability to post to the General channel, and to use @team mentions (because these
generate notifications to everyone).

On an ongoing basis, employees leave and join the company and people lose or gain Teams licenses. When someone leaves the company and their
Office 365 account is removed, their membership of the team is also removed. To handle new joiners and people who gain or lose Teams licenses, a
background process running on Teams clients scans the accounts in the tenant periodically (expect new users to show up within a day) and adds or
removes the user as needed.

Unlike normal teams, members can’t choose to leave an org-wide team. This limitation is the same as exists for dynamic teams, but unlike dynamic
teams, an org-wide team doesn’t use an Azure Active Directory query to calculate its membership. Instead, the background process manages
membership and the Azure Active Directory group for the team has an “assigned” membership, so you don’t need to buy Azure Active Directory P1
licenses for all the members.

Updating an Existing Team to be an Org-wide Team

If you already created and use an all-employees team without benefit of Microsoft's new feature, a tenant administrator can convert the team into an
org-wide team and gain benefit of the automatic membership management. To do this, select the team you want to convert and then use the Edit team
feature to change the privacy setting to org-wide. When you save the setting, Teams updates the membership with all valid accounts. Any users not
included in the automatic membership remain in place, including guest users. You can also change an org-wide team to be a private or public team
using the same approach, and in this case, the existing membership stays in place, but the automatic background refresh of membership is disabled.

Fixing Old Azure Active Directory Accounts with No UserType

Teams runs Graph queries to calculate the membership of org-wide teams and looks for Azure Active Directory accounts with Member in their
UserType property. If an account has a null value in the property, it won’t be included in the collection returned by the query and therefore won’t be
included in the org-wide membership. Some older Azure Active Directory accounts have null values because they existed before Azure Active Directory
populated these values (guest accounts are marked with Guest as their UserType).

To find out whether your tenant has any of these accounts, run the PowerShell command:

[PS] C:\> Get-AzureADUser -All $True |? {$_.UserType -eq $Null} | Format-Table DisplayName, UserType, ObjectId

To fix the problem, update the account with the correct value. For example:

[PS] C:\> Set-AzureADUser -ObjectId Guid_for_User -UserType Member

Joining a Team
Users can join a team when an owner adds them as a member. If a team is public, another member can add someone to the membership, including new
guest users. If a team is private, an owner or an administrator must add new members. To find new and interesting teams to join, users can browse the
set of suggestions Teams makes for them to join (see below). In all cases, when someone joins the membership of a team, they also join the underlying
Office 365 Group and can access all the resources available to both the team and the group.

To discover the teams suggested as good candidates to join, click Teams in the navigation bar and then Join or create a team in the bottom left-hand
corner (guests do not see this choice). Teams uses signals collected in the Microsoft Graph to suggest teams that the user might like to join (Figure 13-
11 ). Many suggestions are based on “common interests” shared by the existing membership and the potential new member. In other words, if a public
team exists with a membership composed of people who are also members of other teams with the user, a reasonable chance exists that the user will
be interested in the topics discussed in that team. Office 365 performs similar calculations when users browse for Office 365 Groups to join. The
computations to suggest teams to users occur behind the scenes and it takes up to a day after its creation or after a team’s access changes from private
to public before Office 365 suggests a team to users. You can also search by name to see if a team is available to join. The search function does not
support substring searches, so use the first word of a team to find it.

To join a team, move over the icon for the team and a Join team button appears. Click the button and the team appears in the navigation bar for the
user to access its resources. Once someone is a team member of a public team, they can add new members by selecting Manage team from the
ellipsis menu for the team, followed by Add Member .

Figure 13-11: Browsing for suggested Teams

After you join a team, you can move the team within the list of teams to which you belong. Teams does not display the list alphabetically but in the
order that you joined a team. To reorder the list, simply drag and drop a team into the position that you want. In most cases, people place their favorite
and most important teams at the top. You can also use the CTRL+Shift+Up and CTRL+Shift+Down key combinations to move a team up or down
within the list (on a Mac, use the Command key). You cannot change the channel order within a team. The General channel is always first, with the
other channels following in alphabetical order.

Joining a Team with a Code

To make things easier for owners to manage teams with hundreds of users, Teams supports the ability of users to join a team with a system-generated
code. The idea is that it is easier to supply potential members with a code that they can input to join a team than it is for a team owner to manually add
them to the membership. Using a code is also a good way for someone to know that they are joining the right team in a situation where many teams
with similar names exist in an organization, such as classes in a school. Team codes work for both public and private teams. The process is as follows:

A team owner goes to the Team code section under team Settings and clicks the Generate button to have Teams generate a unique seven-
character code for the team. The code is a value something like “jny9ota.”
The team owner also can remove a code at any time or to reset the code. A removed or reset code cannot be used to join a team.
The team owner publishes the code to potential team members. For example, if the team is open to anyone in the organization, you could publish
the code on a web site for all to see. On the other hand, if the potential membership is drawn from a more limited population, you could email the
code using a distribution group.
Team codes are only valid for tenant members. Guest users cannot use codes to join teams.
When someone receives a team code, they can join the team by through the normal Join or create a team process and input the code into the
Join a team with a code box, or by typing the /Join command in the command box. For example, /Join jny9ota.
Teams checks the supplied code and flags an error if it is invalid. If valid, Teams adds the user to the membership and opens the team.

Apart from the way that they join a team, members who join with codes are like any other member.

Joining a Team with a Deeplink

A deeplink is a URL generated by Teams when someone gets a link to a team, channel, tab, or message. For example, if you go to the the ellipsis menu
for a message and select Copy Link , Teams creates and copies a deeplink (or hyperlink) to the message to the clipboard. You can paste into another
message or use it to open Teams at the linked message. The same happens if you use Get link to team , Get link to channel, or Copy link to tab to
create a link to these elements.

A deeplink includes several important elements to make this happen. If we look at the link below, three terms are highlighted. These are:
 The reply chain identifier, which identifies the message thread (1523005757318 ). This identifier appears in several places within the link. The
identifier is calculated as a Unix timestamp (epoch), based on the number of seconds since 1 January 1970. The timestamp is calculated to the
millisecond. You can convert the identifier to human readable time with converters such as this web site , which reveals that the identifier shown
above is Friday, April 6, 2018 at 9:09:17.318AM.
The tenant identifier, which tells Teams what Office 365 tenant to access (c662313f-14fc-43a2-9a7a-d2e27f4f3478 ).
The group identifier, which tells Teams the team that the message belongs to (ce23aef8-c551-40d4-bdb6-2bcc1eb64626 ). The identifier points
to the Office 365 Group/Azure Active Directory group that underpins the team.

https://teams.microsoft.com/l/message/19:45ac50d1afa2425a80362e94f381b1e7@thread.skype/ 1523005757318 ?tenantId= c662313f-14fc-43a2-

9a7a-d2e27f4f3478 &groupId= ce23aef8-c551-40d4-bdb6-2bcc1eb64626
&parentMessageId=1523005757318&teamName=The%20New%20Hydra%20Project%20Team&channelName=General&createdTime=1523005757318

The team name and channel name are also included in the link.

If you send a deeplink to another person, they can use it to join the team that the link refers to. The link opens at a dialog to ask the person if they want
to join the team. If the team is public and the user accepts, they become a member of the team. If the team is private, Teams sends a request by email
to the team owners, who can then accept or deny the request to join. In addition to receiving email notifications about requests to join their team,
owners can go to the Manage team page to view requests waiting to be processed under the Pending Requests tab.

Leaving a Team
A user can leave a team at any time by selecting the “Leave the team” choice from the ellipsis menu available for the team and any channel in the
team. Alternatively, a team owner can remove a member by selecting Manage team , moving to the entry for the member, and then clicking the “X” to
the far right of the entry.

Guests can also leave teams. However, this action does not remove their Azure Active Directory account because the guest account might be used for
other purposes, such as membership of an Office 365 Group, another team, or access to some SharePoint or OneDrive for Business documents. If you
want to remove the account, you must do this through the Azure or Office 365 consoles, or by running the Remove-AzureADUser cmdlet.

Maintaining Team Membership

Normally, team owners handle membership. If the team is private, the team owners take care of membership by adding and removing users, by
promoting members to become owners, or by demoting owners to become members, which is a necessary first step before you can remove an owner
from a team. If the team is public, users can join the team without owner intervention, so owners only need to add or remove guests. It is also sensible
for team owners to keep an eye on the team membership in case someone joins that should not be there.

Tenant administrators and other users assigned with user management permissions can update team membership even if they are not a team owner. A
variety of methods is available to do the job:

The Azure Active Directory console . Update the membership of the Azure Active Directory group for the team.
The Office 365 Admin Center . Apart from tenant administrators, accounts assigned the User Management Administrator role can update the
membership of Office 365 Groups, including those enabled for Teams.
The Exchange Administrator Center . Accounts assigned the Exchange administrator and User Management Administrator role can update the
membership of Office 365 Groups through the EAC.
PowerShell . You can run the Add-TeamUser or Add-UnifiedGroupLinks cmdlets to update team membership. Given the choice, use the Add-
TeamUser cmdlet because this mimics what happens when you add a user through a Teams client. See the discussion later in this chapter about
using PowerShell with Teams. For now, this example shows how to add a user to a team.

[PS] C:\> Add-TeamUser -GroupId (Get-UnifiedGroup -Identity "My Office 365 Group").ExternalDirectoryObjectId -User Donald.Vickers@Office365itpros.com

Because of the need to synchronize directories across Office 365, users might experience a short delay before Teams picks up the new membership
data and they can access a team.

Unknown Users : If you see an “unknown user” listed in team membership, it refers to someone whose account has been removed from the tenant
directory., Because Teams cannot resolve the member against the directory, they are deemed to be unknown.

Teams Messaging Policy

New users pick up the default Teams messaging policy set for the tenant. The Teams and Skype for Business Online portal allows you to assign
different policies to specific users. For example, you might want to let most people use personal chats but limit some users to channel conversations.
You can do this by defining a new Teams messaging policy through the portal that restricts access to personal chats and assigning it to the selected
users. To assign a policy, select the user in the Users section and click the Edit icon for Assigned policies . If None is shown for the Teams messaging
policy, it means that the user is governed by the default (global policy). Click the down arrow to see a list of available policies and select the policy with
the restricted setting. Save the assignment and the next time the user signs into Teams they will pick up the assignment.

Teams also supports the assignment of policies for meetings and live events. These policies are discussed in Chapter 16.

Using Teams as Distribution Lists

Teams uses Office 365 Groups to manage team membership. An Office 365 Group is a mail-enabled object, meaning that people can send email to the
group. When this happens, the messages end up in the Inbox folder of the group mailbox. However, team members do not receive copies of messages
because they are not part of the group’s subscriber list. This is by design because Teams has its own non-email-based conversations and for this reason,
when a you create a new team, Office 365 sets the AutoSubscribeNewMembers property for the group to False and does not populate the subscriber
list. See Chapter 12 for more information on controlling email distribution sent to groups.

Normally, no issue arises because the group’s subscriber list is empty. That is, unless you want to be able to use the team to distribute email to all or
some of the members by using the team in the same way as a distribution list. To do this, you must add the members who you want to receive email (or
calendar requests) sent to the team to the group subscriber list. You can only do this through PowerShell. For example, this code adds two users (who
must already be members of the group) to the subscriber list.

[PS] C:\> Add-UnifiedGroupLinks -Identity SeniorLeaders -LinkType Subscribers -Links Jack.Healy@Office365itpros.com, Marc.Vigneau@office365itpros.com

Enabling Existing Office 365 Groups for Teams

If you already use Office 365 Groups, you can team-enable those groups. In many ways, team-enabling a group is an interesting decision because it
switches the focus for conversations in the group away from email to channel conversations. There is no way to migrate existing conversations in an
Office 365 group into channels, so the switch is a one-time all-in operation. However, the files stored in the document library belonging to the group
remain available to the team.

Groups can only be team-enabled by their owners. To begin, a group owner goes through the normal process to create a new team, but instead of
creating a new team, they select Create a team from an existing Office 365 group (see Figure 13-9 ). This choice only appears when the user is an
owner of one or more Outlook-based groups that are not yet team-enabled (Yammer-based groups are unsupported). Click the link to expose the set of
groups owned by the user that are not team-enabled. (Figure 13-12 ). As the UI only shows six groups at a time, it might be necessary to scroll through
in the list to find the group that you want to upgrade. Among the list you will find groups that are hidden from Exchange address lists (because Teams
does not use address lists). However, groups with hidden membership do not appear. Select the group that you want to enable for Teams and click
Choose team to begin the process to populate the team resources and properties for the group. You cannot team-enable several groups at one time.

Whether you create a team from scratch or team-enable an existing group, you end up in the same place with a team-enabled Office 365 group. The
difference between the two methods is that team-enabling an existing group automatically makes the team available to the current group members
because Teams and Groups share the same membership.

A group owner must make the decision to enable the group for Teams. There is no way for a user to search for an Office 365 group that is not team-
enabled and ask its owner to upgrade it. A group owner does not have to be allowed to create new teams to be able to team-enable an existing group.
After you team-enable an Office 365 group, you should consider hiding the group from Exchange clients to force users to use Teams for conversations
(see below).

Figure 13-12: Selecting an existing Office 365 Group to create a new team

Dynamic Teams

You cannot create a team with dynamic membership using a Teams client or with PowerShell. Instead, you create a new dynamic Office 365 group
through the Azure Active Directory portal, add the query to generate the membership, and then select the group when creating a new team as
described in the last section. When it enables the group, Teams uses the membership as calculated by Azure Active Directory. If you examine the
membership (using Manage team ), you see the same membership there as you see in the Azure Active Directory portal. However, if you view team
membership you see a banner to tell that you cannot add or remove members. In addition, the normal Add Member button is not shown.

Team owners are static members of the group and the information about them is stored separately to the set of members computed by the dynamic
query. You can add team owners to the dynamic group through the Teams client, the Azure Active Directory portal, or with PowerShell. You can change
a member in the dynamic set to be an owner. However, if you demote an owner to become a member and they are not in the set computed by the query,
they lose their membership of the team.

You can also use a dynamic group as the source to add members to another team. In this case, just like a regular distribution group or Office 365
group, Teams reads the present membership and adds them to the team. And just like when you use other types of groups to populate membership for
a team, this is a one-time operation and anyone who joins the source team thereafter is not automatically added to the other team.

See Chapter 12 for more information about dynamic Office 365 Groups. Dynamic teams depend on the dynamic groups feature in Azure Active
Directory. Microsoft considers this to be a premium feature. Every member who comes within the scope of a query used for a dynamic team must have
an Azure Active Directory P1 license.

Hiding Teams from Exchange Clients

Teams and Office 365 Groups use the same membership service and a group that is team-enabled shares the same membership. Because it can be
confusing for users to decide whether to communicate via conversations in the Office 365 Group and a team channel conversation in the same group,
Microsoft decided to hide the existence of groups created by Teams from Exchange-based clients (OWA, Outlook desktop, and Outlook mobile).

The creation of a new team sets a flag called HiddenFromExchangeClientsEnabled in the group properties to $True, which then forces the
HiddenFromAddressListsEnabled property to be set to $True . The net effect is that the new group functions perfectly well inside Teams but is not
available in Exchange clients, does not appear in address lists like the GAL, and is not a candidate for discovery by users.

The HiddenFromExchangeClientsEnabled flag is not set retrospectively for groups belonging to teams prior to its introduction. If you want to hide a
group used by a team from Exchange clients, you can do so by setting the property with the Set-UnifiedGroup cmdlet. For example:

[PS] C:\> Set-UnifiedGroup -Identity Team1 -HiddenFromExchangeClientsEnabled

And to see all the groups that are hidden from Exchange clients, we use the command:

[PS] C:\> Get-UnifiedGroup -ResultSize Unlimited | ? {$_.HiddenFromExchangeClientsEnabled -eq $True} | Format-Table DisplayName, Alias

Exchange clients periodically refresh their list of groups from the server, so it might take between 15 minutes and an hour before a hidden group
disappears from a client. When you hide a group from Exchange, you also remove it from user Favorites lists. Another side effect is that because Teams
has no user-accessible directory, once a group disappears from address lists, users won’t be able to browse groups in the GAL and so won’t know of the
existence of public teams that they might want to join (or private teams, for that matter). Also, remember that most Outlook desktop clients depend on
the OAB and that it takes some time for Exchange to publish the removal of groups in OAB updates that must then be downloaded and applied by
clients. See Chapter 3 in the companion volume for information about OAB updates.

Setting the flag one group at a time is acceptable if you only have a few groups to process. If faced with hundreds of groups to process, the best
approach is to create a PowerShell script to search for teams-enabled groups which are candidates to hide and then call the Set-UnifiedGroup cmdlet
to set the HiddenFromExchangeClientsEnabled flag. You can find an example script here .

Using a Team-Enabled Group as a Distribution List

Before rushing to hide the groups used by Teams from Exchange clients, consider that a group also functions as a distribution list and it can be useful
to use the group for that purpose, even when you conduct most discussions inside Teams. For example, you can’t send an encrypted message to Teams,
but you can send a protected email to the group and the members will receive and be able to read the content in the message (if they subscribe to the
group).

It's easier to make the decision to hide a team when its communications are exclusively internal. Making the call for a team with guest members is
harder. When you have a team with many guest members and want to send something important and time-critical to those members, you could post a
message in Teams, but each guest then needs to switch to your tenant to read the message. An email to the group might be quicker and more effective
if you need fast action.

Even when a team is hidden from Exchange clients, you can send email to the group by using its SMTP address through the Office 365 Admin Center
or PowerShell. For example:

[PS] C:\> Get-UnifiedGroup -Identity MyOffice365Group | Select PrimarySmtpAddress

Administrators can find this address relatively easily, but users might not be so lucky.

If you create a team and decide that you want to keep its ability to act as a distribution list, you can reveal it to Exchange by setting the property to
$False:

[PS] C:\> Set-UnifiedGroup -Identity MyTeam -HiddenFromExchangeClientsEnabled:$False

Managing Settings for a Team

Team owners can manage team settings through in the ellipsis […] menu found beside the team name. Select Manage Team from the menu to access
the available settings:

Members : List existing team members, change roles (from Member to Owner and vice versa), remove members from the team, add new
members. When you add or remove members to a team, Teams updates the group membership.
Pending Requests . Process any waiting requests to join the team. This tab is only visible for private teams.
Channels : Add new channels to the team. You can also gain an insight into the level of activity for the channels as you can see the number of
members who have accessed each channel and the date of the last activity (Figure 13-13 ). As mentioned earlier, if a channel is important and
should be brought to the attention of team members, an owner can check the auto-favorite box to force the inclusion of the channel in each
member’s favorite list.
Settings : Change the team picture, assign permissions to users and guests, allow team members to use @mentions to reference other members,
teams, and channels, and use stickers and memes (including the uploading of new memes). Under Member permissions , you control the actions
users can take within a team, such as whether they can add or remove channels, tabs, apps, or connectors; restore deleted channels; whether
owners and members can delete messages; and restrict who can post messages to the General channel to team owners instead of anyone. For
large teams, you can allow anyone to post but warn them that all members will see the message.
Apps : Add or remove an app for the team. Apps include first-party apps like Forms, Planner, OneNote, SharePoint, and Stream as well as third-
party apps like the Hipmunk bot that are installed in the team.

The ellipsis menu has choices to Delete the team , which also removes the underlying Office 365 group and any other resources belonging to the
group, and Edit team, which you can use to amend:

The team name (display name). When you rename a team, you change the display name for the Office 365 Group. The change therefore affects
Groups, Planner, etc. Because of the need to synchronize directories and clients, the name change might take some time before it is fully effective.
Team description.
Privacy (change the team from Public or Private or vice versa).
Data classification.

These settings belong to the group to which the team belongs. Updating them is the equivalent of running the Set-UnifiedGroup cmdlet to change
group properties. For example:

[PS] C:\> Set-UnifiedGroup -Identity HydraProjectTeam -DisplayName "The New Hydra Project Team"

-Notes "A very nice group of people that is team-enabled to get stuff done" -AccessType Public

-Classification Confidential

Changes made to team settings, including those made with PowerShell, appear for team members to see in the General channel. Note that if you
update group membership through another client or by running the Add-UnifiedGroupLinks or Remove-UnifiedGroupLinks cmdlets, it takes a little
while before the change shows up in the Teams app. This is because Exchange Online must synchronize the change from its directory to Azure Active
Directory and from there to Teams. Microsoft recommends that you manage team membership through the Teams client rather than using other Office
365 management interfaces.
Figure 13-13: Managing channels for a team

List All My Teams

To see the set of teams that they belong to, a user can click the cogwheel icon at the bottom of the Teams list to expose the Manage teams screen. The
set is divided into active and archived teams and lists:

Team Name. Click on the name to go to General Channel for the team.
Description.
Membership (Owner or Member).
People: The number of members in the team.
Type: Org-wide, Public, or Private.
Favorite indicator (click to add or remove team from Favorites list).
More options: Manage team, add channel, add members, leave the team, edit team , get link to team, archive team , or delete the team . The
bolded options are only available to team owners.

Members and Manage Team

The Manage team option displays information about a team and is available to members (including guests) and owners. The ability to take
management actions, like change a setting or add a channel, is restricted to team owners, tenant administrators and teams service administrators, but
members can see:

The membership list, including their title, location, and role.

The channels in the team, including the channel name, description, and date of last activity. The ellipsis menu is available to allow the member to
get the email address for the channel (if enabled), get a link to the channel, or follow/unfollow the channel. If team settings allow members to add,
remove, or restore channels, they can take those actions here.
The apps available to the team. If team settings allow members to add or remove apps, they can do so here.

Deleting (and Restoring) Channels and Teams

By default, any member can remove a channel or restore a deleted channel, but you can restrict this ability to team owners by updating team settings
and unchecking the “allow members to delete and restore channels” setting under Member Permissions . Not only will this prevent accidents (unless
a team owner makes a mistake), but it also stops disaffected employees who might be on the way out of the company doing something silly on the way
to the exit (they might not know that the channel can be recovered). Like channel creation, Teams records an audit event whenever someone removes a
channel and writes a notification into the general channel. This information tells you who removed the channel, which might be cold comfort when you
consider how much information might be lost in a deleted channel.

Removing a Channel
You cannot delete the General channel for a team, but you can remove any of the other channels. An audit entry records the deletion and Teams notes
who removed the channel in a note to the General channel. When someone deletes a channel, Teams puts the channel into a soft-deleted state and
starts a 21-day countdown, after which the messages and metadata for the channel become irrecoverable. During the 21-day grace period, you can
recover the channel by selecting Manage Team and then Channels . Any deleted channels are listed under the set of active channels. To recover a
deleted channel, select it and then click Restore ., Teams then restores the channel in its original configuration and makes the conversations available
to users. One point to note is that the restored channel does not regain its previous status as a favorite, so users must mark the channel as a favorite if
they want it to show in their activity feed.

If the 21-day retention period for a deleted channel lapses, you can still recover messages from a deleted channel If the team is subject to a retention
hold. To do this, run a content search to find the copies of the items held for compliance purposes in the group mailbox and export the found items to a
PST. Although you cannot reassemble the recovered items into the threads they had in the channel, you will be able to email them to a channel or cut
and paste content from the recovered items into Teams.

Teams does not remove the files belonging to a deleted channel from the SharePoint document library. If you want to remove this content, you must do
so through SharePoint. One good reason why Teams does not remove the SharePoint content is that the site might come under the scope of an Office
365 retention policy or individual documents might be assigned retention labels that prevent their removal.
Removing a Team
A team owner (through Teams) or a tenant administrator (using an administrative interface, like PowerShell) can remove a team. When this happens,
Office 365 removes the underlying group and all its resources including its SharePoint team site. For this reason, the owner must confirm that the
deletion of the team should go ahead before Office 365 performs the action. Office 365 also removes the group object, team metadata, and chats if
someone removes the group from another application, such as Planner or Yammer, or by using an administrative interface.

Generally, if you want to remove a team, do this from a Teams client as this action makes the team unavailable to all members at once. After removing
the team from its own directory, Teams synchronizes the deletion of the group object to Azure Active Directory. In turn, the deletion command ripples
across Office 365 through the normal directory synchronization process to instruct applications to remove any resources they manage for the group. It
can take up to 30 minutes before Office 365 completely removes all the resources belonging to a team.

In most cases, unless you absolutely want to remove a team, it is better to leave it in place as a disused team. To do this, remove all members from the
team except a single owner, and set the group to be hidden from Exchange clients as described earlier, and update its display name to include a sign
that the group is not in active use. This approach keeps the team in a state where you can revive it if necessary by simply assigning new owners and
members to its membership.

If you remove a team accidentally, you can recover its group (and the team) using the process described in Chapter 12, providing you do so within the
30-day retention period for deleted groups. When Office 365 recovers a team belonging to a soft-deleted group, the recovery process puts all content
back in place into channels and personal chats and restores any connectors. Any email addresses belonging to channels in the deleted team are
reactivated. Although the object for a recovered team is in the directory soon after a recovery begins, it often takes up to 24 hours before all the
resources for a team are reconnected. At this point, Teams makes the recovered team available to users.

Although you can team-enable an existing Office 365 Group, no method exists to remove a team from a group, which you might want to do to “reset” a
team by removing all channels, including the default channel, removing any tabs, bots, and connectors, and reducing the membership to a single
owner. The only way to reset a team is to remove the group (and all its resources) and recreate it from scratch.

Conversations, Meetings, and Files

For many organizations, the central focus and purpose of Teams is the ability to connect team members through short, interactive conversations. Or as
Microsoft puts it “communicate in the moment and keep everyone in the know .” The individual messages that make up conversations are persistent,
meaning that they continue to exist unless the author or a team owner decides to remove them. Teams is a highly-functional chat application (Figure
13-14 ), complete with all the modern ways to brighten conversations (Likes, emoji, and memes). Likes are an important signal to the Office Graph to
show that someone is interested in a certain conversation. If desired, you can disable the ability to add images to conversations through the Teams
settings in the Office 365 Admin Center.

Conversations can be public or private, but you can’t change a private conversation to be public or vice versa. You control who you communicate with
in private chats by adding people to the conversation. Public conversations occur in channels and are open to any member of the team that owns the
channel. The mechanics of contribution are identical for public and private teams, but it is true that people tend to be more chatty or informal in
personal chats and more formal and perhaps verbose in channel conversations. In either case, participants type their thoughts into the compose box
and then post them to the chat or channel. Or just use the like this message feature (thumbs up) to indicate that you’ve seen and approved of a
message without the need to post a more complex response.

Chats are persistent, and people can leave and rejoin the conversation as they wish. Persistent means that conversations are enduring and do not start
and finish like those in Skype for Business IM. For performance reasons, Teams caches recent chats in memory and always available while clients
might have to page older chats back from storage before being able to display the content. Normally, it takes a client just a few seconds to retrieve and
show old chats.

Figure 13-14: A conversation develops in a channel

Conversations can develop into a voice or video meeting. Click the Meet Now icon under the Start a new conversation box to get the ball rolling.
Any member of the team can join a call launched from a team. Up to twenty people can join a private chat, including the person who starts the
conversation. Private chats can include video or voice conversations and file sharing. The people taking part in a private chat do not have to be
members of any specific team. If someone has not used Teams before, they must connect to the service before they can join the conversation.

It is important to note that you cannot move a personal chat into a channel or convert a channel conversation into a personal chat. Both types are
independent of each other. You also cannot move a chat from one channel to another, either within a team or between teams. If you find that you want
to develop a personal chat into a more general discussion, you can either expand the chat to the maximum number of participants (20) or start a new
conversation in an appropriate channel.

Spell Checking : Teams has its own spell-checking dictionary, which it downloads to %UserProfile%AppData\Roaming\Microsoft\Teams\dictionaries
(you might find several language files there, one for each language you use Teams with). This dictionary is separate to any other used by Office
applications, and you can’t add new words (like technical terms) to it.

Keeping Conversations Focused

Team conversations tend to be “high velocity,” meaning that contributions flow quick and fast. Typical interactions are often brief and use simple
formatting, just like text messaging. Like any conversation, people are most productive and get best results when everyone obeys some simple
guidelines to structure communications.

A common mistake is to create new conversations instead of replying to an existing conversation. This quickly creates a confusing mass of posts that
make it difficult for anyone to understand what happens in a conversation. People sometimes assume that Teams is like email and that it does not
matter where they post. This is untrue. Email can preserve the context of a conversation by including the text of earlier replies, meaning that a
recipient can see how a conversation unfolds by reading those replies. However, Teams conversations do not include earlier replies, so readers can only
see the full context of a discussion if people reply to the conversation and do not split off into new conversations.

Teams Conversation Etiquette : Left to their own devices, it is easy for users to reduce a channel to a confused mass of incoherent conversations.
Just like any other communications medium, it is sensible to give advice to people so that they understand how to extract full advantage from their
contribution. Here are eight simple rules to help keep Teams conversations civil and focused:

Make sure new conversations have subjects : Team conversations do not have a subject by default, but you can add one if you want to
highlight the commencement of a new thread. To add a subject, click the expand compose box to expose the extended editor options. You can now
add a subject, which will appear in bold text to signify that the conversation has changed direction.
Use existing threads when possible : The temptation exists for users to start new conversations to express their ideas when existing
conversations are better locations for their thoughts. Ask people to check if their contribution should go into an existing thread instead of starting
a new one as this will help the channel avoid degenerating into many split and duplicated conversations. In addition, when you keep all the posts
relating to a topic in a single thread, it maintains the full context of the discussion and makes it clear to everyone what is being discussed.
Mark important messages . You can mark a message as being especially important by clicking the exclamation mark when composing the
message or using the CTRL+Shift+1 key sequence. Teams then includes the text “Important!” (red uppercased and bold) in the message and
highlights the message with a red and white exclamation tab as a visual indicator that this is a “must-read” message.
Use likes instead of short replies . If you receive a question in email, you might respond with a one-line answer like “OK” or “Go ahead.” You
can do the same in Teams, but it’s usually better to Like a message to let the author know that you’ve seen and approved of its contents. Apart
from anything else, this also reduces the number of messages in a thread and makes the conversation easier to read.
@mention people when you want a response . Channels can be busy places where it’s easy for people to overlook a question to which you
want them to respond. Highlight the need for specific individuals to respond by using an @mention. And if you expect everyone in a team to
respond, use an @mention for the team. It’s also true that you cannot assume that someone has seen something in a channel conversation unless
you @mention them to force the conversation into their activity feed.
Be Friendly . If you @mention someone, Teams automatically inserts their full display name into the text. You can use the backspace key to
remove everything but their first name, which seems a little friendlier than something like “Kim Akers (Operations).”
Use styles to highlight text . The in-built styles exist to help users highlight important parts of their conversations. You can add headings,
quotations, or code examples.
Use private chats for personal conversations : Teams allows people to share their ideas and viewpoints to anyone who can access a team, but
sometimes it is best to take a conversation to somewhere more private. Personal chats exist for this purpose. Use these chats when small groups
need to thrash something out before making a topic public.

You can save important conversations for quick access later, which is usually an easier way to find something important than searching. To do this,
hover over the right-hand side of the chat (where the Like icon is) and the Save icon appears. Click the icon to save the conversation. The set of
conversations saved by a user are available by selecting Saved in the options revealed when you click your avatar or by selecting /Saved in the
command box.

Making Your Point

When you need to compose more than a simple sentence or two, click the “expand compose box” button to expose a more capable editor that supports
highlighting, font size and color, bulleted and numbered lists, bolding, italics, underlining, and a small set of styles. You can apply basic formatting to
text, such as different headings or marking some text as a quote from a source (you can also use the markdown convention and highlight text as a
quote by prefixing it with the “>” character). There is also a code style to include code examples in a code box (click the </> icon) to help
programmers share ideas with each other. As shown in Figure 13-15 , you can name a code example and choose the language (in this case, PowerShell)
from a list including C++, CSS, HTML, JSON, Markdown (GitHub), Python, Ruby, and TypeScript.
Figure 13-15: Sharing code snippets in a conversation

If you make a mistake in something you write, you can edit the content to fix the error or remove the item if it is no longer valid. Team owners can also
exercise editorial control by removing messages from conversations if the need arises, but they cannot edit a message.

The Teams editor does not offer as much control over formatting as a full-feature editor like Word, but if you need to compose something like a complex
announcement, you can write it in your preferred editor and cut and paste the text into Teams. Text and basic formatting, including hyperlinks, survive
the transition. Specific styles and embedded graphics do not. You cannot copy and paste graphics from other applications directly into a Teams
conversation. If you want to include a graphic, copy it to the clipboard and then paste it into Teams. If you have a complex document that you want to
share and discuss with others, upload it to the channel where you want to discuss the content. Teams stores these documents under Files, but members
can review and discuss the content in the channel.

If you compose text outside Teams and then want to include that text in a conversation, you can paste up to 2,000 words (approximately 12,000
characters) into Teams. Including tables and other complex structures reduces the number of words you can paste. If the message is too long, Teams
tells you to shorten it. You cannot drag and drop a message from one channel to another or from one team to another. If you make a mistake and post to
the wrong channel, you must copy the message and repost to the right place.

Bookmarks
Bookmarks lets users mark important messages in channel conversations and personal chats so that the messages can be quickly found when needed.
Click the bookmark symbol in the read message pane to “save this message.” Later, to display the list of saved messages, type /saved in the command
bar. If you don’t want to bookmark a message but still want to be able to go back to it later, mark it as unread. The message can then be found by
entering /unread in the command box.

Translations
Teams uses the Microsoft Translator service for inline translation of messages in both channel conversations and personal chats. In many cases,
translation is not needed because the members of a team know each other and are able to communicate quite happily without cloud resources. For
larger teams, such as those used to share and invite ideas within a company, being able to express a thought in your native language might reduce the
barrier to collaboration, and that’s why Teams supports inline translation.

Figure 13-16 shows a conversation in three languages – English, French, and Finnish. If the user selects Translate from the ellipsis menu, Teams calls
Microsoft Translator to first detect the language that the message is composed in and then translate it into the language selected in Settings . Guest
users don’t have a preferred language, so the option to translate doesn’t appear for them.

Languages are complex and regional accents, local sayings, idioms, jargon, argots, and technical terms cause problems in translation. Some glitches
might still happen even with simple text, but the translated output is usually good enough to convey the sense of a message. At the time of writing,
Microsoft Translator supports more than 60 languages .

The Translate choice only appears in the ellipsis menu when the AllowUserTranslation setting in the Teams messaging policy assigned to the user is
$True . To make translation available to users, you can either:

Update the global (default) policy by setting AllowUserTranslation to $True.

Create a new messaging policy in the Teams and Skype for Business Online portal and then assign that policy to specific users. New messaging
policies created in the portal automatically set AllowUserTranslation to $True .
Figure 13-16: Teams translates messages in different languages

The AllowUserTranslation setting is not currently exposed in the Teams and Skype for Business Online portal, so you must manage it with PowerShell.
For example, to update the global policy, use this command:

[PS] C:\> Set-CsTeamsMessagingPolicy -Identity Global -AllowUserTranslation $True

To check the policies that have AllowUserTranslation set to $True , use the command:

[PS] C:\> Get-CsTeamsMessagingPolicy | Format-Table Identity, AllowUserTranslation

To assign a specific messaging policy to a user, use a command like this:

[PS] C:\> Get-CsOnlineUser Brian.Weakliam@office365itpros.com | Grant-CsTeamsMessagingPolicy

-PolicyName "Advanced Users"

These cmdlets are part of the Skype for Business Online PowerShell module.

The Activity Feed

A busy channel can have dozens of ongoing conversations, which can make it difficult to keep track of what is happening when someone is a member of
multiple teams. The Teams Activity Feed (Figure 13-17), accessed through Activity in the top left-hand corner, brings important items to the attention
of a user. As people add replies and likes to personal and channel conversations, they appear as notifications in the browser and desktop clients and in
the activity feed for people involved in the conversations. The default view is Team Activity , which means that you see a list of recent activity related to
you happening in the teams to which you belong. For example, someone uses an @mention to ask you a question or to bring something to your
attention. Along with a text snippet for each event, Teams uses different icons as visual clues of why the event appears in the feed. The clues include
@mentions, channel mentions, team mentions, replies, and likes. The number in the red circle beside the Activity Feed “bell” shows how many
channels have recent activity relevant to you.

The Feed menu also offers the My Activity view. In this case, Teams lists recent contributions you made to conversations. The difference in the two
views is that you might not be directly involved in what shows up in Teams Activity while you always are in My Activity .
Figure 13-17: The Teams Activity feed

If you are a member of some busy teams, your activity feed is also likely busy. To help find messages in specific categories, you can filter by:

Unread : Only show unread messages. You can also type /unread in the command box to see unread items in the activity feed.
Mentions : Only show messages where someone has @mentioned you in a conversation. Typing /mentions in the command box has the same
effect.
Replies : Only show replies to your messages.
Following : Only show messages in channels that you follow.
Likes : Only show your messages that someone else has liked.

If someone responds to one of your messages or mentions you in a conversation, Teams generates a notification in your activity feed. However, if the
message is later removed by its author or a team owner, you might see an entry in the activity feed that doesn’t point to anything. This doesn’t happen
very often, but it can.

@Mentions
To make sure that an item shows up in a user’s activity feed, you can address them with an @mention. Three options are available.

Direct @mention : Input the @ sign followed by the name of the team member. For example, @Kim Akers. It is good practice to @mention
someone when you want to bring something to their attention. You can include a list of people, each prefixed by @. You can also use direct
mentions in personal chats.
Channel @mention : Instead of the member’s name, input the name of the channel. For example, @Budgets. Use a channel mention when you
want to highlight something to the team members who have favored that channel.
Team @mention : Use the team name. For example, @Engineering. Use a team mention when you want to bring something to the attention of
everyone in the team.

Team owners can control whether members can use channel and team @mentions through team settings. By default, team settings allow users to use
mentions in conversations.

Toast Replies : When someone sends you a message or replies to you in a personal or group chat (or meeting), Teams sends a “toast” notification to
tell you what’s happening. You can then respond to that user by entering your reply in the toast, but only with the Windows and Mac clients. This is a
great way to send a quick response, but the reply cannot exceed 1,000 characters and can only include text.

When you type an @ sign to create a mention, Teams checks the input against a cache that it creates based on people you communicate with often, the
members of the current team, and the names of the channels. You cannot @mention someone who is not a member of the team hosting the
conversation.
Figure 13-18: Visual indicators in conversations

When someone involves you in a conversation with a mention, a notification appears in your activity feed. You can click the notification to go the
conversation and pick up the thread. Figure 13-18 shows how Teams uses different icons to highlight conversation threads, including a team mention,
personal mention, and an item marked as important.

Unlike most email systems, there is no equivalent of an “all users” distribution list that people can use to notify every account in the organization. If
company-wide announcements are important to you, the closest equivalent is to create a team that has everyone as a member and use that team for
general announcements. If you use dynamic Office 365 groups, this is an easy way to keep the membership updated and ensure that new people
automatically join the team when an administrator creates their account.

Friendly Names : When you @mention someone, Teams inserts their full display name. To make the reference friendlier, use the backspace key to
remove their surname. @Tony is nicer than @Tony Redmond! The same technique works with @mentions in messages posted in Office 365 Groups and
Outlook.

Processing the Activity Feed

Once you’ve worked with Teams for a little while, you’ll probably have a lot of items showing up in the activity feed. In email terms, an overloaded
activity feed is like an overloaded inbox – it’s hard to know where to start processing items, especially if you’ve been away from Teams for a while. One
strategy often used to impose order on an activity fee is:

Be very careful about what channels you follow. Unfollow channels that host discussions of lower importance and try to keep the channels you
follow to the bare minimum. This will reduce the number of notifications you see.
Process direct questions first. If someone @ mentions you, it means that they want to bring something to your attention, so you should scan these
entries and decide which ones need attention.
Check replies to conversations you participate in next. You’ve joined these conversations, so you probably want to find out how they end.
After you’ve cleared items from important channels, questions, and replies, you can leave other channel conversations until you have time.

Searching Teams Content

Teams includes the ability to search messages, people, and files. To search, type something into the Search box and press return. Because this form of
search looks through all content available to Teams, it is likely that many matches result, divided into messages, people, and files. If you do not see
what you are looking for, click the Filter icon to create a more focused search. Filtering supports preset date ranges like “Last month” or “Yesterday”
and can restrict searching to a specific team or channel or look for items with attachments. If you look for Files, you can restrict the search to a certain
file type (for instance, PowerPoint).

Although easy to use, Teams search is not as precise as the search facilities in SharePoint, Delve, or OWA. Sometimes it is hard to focus in on exactly
the right content using criteria such as “messages about Office 365 from three users created between 1-Dec-2016 and 1-Jan-2017 that have a Word
attachment.” This area is a candidate ripe for improvement in future releases.

Removing Messages
Some organizations hate the idea of anyone being able to remove anything they posted electronically (to email, SharePoint, or Teams) and some think
this is sensible. After all, you can always have second thoughts and want to remove something written in error. Teams controls the ability of users and
team owners to remove messages from conversations at a tenant level and within individual teams.

The Messaging Policies section of the Teams and Skype for Business Admin Center includes the tenant-wide controls for message deletion.

Allow owners to delete all messages controls whether team owners can remove messages in any channel in the teams they own.
Allow users to delete their own messages controls whether people (including guests) can delete messages that they post.

The intention behind owner control is to allow them to police conversations within a team and be able to remove anything posted that is inappropriate,
insulting, or otherwise objectionable. Within a specific team, an owner can select Manage Team from the ellipsis menu, and then Settings to access
the settings for to that team. Navigate to Member Permissions to select whether:

Everyone can edit their messages.

Everyone can delete their messages.
 Owners can delete all messages.

The settings apply to all channels within the team. If the tenant-wide settings do not allow users to remove or edit messages, Teams hides these
settings at the team level.

When someone removes a message, Teams flags it with “This message has been deleted. ” Contributions to the conversation before or after the deleted
item are unaffected. If someone removes a message from a channel, they can restore it by clicking Undo. Copies of messages captured for compliance
purposes are not removed and continue as evidence that a conversation occurred.

Although you can remove individual messages from personal chats or channel conversations, you cannot remove a complete thread (all the messages
that compose a conversation). Attachments pose another issue. When someone includes an attachment in a message, Teams uploads the file to that
user’s personal OneDrive for Business site (for personal chats) or the channel folder in the SharePoint document library belonging to the team. If the
team owner or author later removes a message that has an attachment, Teams removes the message but leaves the attachment intact. I can understand
the logic of leaving the file in place, but it might be the case that the attachment holds the objectionable content that you want to eliminate. If so,
checking and cleaning up SharePoint and OneDrive content are extra steps in the removal process.

Searching for and removing content from multiple teams is another issue. This might happen if someone posts something inappropriate to one or more
teams or people copy content posted to one team into others. To purge the content, you then must find and remove each post individually. Unlike
Exchange’s Search-Mailbox cmdlet or the purge action available for content searches, Teams does not have a method to scan all teams in a tenant to
find and remove specific content.

When Members Leave

Team conversations are persistent, so they exist even when people involved in conversations leave the organization. The contributions of removed
users to private chats and conversations in channel stay in place. The user no longer exists in the membership list (or the tenant directory), which
means that other members can no longer @mention the user. Because their account is no longer in the tenant directory, the removed user shows up as
an “unknown user” as a participant in private chats. However, Teams keeps their display name for their contributions in the conversation history.

If you restore a user account within 30 days of deletion, they regain their membership of groups and teams and appear as normal.

Personal Chats
So far, we have focused on using channels within teams as the basis for conversations. Channel conversations are under the control of the organization
because channels only exist inside teams. By comparison, personal chats occur when two or more people converse on an ad-hoc or persistent basic and
are under the control of the participants, who decide when to start the conversation and who should take part. You cannot “promote” a personal chat to
move the conversation to a channel in a team.

Personal chats are either one-to-one (1:1), meaning that two people take part, or group chats, meaning that more than two people are involved. Both
kinds of personal chats are accessed through the Chat section of the Teams client and their content is confidential to the participants. Like
conversations in channels, personal chats are persistent and are always available.

Although you cannot drag and drop personal chats to arrange them into a preferred order, you can mark important chats as favorites by selecting the
Pin option from the ellipsis menu. Pinned conversations appear at the top of the chat list.

Starting Chats
To start a new chat, click Chat , right-click, and then New Chat (or click the New Chat icon in the menu bar). Teams invites you to start typing a name
or group to chat with in the To: line. You can add the following:

Tenant accounts.
Guest accounts.
People in other Office 365 tenants by entering their User Principal Names (as explained below, these people can only participate in 1:1 chats).
The name of an existing group. In this context, “group” means the name of an existing group chat rather than an Office 365 group or a distribution
group. If you want to chat with a team, you do so within a channel in that team.

If you start a new chat with a set of people with whom you have already chatted, Teams recognizes that a chat with those people exists and displays the
messages from the earlier discussion to allow you to continue from that point.

You can also start a chat or continue a conversation with someone by typing @ and their name in the command box, followed by the message you want
to send. This approach works well when you want to send a quick note without switching context away from something else, like a meeting or
discussion in a channel. You can’t address more than one person through the command box.

Posting Files
Posting a file to a personal chat is exactly like posting to a channel. If you include a file in a conversation, Teams uploads the file to a folder called
Microsoft Teams Chat Files in your personal OneDrive for Business site and shares the file with the other users in the conversation. If you make a
mistake and share the wrong file or share a file with the wrong set of people, you must remove the file from OneDrive as Teams does not offer the
choice to remove a file from a conversation.

Group and Personal Chats

Group chats involve more than two people. Conversely, 1:1 or personal chats occur between two people. The two participants in a personal chat can be
accounts belonging to the tenant, between a tenant account and a guest account, between two guest accounts, or between a tenant account and a
Teams user in another Office 365 tenant. Guest users can also create new personal chats and include other users and guests in the conversation. In
fact, a guest can create a chat that only guest users join. Anyone taking part in a group chat can add someone else to the conversation by clicking the
Add people icon in the top right-hand corner, which you might do to include an expert to answer a question that has come up in the conversation.

When you start a group chat, you can give it a name (Figure 13-19 ) to let people know what topic you want to discuss. It is always good to assign a
name to group chats as it makes it easier to find the right chat the next time you want share a thought. If necessary, anyone taking part in a chat can
rename it at any time to reflect the current scope and focus of the conversation. One-to-one chats use the name of the person with whom you chat.
Figure 13-19: Creating a new group chat

You cannot add a guest to a chat unless their account exists in the tenant directory, meaning that they must have previously been added to a team.
When you add someone to a group chat, you decide whether they can see all or some of the prior message history. For instance, if you bring someone
into a conversation to answer a specific question, you might allow them to see messages from the last five days so that they understand the context. If
you have shared any documents with people in the conversation, you must share the documents stored in your OneDrive site with people who join a
conversation if you want them to be able to access this content.

Personal chats behave differently to group chats in that you cannot reveal previous messages with someone you add to a 1:1 chat. The reason is simple.
1:1 chats are only shared with the other person in the conversation. Chats are persistent and so if you could reveal prior messages to someone joining a
conversation, you might impinge on the other person’s privacy by revealing something that they prefer to keep confidential. An argument can be made
that people should control their own data and decide how to share conversations, but in this case, Microsoft decided that it was best to limit the
sharing of prior messages in a conversation to group chats.

Leaving and Muting

You cannot force someone to leave a conversation. If you make a mistake and discover that the wrong set of people are in a group, you can close the
group down and restart the conversation inviting the right set to join. Participants who do not want to be in a group can leave it at any time by
selecting it from the list of personal chats and then Leave from the right-click menu. When you leave a chat, you no longer see new messages posted to
the chat, but you can still see any message posted up to the time you left. Alternatively, if you simply want to remove a chat from the list of active chats,
you can Mute the chat (another choice in the right-click menu). The chat is still there, and you can take it up again at any time, but notifications for
new content in the chat do not show up in your activity feed.

Adding Tabs to Chats

Users can add tabs to group or 1:1 chats. For instance, you might decide that you want to discuss a web site or a PDF file in a web site. Click on the
plus sign in the menu bar to add tabs for:

Documents that you have shared in the conversation (Word, Excel, PDF, PowerPoint)
Link to videos stored in Stream.
Link to a web site page.
Link to Power BI report (such as the Office 365 usage analytics).
Apps authorized by the tenant.

The intention behind adding tabs to a conversation is to give fast access to information being discussed or needed for the conversation. Participants
can quickly move from the conversation to a tab and back again without pause. Teams opens objects as soon as the tab is accessed to make it even
easier to access the content.

Chatting with People Outside the Tenant

Users can chat with people enabled for Teams in other Office 365 tenants if:

External access is enabled in the organization settings for the tenant.

The remote user’s domain is not blocked by your tenant.
The remote user’s tenant also allows external access.
The remote tenant does not block your tenant.

With these conditions in place, you can enter the full User Principal Name for the remote user to set up a chat. After you send a message, Teams will
try to make a direct connection to Teams in the other tenant. If Teams in the remote tenant reports that the user is unavailable (offline), Teams sends
the message by email. During this kind of chat, Teams flags that you’re chatting with a remote person with the banner shown above. You won’t be able
to see the remote person’s presence, nor can you share or your desktop files with them or invite another user to join the conversation.

Teams only supports 1:1 chats with remote participants. If you want to host a group chat with more than one remote participant, consider:

Adding the remote participants as guest users in your tenant. The advantage is that in addition to being able to join in chats with multiple people,
the guest users can access resources belonging to the teams in which they are members. The disadvantage is that your tenant might not want to
add these users as guests.
Create a meeting and invite the remote participants to use anonymous join to connect. This is the quickest and easiest solution.

Meetings
You can schedule meetings with the Teams app or the Teams add-in for Outlook. Three types of meetings exist inside Teams:

Private: schedule private meetings through the Meetings section in the navigation pane or by using the Teams add-in for Outlook for Windows.
Like personal chats, you dictate who receives invitations to join the meeting, including guests. Up to 250 people can join a meeting. Private
meetings include the ability to chat to allow participants to share information pertinent to the meeting such as an agenda or a list of action items.
 If a participant cannot join the video or audio part of a meeting, they can still contribute to the conversation. Teams considers meetings created
with the Teams add-in for Outlook as private, even you add a team as an attendee for a meeting. You can also invite people outside your
organization to a meeting, even if they are not a guest, by adding their email address as a participant. This is known as an anonymous join and
generates an invitation to the email address that allows the recipient to join the meeting. Anyone joining a call from outside your organization
enters through a virtual lobby that flags waiting participants to meeting organizers and allows them to control their entrance until everything is
ready for external people to join.

Team: these meetings are like personal meetings in that you create them through Meetings . The difference is that instead of selecting
individuals to take part, you also select a channel within a team. This step advertises the existence of the meeting in the channel and allows any
member of the team to join the meeting. Any side conversations that happen between participants during the meeting show up in the channel and
are visible to all members, including those who do not take part in the meeting. Logically, you can only schedule team (channel) meetings with
members of a team to which you belong. You can also invite specific people from a team to a meeting, but you cannot invite people outside the
selected team.
Ad-hoc: are like team meetings in that they occur in a channel. To start an ad-hoc meeting, click the Meet Now icon in the compose message box.
Team members who are online can join the meeting if they wish.

When you schedule a personal or team meeting, a meeting invitation goes via email to participants like other meetings you might organize with
Outlook. Recipients can then decide to accept or decline the invitation. Like Outlook meetings, Teams has a scheduling assistant to help find the right
time for a call. You can also schedule recurring meetings by selecting the Repeat checkbox.

Meetings appear in the personal calendars of those who accept the invitation and show up in the Meetings section of the Teams client, along with other
personal meetings synchronized from the user’s Outlook calendar. If it is a team meeting, it also appears in the group calendar for the team. If
reminders are set for meetings, Outlook notifies users when meetings are about to happen. Teams does not have a notification mechanism of its own,
so users need to either join meetings from these notifications or keep an eye on the meetings listed in Teams through the Meetings section or the
meeting details in a channel. Participants can join with the desktop and mobile clients or the browser client on Edge or Chrome. Meetings use the same
communications stack used by Skype consumer, so the audio and video controls available inside a meeting are familiar.

Although more than four team members can take part in a meeting, Teams only shows the video stream from the four most recent speakers. Like Skype
for Business Online, Teams video calls support desktop sharing (including the ability to only share a specific window rather than the entire screen),
recordings, muting noisy attendees, and transfer control to other participants so that they can run a meeting. If you record a meeting, the audio, video,
and screen sharing activity is captured and processed by Microsoft Stream. The recording is then available for playback from Stream.

Outlook add-in for Teams Meetings : Users of Skype for Business can schedule a Skype meeting from Outlook. Microsoft also has a Teams add-in
for Outlook . The add-in allows users to schedule a meeting for specified members within a team or to join a scheduled team meeting. Outlook enables
the add-in automatically if it detects that the Teams desktop client is on a workstation, but the add-in only appears if you sign into Outlook and Teams
with the same account.

Teams and the Phone System

Teams is the preferred communications solution for Office 365 and supports two methods to enable users to make, receive, and transfer calls to
landline and mobile phones connected to public networks (PSTN). You can buy Phone system licenses with a calling plan (various plans are available in
different countries for national and international calls), in which case the phone numbers assigned to users come from a pool assigned and managed by
Microsoft. Alternatively, you can use Direct Routing (or direct connect) to link Teams to your existing connection to the PSTN network via a Session
Border Controller (SBC). The technology for Teams calling comes from the Microsoft Phone System (originally called Cloud PBX), which is also used by
Skype for Business Online. The features supported by the two platforms is different and will continue to evolve as Teams builds out its calling
capabilities. Even if you don’t have a calling plan, Teams includes a Calls button in the left rail for all users to allow people to make VOIP calls to other
Teams users.

Figure 13-20: The Teams Calls interface

In Figure 13-20 , we see a Teams client positioned in the Calls section, where you can assign different contacts to the speed dial or to other groups to
make them easier to call. Only one of the six users listed as speed dial contacts is online. Originally, the Calls button only appeared in Teams when the
user had a license for a calling plan.

See Chapter 16 for more information on how to connect Teams to the Phone System and calling plans.

Skype for Business and Teams Interoperability : Microsoft’s announced direction is to phase out the Skype for Business Online client and replace
it with Teams. During the transition period as Microsoft builds out the calling capabilities of Teams to match and surpass those available in Skype for
Business Online, it is important to have solid interoperability between the two platforms. Users should be able to converse with, call, and see the
status of other users without regard to the platform currently in use. As this is an evolving story, you should consult Microsoft’s interoperability guide
for the latest information about how to configure the two applications to achieve maximum interoperability.
Viewing Organizational Information
If the Show organizational chart in personal profile tenant setting for Teams is On , users can view the organizational information about other
users by clicking their avatar in a chat or searching for their name, and then selecting the organization view. Or, if you hover over someone’s people
card, you can see basic information about the person (their address, phone number, title, and if set, their email autoreply or out of office message)
together with the ability to:

Begin a personal (1:1) chat.

Send a quick message to the person (the message shows up in your chats).
Email the person. In this case, Teams launches the default email program configured for the workstation (for example, Outlook) with a new
message addressed to the person using their email address defined in their Azure Active Directory account.
Call the person.
Start a video chat with the person.
View organization information about the person.

The organizational information about a person (Figure 13-21 ) comes from the reporting relationships, job title, and other information held in Azure
Active Directory. Obviously, the information is only as good and as accurate or complete as it is in the directory, so it’s a good idea to invest effort in
populating and maintain the directory.

If incorrect or incomplete information about management relationships exists in Azure Active Directory, it is impossible to get much insight into the
organization through Teams. An inconsistent directory also makes it much harder to use features like dynamic Teams which depend on queries run
against the directory. To ensure that up-to-date information is available to end users, some companies use their HR system as the directory of record
and have a direct feed between it and Azure Active Directory, with updates in reporting relationships, job titles, phone numbers, and so on being
transmitted periodically, while others use third-party products like Hyperfish to find and fix problems in Azure Active Directory. For instance, you might
find inconsistent use of account properties like city or state and province. Another frequent problem is missing values for properties like department or
office.

Figure 13-21: Viewing organizational information about a user

Although you can edit the contact information section of a user’s account through the Office 365 Admin Center to update properties like their job title,
department, office, phone number, address, and so on, you can’t update a user’s reporting relationship in this manner. Instead, use either:

Azure Information Directory portal : Select the user’s account and update the job title and manager properties in the Job Info section. Update
contact information (phone number, address) in the Contact Info section.
Exchange Administration Center : Select the user’s mailbox and update their manager in the Organization section. You can also update their
contact information in the Contact Information section.

Changes made in EAC synchronize to Azure Active Directory and are then available to Teams. Guests do not have access to organizational information.
They can see someone’s title, work address, and phone number, but the org chart icon does not appear on the people card.

When you view information about a person by, you can also see details of personal chats you have had with them (Conversation), if you share files from
your OneDrive for Business site with them in a 1:1 chat (Files – documents shared in group chats do not show up here), and recent posts from them to
channels in teams to which you belong (Activity). Guests can only see the Conversation and Activity tabs.

The Teams mobile client also exposes organization information through an icon on the home screen. A user can click Organization and then search by
name or follow the organization from their own place in the structure to discover others and their position. Another way of navigating the organization
structure is to use the WhoBot, discussed in the section on bots later in this chapter. WhoBot can track connections between users in a tenant, but it
still depends on an accurately-populated Azure Active Directory to understand the organization structure.

Files: The Link Between Teams and SharePoint

Online
When you create a team, Office 365 provisions a SharePoint team site for the team and creates a document library within that team site, along with a
folder for the General channel. The purpose of the folder is to hold documents about channel conversations. The creation of a new channel in a team
also creates a new folder in the document library, which takes the name of the channel. However, if you rename the channel afterwards, the folder
keeps its original name. Each folder also has a sub-folder called Email Messages where Teams stores copies of email sent to the channel (if it is email-
enabled). Teams creates this folder the first time someone sends an email to the channel.

Users access documents in the library through the Files tab for a channel, which brings them to the folder created to hold channel files. They can
upload documents to the folder or drag and drop files from Windows Explorer or the Mac Finder to move copies into the folder. Naturally, any files
created in the folder through the SharePoint browser interface are also available. Users can edit Word, Excel, and PowerPoint documents within Teams
or open the file with the Online version of the app. The editing experience is similar but opening the file with Office Online or the desktop application
allows the user to continue to do something else with Teams during the edit.

Users can navigate to the Email Messages folder or any other sub-folder belonging to the channel, but if they wish to move to another folder in the
document library, the user must click Open in SharePoint to use the regular browser interface. They can then access other folders in the document
library, such as those belonging to other channels, or, if you team-enable an existing group, folders previously created in the library. An easy
workaround to this problem is to create a tab pointing to the root folder in the SharePoint documents library for the team as described below.

SharePoint Tabs
To create access to other SharePoint resources from within Teams, you can add two different SharePoint tabs to bring users direct to a specific folder
in the document library, another document library in the tenant, or a page in the SharePoint site belonging to the team. Here’s how:

Select the channel you want to make SharePoint resources available to team members, and then click the plus sign (+) to invoke the Add a tab
dialog. Now select the Document library tab to add a link to a document library or folder or the SharePoint tab to link to a page in the team
site.
If you select the Document library tab, you can now select one of the Relevant sites (essentially, a list of the sites that you accessed recently) and
then a document library from that site; or select Use a SharePoint link and input the URL to the library or folder you want to make available
through the tab. The easiest way to get the URL is to open the target location in a browser and copy the URL from there to Teams. Make sure that
team members have the necessary permissions to open the target. If the target is a document library, you select a specific folder within the library
to act as a starting point for the tab.
Give a name to the tab to help users understand its purpose (you can rename the tab later if necessary) and then Save .
If you select the SharePoint tab, Teams retrieves a set of available pages to choose from. Select the page in the team site to display (for example,
Home or News) and Save the setting. The tab will take the name of the chosen page (you can rename the tab if you want to).

Figure 13-22: Working with Files in a SharePoint document library

Figure 13-22 is an example of linking Teams to a document library. In this case, you can see the source Word documents for this book in a SharePoint
library using a custom tab. Although the options available to work with files through Teams are a subset of those available in the SharePoint browser
interface, many find that it is easier to work with documents through Teams. The most common actions are available to open, move, copy, download,
and delete files, which allows users to get on with what they need to do with documents. Document co-authoring also works because Teams includes
the WOPI and FSSHTTP protocols, as well as the ability to call online viewers for more than 300 file types.

An example of linking to a specific page is when you want to publish news articles. You can bring news articles into Teams as cards created by the
News connector, but if you link to the News page for the site, you see all the news items posted, including the web parts (images, etc.) used for each
item, and can comment on an item. In other words, you can interact with the page instead of just seeing a static snapshot of the content.

Most of the issues that people have with working with files through Teams revolve around the restrictive nature of the user interface. For example, you
cannot access document properties through Teams, which means that you cannot assign a label to a document unless you open the folder in
SharePoint. This might mean that some documents do not get the labels they need for retention or compliance purposes. If this is a concern, make sure
that users receive advice about how to apply labels (see Chapter 19) to documents or use auto-label policies. Another difference is that Teams applies
its own view when it displays files from SharePoint; if you want to see customized views with custom fields or use filters with SharePoint content, you
must open the site in a browser. Finally, the OneDrive sync client does not support synchronization of SharePoint content when you sign into a tenant
as a guest, so guest members in a team cannot download SharePoint files to work with documents offline.

Linking to Files
The Get link option exposes two links that you can copy and use.

The Microsoft Teams link is a deeplink (URL) to the channel that owns the tab. For example: https://teams.microsoft.com/_#/tab%3A%3A5f159e8c-
3a36-47b1-970a-7dfa22743cfa/General?threadId=19%3A09151162b14b4feab4cd8af27b4047a1%40thread.skype&ctx=channel
The SharePoint link is the URL pointing to the folder in the document library that is the target for the tab.
One way of using the channel link is to add it to as a new navigation link for the SharePoint team site so that users can quickly switch back to Teams
when they work with SharePoint through a browser. When the user clicks the link, a new browser tab opens and loads the Teams web client positioned
in the conversations for the target channel.

Apart from tabs linking to SharePoint sites, team owners can add tabs to a channel for fast access to a specific document stored in SharePoint. Open
the SharePoint library with Teams, select the document, and then Make this a tab . Alternatively, add a tab and then select the file type (Word,
PowerPoint, etc.), and then select the document you want from the document library belonging to the team (you cannot select a file from another
document library). When you create a tab for a file, selecting the tab invokes an Office viewer to display the content of the file. You can also edit the file
in either the online app or by launching the desktop app, if it is available on the workstation.

The Files link in the navigation pane is different to the Files tab for a channel. Instead, the default view for this link lists the recent files worked on by
the user stored in SharePoint sites and their OneDrive for Business site. Some other shortcuts appear under Files:

Recent is a list of files in SharePoint document libraries belonging to teams that the user recently accessed. Guests do not see this link.
Microsoft Teams shows a list of files, including email messages, recently loaded into teams to which the user belongs.
The OneDrive link exposes a list of folders and files stored in the user’s OneDrive for Business site. Guests do not see this link because they do
not have a OneDrive site in the tenant.
Downloads shows the files in the Downloads folder on the workstation.
If you connect a cloud storage service to Teams, like Google Drive, a shortcut to that service appears here.

OneNote
When you create a tab in a channel to access OneNote, you have the choice to create a new notebook, browse to find existing notebooks to link to the
tab, or paste a notebook link to bring users to a specific notebook, or a section, page, or paragraph within a notebook. This behavior is more flexible in
the past because it does not depend on the shared notebook owned by the underlying Office 365 group.

Sharing Files
When someone shares files in a public conversation, Teams captures copies of the files in the folder of the group document library for the channel.
However, when someone shares files in a private chat, Teams first uploads the files to a folder called “Microsoft Teams Chat Files” (under the Files
folder) in the owner’s OneDrive for Business site and then shared with the other chat participants.

SharePoint News Connector

SharePoint news items are a special form of web page created in a SharePoint site. The SharePoint News Connector creates notifications about news
items in a channel after they are posted. You can only create notifications for news items posted to the SharePoint site belonging to a team. News items
posted to other sites are ignored. After the notification messages appear in the channel, users can click on a message to go to the full news item in the
SharePoint site or use the item to begin a conversation.

To create a connector, select Connectors in the ellipsis menu for the channel where you want the notifications for news items to appear, search for
SharePoint News , and then click Configure . Teams then creates the webhook to check for news items posted to the site. Click Save to set up the
link.

Cloud Storage
If allowed by the Teams settings for the tenant, the Files tab supports access to different cloud services such as Dropbox, Box, ShareFile, and Google
Drive (including personal drives) that the tenant might use instead of OneDrive for Business or SharePoint. The process of connecting to a cloud
service means that you provide credentials to connect to an account in the service and use that account to select a folder in the storage accessible to
the account. Once connected, a link to the cloud service appears in the list of files. Users must be able to authenticate with the service and access the
files in the chosen storage location. Once connected, you can view, edit, upload, and remove files stored in the cloud storage. You can also start a new
conversation about a file in cloud storage, just like you can discuss files stored in SharePoint.

Guest Access for Teams

The mechanics of external access for Teams function in a comparable manner to those for Office 365 Groups as explained in Chapter 12. Team owners
add guests to teams, invitations to join go to the email address that identify the guests, who then redeem the invitation to complete the process and
join the team. As part of the Azure AD B2B collaboration process, Teams creates guest accounts in the tenant Azure Active Directory to allow guests to
authenticate their access to resources. The Invitation API generates the invitations and Teams manages the redemption process, including the
verification of the guest account. You can invite anyone to be a guest, if they have a valid email address.

Once a guest has joined a team in a tenant, they don’t have to accept invitations to join other teams through the links contained in the emailed
notifications. Instead, the Teams apps detect that the guest is now part of other teams and perform an in-app redemption to add those teams to the list
available to the guest.

A user of the free Teams version can be a guest in a tenant that uses the enterprise version and can switch between free and enterprise tenants. The
same is true for users belonging to enterprise tenants, who can switch to guest accounts in free tenants.

Enabling Guest Access for Teams

Because Teams consumes multiple Microsoft cloud services, different settings in the services combine to control guest access. In order of priority,
these are:

1. Azure Active Directory . Under User Settings , Admins and group members must be able to invite guests (people with email addresses that do
not belong to your tenant). Without this setting, no guest can access any application that depends on Azure B2B collaboration, including Office
365 Groups and Teams.
2. Office 365 Groups . In the Settings-section of the Office 36 Admin Center, select Services & Add-ins, then Office 365 Groups, and make sure that
Let group members outside the organization access group content and Let group owners add people outside the organization to groups are both
On. When set, you can invite guests to join Teams. If you have specific teams that discuss sensitive information, you can block the ability to add
guests to those teams by editing their properties. See the discussion about how to do this for Office 365 Groups in Chapter 12.
3. Teams . Guests are automatically granted access to Teams and don’t need a license. To control what guests can do inside your tenant, use the
settings in the org-wide Guests policy available in the Teams and Skype for Business Online Admin Center.
4. SharePoint . In the SharePoint Admin Center, under External Sharing, set Let users share SharePoint Online and OneDrive for Business content
with people outside the organization to On. This enables guest access to the Files in SharePoint document libraries used by Teams and to files in
users’ OneDrive sites shared in personal chats.

With all settings turned on, external access is now available for all teams in the tenant.

Adding Guests to Teams

With all the correct settings in place, team owners can add guests. It is important to understand that the Office 365 group and its associated team
share a common membership list, including guests. Therefore, if you add a guest to the group, the guest gains access to the team and vice versa.
Sometimes a small delay happens between adding or removing a guest from the membership and that action showing up when viewing membership,
but background synchronization processes make sure that any addition or removal of guests applies across both Teams and Groups.
To add a guest member, select Manage team and then Add Member , or use Add members from the ellipsis menu. You then input the email address
for the new member (Figure 13-23 ). In this case, the email address entered for the guest was John.Smith@contoso.com. Left as is, Teams uses
John.Smith (Guest) as the display name seen by other team members.

The (Guest) suffix is a language-specific string added by Teams to mark someone as external. The string is translated in Teams clients and notifications
– for example, if a guest user runs Teams in Spanish and you receive an email notification for something they post, you’ll see (Invitado) after their
name.

Figure 13-23: Adding a guest to a team

You cannot change the guest suffix because that is a visual reminder to other members that a guest is an external person, but you can amend the other
part of the display name by clicking the pencil mark beside the name to edit guest information (Figure 13-23 ). You can now update the display name to
be whatever you want. For instance, by adding a company name so that the display name is “John Smith (Contoso)” to inform other members what
organization the guest is from.When the information for the guest member is complete, press Add . Teams checks whether a guest account for this
address already exists in the tenant directory and adds it to the membership if found. If not, Azure Active Directory creates a new guest account. Teams
then adds the account to the team membership and generates an invitation for the user to redeem. The invitation has all the information necessary for
the recipient to know about the team they are joining, including a GUID to find the right Office 365 tenant, another GUID for the team, and a token
request to redeem the invitation to join the team. The recipient redeems the invitation by clicking the Open Microsoft Teams link in the message,
which then invokes the necessary flow to confirm the user’s details, update the Azure Active Directory guest account for the user with credentials to
allow them to connect to the tenant, and access the team. Azure Active Directory logs the redemption in a “redeem external user invite” audit record.

Updating Display Names : Do not worry if you forget to update the display name for a guest when you add them as you can always update it
afterwards. You can edit the contact information for the guest through the Office 365 Admin Center, update the guest account properties in the Azure
AD portal, or run the Set-AzureADUser cmdlet. For example, this code updates the display name for the guest account created for
John.Smith@contoso.com by passing the user principal name as the object identifier. It also updates the user’s job title and city as Teams displays
these properties when you view a team. Finally, we update the telephone and mobile numbers for the user so that they appear when someone looks at
their contact card.

[PS] C:\> Set-AzureADUser -ObjectId John.Smith_contoso.com#EXT#office365itpros.onmicrosoft.com

-DisplayName "John Smith (Contoso)" -JobTitle "Account Manager" -City "NY" -TelephoneNumber "+1 617 551 6531" -Mobile "+1 650
561 4136"

A guest has an identity within the tenant through their guest user account. They do not own any resources. This means that although you can amend
settings in the guest user account (like their photo), a guest can’t have an out-of-office status because they don’t have a mailbox.

Multi-Factor Authentication for Guests

Like the other Office 365 applications, Teams supports multi-factor authentication (MFA). If your tenant implements an Azure conditional access policy
to require MFA, you can include guest members in the scope of the policy to force them use MFA to connect to Teams, even if their home tenant does
not require MFA for connection to services. The logic here is that a tenant always controls its resources and can therefore dictate the level of
authentication required to access those resources.

To ensure that a conditional access policy applies to guest users, choose “Guest Users” as the target group for the policy to apply to.

Blocking Guests from Specific Domains

If you do not want team owners to add guests from specific domains, you can create a block list using the Azure B2B Collaboration policy (described in
Chapter 12). The same policy can also create an allow list to restrict external access to specific domains, but a tenant can only support either an allow
or a block list. When a policy is in place, team owners cannot add guests from blocked domains. However, any guests from blocked domains who are
already members of teams continue as before. If you want to revoke membership for guests belonging to blocked domains, you must use PowerShell to
search the membership lists for all Office 365 Groups and remove any members found that belong to those domains. An example of how to remove a
user whom you wish to block is in Chapter 12.

One thing to remember is that if a guest user account from a blocked domain already exists in Azure Active Directory, Teams will allow owners to add
that account to team membership. In other words, Teams ignores the deny or block list for these accounts. It could be the case that the account was
added by another application (to share a SharePoint document, for instance) that has nothing to do with Teams. If you want to ensure that no guest
users are added to specific teams, disable the ability for team owners to add guests to those teams as described in Chapter 12.

Switching Between Tenants

Teams runs within the context of an Office 365 tenant. When a guest wants to access teams in your domain, they must “switch” Teams client by signing
into your tenant using their guest account. Switching means that the guest has a limited view of the Teams environment within the tenant. Guests can
only see teams to which they belong and cannot browse to join other public teams; they cannot see organizational information about other members,
nor can they create new teams or add apps to channels. Because guests cannot access the directory, they cannot update the picture for their account or
any other setting related to their account such as the display name. In addition, they cannot browse the directory to find people to chat with. Instead,
guests enter the email address of the person they are looking for and Teams checks the directory and creates a chat if that person exists.

On the other hand, guests can join in private and public conversations, take part in video and audio chats, and access the files in the team SharePoint
library. Another point to note is that guests can only access applications available to the team if they have the right credentials and the application
supports external access. It is as if they signed into your tenant with more restricted access than a normal tenant user has, which is exactly what you
want.

Switching accounts is simple. On the desktop client, click the tenant name in the command box and select the tenant to which you want to switch. In
Figure 13-24 , four tenants are available. When you select a different tenant, Teams signs-out of the active tenant, signs-into the target tenant., and
then displays the teams that the user belongs to in that tenant. The sign-in process ensures that the user picks up the rights and permissions they enjoy
within the tenant; when they are a guest, they have those rights in that tenant. When they return their host tenant, they can interact with Teams with
all the rights of that tenant account. Switching on a mobile client operates in much the same way, except that you click the menu in the top left and
then select the target tenant.

Figure 13-24: Selecting from a list of available tenants

On desktop clients, you can examine the Teams client log to find details of the switch to the other tenant (see example below), so this is a good place to
look for information if someone experiences a problem switching to another tenant. One common reason why switching fails is that the remote tenant
requires guest accounts to use MFA and the user has not yet configured MFA for their guest account.

Thu Feb 22 2018 16:40:08 GMT+0000 (GMT Standard Time) <9312> -- info -- SSO: correlationId - 8e08de9a-298a-4b40-ae26-d63f6113b6fe Returning cached user.

Thu Feb 22 2018 16:40:08 GMT+0000 (GMT Standard Time) <9312> -- info -- ADAL: [IPC] Switching tenant:c0a3a43f-9257-4949-be1b-f83cfb83f65f, isHomeTenant:false

Thu Feb 22 2018 16:40:08 GMT+0000 (GMT Standard Time) <9312> -- info -- Guest Tenant Id set to c0a3a43f-9257-4949-be1b-f83cfb83f65f

Thu Feb 22 2018 16:40:08 GMT+0000 (GMT Standard Time) <9312> -- info -- ADAL: Going to show intermediate page

Thu Feb 22 2018 16:40:08 GMT+0000 (GMT Standard Time) <9312> -- event -- status: success

Users do not see notifications in their activity feed for events in other tenants. Although Teams flags activity with indicators in your avatar, the lack of
notifications can be a problem if you need to keep an eye on an active conversation about an evolving situation as you must then sign into the tenant
hosting the relevant team to check what’s happening. Teams does not have a good solution for concurrent access to multiple tenants, but you can
consider using the approach described in this article to create “wrappers” for each tenant where you use Teams. You can then move from one tenant to
another while keeping access to Teams in the other tenants.

Removing Guests
You remove guests from a team in the same way as you remove a tenant user. Select Manage team from the menu and then Remove for the user in
the member list. Alternatively, you can run the Remove-TeamUser cmdlet to remove the user from the group. For example, this command removes the
guest Joan.Smith@contoso.com from the Industry News team.

[PS] C:\> $TeamId = (Get-UnifiedGroup -Identity "Industry News").ExternalDirectoryObjectId

[PS] C:\> Remove-TeamUser -GroupId $TeamId -User "Joan.Smith@contoso.com"

Removing an external member from a team does not remove the Azure AD guest account. If you want to remove an external user from all teams in a
tenant, you can remove their guest account from Azure AD. You can remove a guest account through the Azure portal, the Office 365 Admin Center, or
by running the Remove-AzureADUser cmdlet. Removing the account removes membership from all teams and groups and removes any sharing
permission the user has for SharePoint and OneDrive for Business files in the tenant.

Emailing Teams
If allowed by the tenant-wide “Email Integration” setting managed through the Office 365 Admin Center, Teams publishes SMTP addresses to allow
people to communicate with channels by email. This is a useful way to introduce a new topic into a channel using information contained in email. To
retrieve the email address of a channel, use the desktop or web client to click the ellipsis menu for the channel, select Get email address and then
Copy (Figure 13-25 ) to copy the address to the clipboard. If an email address does not already exist for the channel, Teams creates one. You can then
copy the address to be a TO:, CC:, or BCC: recipient for a message that you want to send to the channel. Any member can retrieve the email address
for a channel.

Figure 13-25: Retrieving an email address for a channel inside a Team

By default, anyone can send a message to the channel using the email address to, including people from outside the tenant. To restrict access to a
channel, click the Advanced Settings link to reveal options to restrict the email address to:

Only members of the team.

Specific email domains. For example, Microsoft.com. Note that you cannot specify a domain like *.onmicrosoft.com. You can enter multiple
domains in a comma-separated list.

You can also create an authorized senders list of SMTP domains for the tenant that applies to all channels in the Email integration section of Teams
settings in the Office 365 Admin Center. Use this method if you want to stop email arriving into channels that originate from anywhere except the
domains included in the list. If an unauthorized user attempts to send email to a channel, they receive a non-delivery notification telling them that they
do not have permission to send email to the channel.
Team members can use Remove email address to nullify the email address for a channel. Allowing any member to remove the email address used for
inbound communications is in line with the general rule that everyone shares equal access to resources, but it is easy to see how this could lead to
problems if people rely on the address to communicate with the channel. It is easy to restore email access with Get email address again., but Teams
does not restore the old address to the channel and generates a new address instead.

Channel Email Addresses

When Teams creates an email address for a channel, it also creates a mailbox in a special tenant dedicated to Teams email processing and assigns the
address to the mailbox. The channel’s email address is in the form <unique-identifier>.tenant-name@region.teams.ms . For example, the email address
b400aa20.tenant-name@emea.teams.ms belongs to a tenant in the EMEA region. Teams then creates a connector (see below) to link the mailbox to the
channel so that any email arriving into the mailbox flows through to the channel. A tenant exists to host Teams mailboxes in every Office 365
datacenter region supported by Teams. You cannot change the address to make it more human-friendly or to use a vanity domain belonging to your
tenant.

Use BCC to Stop Mail Storms : The email address for a Teams channel functions in the same way as any other email address in that you can use it
as a TO:, CC:, or BCC: recipient. However, there’s a very good reason for always using BCC for Teams email addresses. The purpose of a Teams email
address is to allow users to start new discussions in a channel with content that already exists in email. You do not want the channel to take part in a
lively back-and-forth email conversation because every interaction shows up as a new thread in the channel and can confuse team members. Use BCC
to address email for a channel and then develop the conversation forward in the channel.

Message Hygiene for Inbound Email to Teams

Because the mailboxes used by Teams are part of the Office 365 infrastructure, inbound messages go through Exchange Online Protection (EOP).
Although EOP will stop the delivery of malware to Teams, the special nature of the configuration used by Teams means that inbound mail to does go
through the same processing as the email stream for a regular customer tenant. For example, if you license Office 365 E5 and configure the Advanced
Threat Protection (ATP) policy to enable Safe Attachments and Dynamic Delivery, you expect that EOP processes all attachments in this manner.
However, this will not happen for messages routed through Teams. On the other hand, if you configure ATP for SharePoint, when Teams captures the
message files in the SharePoint document library (see below), ATP processes any attachment at that point.

Teams will not deliver a message to a channel if the message has more than 20 file attachments or more than 50 inline images. Exchange Online
generates a non-delivery-notification to the sender when Teams rejects the message.

Audit Records for Email Addressees

When a team member creates the email address for a channel, Teams generates the email address and records the action in a “Connector created”
audit record. The connector type is an “Email Connector.” If a team member then removes the email address from the channel, Teams captures the
action in a “Remove Connector” audit record. The capture of these records means we can search the audit log to find out who added or removed email
addresses for channels with a command like:

[PS] C:\> Search-UnifiedAuditlog -Operations ConnectorAdded, ConnectorRemoved -StartDate $Startdate -EndDate $EndDate | Format-Table CreationDate, Userids, Operations

Details of the team and channel associated with the email address are held in the audit record. See Chapter 21 for details about how to search the
Office 365 audit log using the Security and Compliance Center or PowerShell.

Capturing Email Sent to Teams

When a channel receives email, the text of the message appears as a new topic in the channel. If the text exceeds the limit for a contribution to a chat
(25 KB, or roughly 20,000 characters including spaces). In this case, you see some text but need to download the “original email” to view the full
content. You cannot reply to an inbound message because Teams has no other access to email except its ability to receive inbound messages. Any
attempt to reply to an inbound email generates the response that your reply will only be visible in Teams.

When Teams receives a message via email, it strips any attachment and stores the file in the “Email Messages” folder of the channel folder created in
the SharePoint Online document library for the team (a separate “Email Messages” folder exists for each channel in the team). In addition, Teams
captures a copy of the message as an .eml file and keeps the file in the same location. Figure 13-26 shows a set of messages and attachments stripped
after receipt by a channel. Keeping copies of message and attachments received by Teams in SharePoint means that the Search Foundation can index
the content to make them available for eDiscovery.
Figure 13-26: Attachments and .eml files created from inbound email to a channel

Messages sent to the email address of the Office 365 group belonging to the team go to the group mailbox. Members who subscribe to the group
receive copies, but Exchange will not deliver a copy of the message to a channel in the team unless you add the channel’s email address as an external
member of the team. For example, let’s assume that you want any email sent to the Corporate Business Development group to show up in Teams and
you create a channel called Email Communications for this purpose. You can then get the email address for the channel and add it to the group with
OWA. Exchange will deliver a copy of any messages sent to the group afterwards to the channel.

Because copies of messages sent to channels end up in SharePoint, the delivery of a message to a channel is captured in an Office 365 “Uploaded File”
audit record. You’ll see that the user noted in the audit record is app@sharepoint , a background process that performs many management operations
for SharePoint. This process copies the message from Teams to SharePoint.

Teams and Compliance

Teams stores its messages for channel and personal conversations in a data store hosted by Azure. Because it lies outside Office 365, the information in
the Teams data store is unavailable to the data governance framework. It is obvious that many interesting conversations occur in Teams and that the
information discussed in these conversations might be of interest to eDiscovery investigations. The same fear that eDiscovery investigations might not
have access to all relevant information exists when employees use third-party chat networks like Slack.

Teams does not support the full range of data governance technologies available in Office 365. Microsoft says that they will add data loss prevention
checks for chat and channel conversations by the end of 2018 and will support Office 365 labels in the future. For now, the compliance features are:

eDiscovery : Teams captures and indexes compliance records so that content searches can find, preview, and export these items. Compliance
records are captured for conversations, meetings, and calls.
Retention : Teams supports the application of Office 365 retention policies against messages in Teams channels and private chats.

Capture of Teams Compliance Records

To ensure that Teams conversation are available for eDiscovery, Teams captures copies of the messages that make up conversations as items in
Exchange Online mailboxes. As people chat, the “Office 365 substrate” captures each contribution in a separate item, or compliance record. In effect, a
background process like a mailbox assistant looks for new contributions and captures them as items into user or group mailboxes. The items have a
sender (the author), a recipient (the group mailbox for channel conversations or user mailboxes for personal chats), and a timestamp for when Teams
created the item. Like other mailbox items, you can examine the properties of Teams compliance items using the MFCMAPI program. A complete
conversation between multiple people might have thirty or more contributions, each of which exists as a separate compliance record. Exchange Online
uses a similar approach to capture Exchange mailbox audit events in a hidden folder in user mailboxes.

Teams captures compliance records as follows:

Personal chats : Office 365 creates compliance records in the mailboxes of the participating users. For example, if John and Pat have a chat,
Office 365 creates compliance records for the chat in both their mailboxes. If a conversation involves ten people, compliance records for the chat
exist in all ten mailboxes.
Channel conversations : Teams captures contributions for all channels in the group mailbox belonging to the team. The compliance records for
conversations in the different channels in a team are intermingled in the mailbox.
Meetings and Calls . The calling infrastructure generates records of meetings and calls. Meetings include scheduled meetings (in the calendar)
and ad-hoc meetings associated with a channel. Group chats that include more than two people are also considered in this category. Calls cover
one-to-one calls.

Compliance records captured for personal or channel messages are available very soon after the message is sent (usually a matter of seconds, never
more than a few minutes) and include the text of the message as entered by the sender. Like other Exchange items, a compliance record is composed of
a set of MAPI properties that can be viewed using a MAPI editor like MFCMAPI . The properties include:

SkypeInternalIds : a list of GUIDs for each of the participants (other than the sender) in the conversation. Teams uses GUIDs internally to avoid
problems with user display and principal names, both of which can change over time due to marriage, divorce, or other circumstances. Teams
resolves the GUIDs for display and stores the display names in the PR_DISPLAY_TO property.
ParentMessageId : The reply identifier for the message thread (also in the LinkId property).
CreatedDateTime : Date and time of the message.
PR_BODY : Text of the message.
PR_SENDER_NAME : The display name of the sender. The GUID for the sender is found in the FromSkypeInternalId property.
To resolve the GUIDs used in the compliance records, remove the “&:orgid:” prefix from the value and run the Get-AzureADUser cmdlet. For example:

[PS] C:\> Get-AzureADUser -ObjectId c7745bc8-6f5c-4d45-82c7-fee55f384985

DisplayName UserPrincipalName

----------- -----------------

Ståle Hansen stale.hansen_cloudway.no#EXT#@office365itpros.onmicrosoft.com

Call Records
Compliance records captured for meetings and calls depend on the Call Record Processing service (part of the next-generation Skype service). In
telephony terms, the compliance records captured for calls and meetings are known as call detail records, or CDRs. The CDRs are created in the
mailboxes of all call participants in the same hidden folder used to store the compliance records for messages (see below for details). However, it can
take up to eight hours before CDRs are available for searching. This usually is not a problem because eDiscovery searches normally happen after an
event occurs.

CDRs do not include audio or video events; if you want to have audio or video recording, you can capture a recording and store it in Stream to take
advantage of features like automatic transcript generation and face recognition. CDRs capture textual details of a meeting or call such as:

The start and end time of the meeting, and the overall duration of the meeting.
Notes of when each participant joined and left the meeting using the VOIP, PSTN, anonymous, federated, and guest user mechanisms.
Calls to voicemail.
Missed or unanswered calls.
Call transfers (which are represented as two separate calls).

Microsoft plans to capture more detail about calls such as screen and app sharing in the future.

Versions of Compliance Records

If an Office 365 retention policy is in place for Teams, any changes made by a user to a message are recorded in the Versions sub-folder of Recoverable
Items in the target mailbox. Thus, a content search might uncover several different versions of the same message, with the last one stored in the Team
Chat folder being the actual content visible to users (the others are in Recoverable Items).

It is important to understand that the compliance records stored by Teams in mailboxes are versions of the actual messages in channels and private
chats. They are not the actual messages, nor are compliance records perfect replicas of those messages. The compliance records include any
embedded graphics and GIFs, but they do not include “likes” given to messages. These which can be important signs that certain individuals have seen
a conversation in the same way that changing the read status of an email from “unread” tells you that the message was opened.

Compliance Records for Non-Tenant Users

Hybrid users with on-premises Exchange mailboxes, external people contacted through 1:1 chats, and guest accounts do not have Exchange Online
mailboxes, so Office 365 cannot create compliance records for these users in the same way as it does for those with tenant mailboxes. Instead, Office
365 provisions special “phantom” mailboxes the first time these users send a Teams message. The phantom mailboxes are created in the same
datacenter region as the host tenant. They are only used to store compliance records and cannot be logged into or to send or receive email.

The on-premises users must be part of a hybrid Office 365 organization and synchronize their accounts with Azure Active Directory using AADConnect.
If a hybrid mailbox is moved from on-premises to Exchange Online, the transfer process moves the compliance records from the phantom mailbox into
the user’s new cloud mailbox.

Although Teams supports Office 365 retention policies, those policies do not apply to the phantom mailboxes for by guest and hybrid users. Likewise,
you cannot apply retention holds to these mailboxes.

Storage of Teams Compliance Records

No matter what mailbox used to store the items, compliance records captured for Teams conversations are in a special hidden folder called
Conversation History\Team Chat (for English language mailboxes). This is a sub-folder of the Conversation History folder used to store copies of IM
conversations for Skype for Business calls. Because the Team Chat folder holds compliance data that users should not be able to interfere with, clients
like OWA and Outlook desktop do not expose the folder in their GUI. If you need to examine compliance records, you can open the folder in a personal
mailbox (but not a group mailbox) to examine individual items with a program like MFCMAPI.

Depending on load, the delay before a compliance record appears in the mailbox varies from a few seconds to a few minutes. Once captured in a
mailbox, Exchange Online automatically indexes the compliance records to make them discoverable by content searches. To see how many Teams
compliance records are in a user or group mailbox, use a command like this:

[PS] C:\> Get-MailboxFolderStatistics -Identity "Budget Planning" -FolderScope ConversationHistory

-IncludeOldestAndNewestItems | ? {$_.FolderType -eq “TeamChat”} | Format-Table Name, ItemsInFolder, NewestItemReceivedDate, OldestItemReceivedDate

Name ItemsInFolder NewestItemReceivedDate OldestItemReceivedDate

---- ------------- ---------------------- ----------------------

Team Chat 4 227 2 Aug 2018 16:10:41 11 Mar 2017 15:41:34

The reason why we use the IncludeOldestAndNewestItems switch with the command is to force Exchange Online to return information about the oldest
and newest items found in the folder. This is interesting information if you want to know if the Office 365 retention policy assigned to the team is
cleaning out old items as their retention period expires.

The name of the folder used to capture compliance records varies with language. For instance, it is “Discussion d’equipe” for a user or group mailbox
with its language configured as French, or “TeamChat” for German. One complication is that it is possible to change language for a mailbox after its
creation. When this happens, the name of the Team Chat folder does not change. Because the name of the folder varies with language, we check
against the folder type (TeamChat) to ensure that the command shown above works for mailboxes configured to use languages other than English.

The Team Chat folder is created along with the other default folders for new user and group mailboxes. Because they never reveal the existence of the
Team Chat folder, Outlook clients do not synchronize this folder to a local copy in the OST. Therefore, if you need to examine the items in the folder
with a utility like MFCMAPI, you must configure the Outlook profile used with MFCMAPI to run in online Exchange mode instead of cached Exchange
mode.

In addition to compliance records captured for conversations, Teams captures copies of email messages sent to a channel in a sub-folder called “Email
Messages” of the channel folder in the SharePoint document library belonging to the underlying Office 365 Group. The messages are in individual
indexed files discoverable by content searches. Any files uploaded to the SharePoint document libraries used by Teams or to users’ OneDrive for
Business sites come under the usual compliance controls deployed to those workloads.
Searching Teams Compliance Records
Office 365 content searches do not process Teams data stored in its Azure-based data services. Instead, searches depend on the Teams compliance
records held in Exchange. You do not have to do anything special to include compliance records in content searches because Office 365 automatically
searches these items whenever you include the mailboxes (group or user) or sites used by Teams in content searches. The sole exception is for users
who have their mailboxes on on-premises Exchange servers. Office 365 does not index the content in these mailboxes and they cannot be included in a
content search. In addition, you cannot apply a hold from Office 365 to content stored in on-premises mailboxes.

Figure 13-27 shows how Teams compliance records appear as preview items found by a content search. In this case, the record is for a contribution to
a conversation in a channel. We know this by examining the item properties, where we can find:

From : The display name and email address of the user who posted the message.
To : The name of the team and the email address of the group mailbox used by the team or the names and email addresses of the participants in a
personal chat.
Subject : If the item belongs to a personal chat, the subject is “IM.” For channel conversations, if the item is posted to any other channel than the
default General channel (which is, in effect, the team), the channel name is included in the subject. The highlighted item has a subject of
Engineering Testers and Engineering Plans 2018 followed by the reply chain identifier (a number that links together all the messages in a
conversation), so we know that this item belongs to the Engineering Plans 2018 channel in the Engineering Testers team.
Send date : The date and time in UTC format when the sender posted the item.

Figure 13-27: Team compliance records previewed by a content search

As noted above, Teams compliance records for channel and personal conversations use the message item type of IM rather than the normal “Email”
used for mailbox items. Note that messages created in channels by Office 365 connectors like the Twitter connector also use the IM message type.
Compliance records for meetings have the “Meeting” message type while personal calls use the “Call” message type. If you want to search for all
compliance records generated by Teams, you can use the special MicrosoftTeams message type, which includes “IM”, “Meeting”, and “Call.”

Unlike Skype for Business Online, which uses a transcript format to record conversation details spread over one or more records (depending on how
long the conversation lasts), the way that Teams captures individual compliance records for each contribution to a conversation makes it more difficult
to reconstruct a full conversation from start to finish. Normally, when eDiscovery investigators review information, they want to see the full context to
understand who said what to whom and how a conversation developed. Skype for Business transcripts give context by recording the different
interactions of participants in a conversation. Email can do this too by including the text of prior replies in a message.

It is possible to build up a complete conversation from the compliance records captured by Teams, but to do this, you must:

Search for and find the compliance records for all contributions to the conversation. You won’t find all the compliance records belonging to a
conversation unless you use its reply chain identifier as a search keyword. For example, you could search for 1529919175913 to find all items in
the conversation with that reply chain identifier.
Because items from all channels are in the same mailbox, be careful not to mix items from different channels together.
Arrange the items in date order using the timestamp. Some commercial products, like GlobalNet Merge1, combine compliance records to form
threads when they present search results.

You can export information found in Teams by a content search to PST files or as individual items, and then give the PST or items to external
investigators for their review. See Chapter 20 for more information about how to build and run content searches and export items found by the
searches.

Searching Hybrid and Guest Compliance Records

You can ask Microsoft to update the GUI for the Security and Compliance Center to let search administrators specify that they wish to search “app
content for on-premises users,” which also includes guest users. The update process takes a few weeks to complete (see directions in this article ).
Even without updating the Security and Compliance Center, you can create and run content searches with PowerShell to find Teams compliance
records generated by hybrid and guest users.

Two parameters for the New-ComplianceSearch cmdlet are of special interest:

Set AllowNotFoundExchangeLocationsEnabled to $True to tell Office 365 that you want to search the phantom mailboxes. The search won’t
try to check that the Exchange mailboxes specified for the search exist (they do, but they are phantoms). It also means that content searches will
check compliance records generated by guest users.
Set IncludeUserAppContent to $True to tell Office 365 that some or all the mailboxes specified for the search are phantoms.

All that remains is to specify the set of on-premises mailboxes to include in the search in the ExchangeLocation parameter or set the search to include
all Exchange mailboxes. For example, these commands create a content search for chat records and then start the search.

[PS] C:\> New-ComplianceSearch -Name "Teams Chat Scan" -Description "Search for Teams Chat Information about Finance" -IncludeUserAppContent $True -
AllowNotFoundExchangeLocationsEnabled $True -ExchangeLocation All -ContentMatchQuery "Finance AND Kind:MicrosoftTeams"

[PS] C:\> Start-ComplianceSearch -Identity "Teams Chat Scan"

After the search completes, you can use the GUI of the Security and Compliance for content searches to preview search results, refine the search
keywords and qualifiers, and export results for further investigation.

Teams and Office 365 Retention Policies

Apart from serving as the basis for eDiscovery, the compliance records stored in the Team Chat folder are also how Office 365 manages retention for
personal chats and channel conversations. As explained in Chapter 19, you can create an Office 365 retention policy for Teams. The Managed Folder
Assistant (MFA), which also processes Exchange Online mailbox retention policies, applies the policy settings against the compliance records. When
the MFA removes items from Team Chat, the Office 365 substrate synchronizes the deletions back to the Teams chat service, which then removes the
relevant messages from its store. Later, Teams clients connect to the Teams service and synchronize their local cache to complete the removal cycle.

If an in-place hold or litigation hold applies to group or personal mailboxes that include Teams compliance records, those items come under the scope
of the hold. Office 365 captures any attempt to remove or edit a compliance record by keeping the copies of the removed or edited item in the \Purges
or \DiscoveryHolds sub-folders under Recoverable Items in the mailbox.

When you create a retention policy for Teams, you can choose to keep messages for the retention period or remove items after the retention period
elapses. What you cannot do is give users the ability to mark specific messages to force Office 365 to remove those items sooner or keep them longer.
SharePoint and Exchange support this kind of flexibility through Office 365 retention labels or mailbox personal tags.

For instance, if you need to keep some content for ten years for audit purposes and the retention policy removes items after six months, you can assign
a personal tag or retention label to items in a mailbox or retention labels to documents in SharePoint or OneDrive for Business sites. Although Teams
retention is based on the compliance records in user mailboxes, clients cannot access the Team Chats folder to apply retention tags to the items stored
there.

It’s possible that Microsoft will give Teams the ability to use Office 365 labels in the future. Supporting Office 365 labels might also allow Teams to
exploit advanced compliance functionality like disposition review or event-based retention.

Team Expiry
Removing old Teams when their period of usefulness expires is part of compliance planning. If your tenant uses an Azure Active Directory group
expiration policy (see Chapter 12 for details) and the group used by a team comes within the scope of the policy, an extra section called Team Expiry is
visible to team owners under the Settings tab to inform owners when the team needs to be renewed. A team owner can click the Renew Now button to
extend the lifetime of the team for whatever period is set in the policy (normally a year or two). Renewal can happen at any time.

When a team is within a month of its expiry date, Teams displays a warning triangle beside its name in the navigation pane (but only to team owners).
Hovering over the warning triangle reveals the expiry date and the choice to Renew team appears in the ellipsis menu.

If a team reaches their expiry date and isn’t renewed, Office 365 soft-deletes the group and all associated resources, including the team, and removes
the ability of users to access those resources. An administrator can restore the team at any time within 30 days of its being soft-deleted. Once the 30-
day period elapses, Office 365 permanently removes the group and all its associated resources, and the group becomes irrecoverable.

Figure 13-28: Showing the Teams expiry date

Archiving Teams
Expiring a team removes it from the tenant. Archiving a team is another way of dealing with teams that are no longer active. When you archive a team,
you make the elements controlled by Teams (like channel conversations and the wiki) read-only. Afterwards, members see a notice that the team is
archived, and Teams doesn’t allow them to make any more changes to content. The idea is that the team is available with its membership in force to
allow users to continue accessing information while not being able to add to that information. You can update the membership of an archived team to
add or remove members, including guests, or promote members to be owners.

To archive a team, click Teams in the navigation pane to expose the list of teams, then the Manage cogwheel icon under the list of teams. You now see
the list of teams that you belong to, divided into active teams and archived teams. Only team owners, tenant administrators, and Teams service
administrators see the choice to Archive team in the ellipsis menu for the team (Figure 13-29 ). You cannot currently archive a team using PowerShell,
nor can you get a list of archived teams with PowerShell.

When you archive a team, the messages in the channels within the team become read-only. Users can access messages in an archived team, but they
cannot post new messages, edit messages, or remove messages from a channel. In addition, members can access files in the document library
belonging to the team, but they cannot upload new documents, remove files from the library, and the link to open the document library in the
SharePoint browser interface is not available. When users open an archived team, they see notices to tell them that they can no longer post to the
team. Teams also displays an icon (a closed drawer) alongside the names of archived teams in the team list (you can favorite an archived team to have
fast access to its contents).

Figure 13-29: The option to archive a team

During the archival process, a team owner can choose to make the SharePoint site read-only for team members (this also sets the wiki to be read-only).
Team members can continue to use a browser to open the SharePoint site belonging to the team after it is archived, but if the site is read-only, users
who access the document library will discover that they have restricted options. This is because Teams adjusts the site permissions for team members
to remove their write access. For example, members cannot upload files to the library, rename or remove files, update document details, assign Office
365 labels, and so on. They can still synchronize the library and download files. Team owners continue to have read-write access to the SharePoint site.

Setting read-only access to the conversations and files belonging to a team is an effective way of putting it into an archive status, but one of the big
selling points of Teams is that it serves as an integration point to bring many other third-party and Microsoft applications to users. To make the archive
status fully effective, every application connected to Teams must understand when a team is archived. SharePoint does this, but the other connected
apps don’t, which means that team members can continue to have read-write access to other apps like Planner, OneNote, and apps added to the team
as tabs, and bots. This makes sense for third-party apps as they might be shared across multiple teams.

To restore an archived team and make it read-write again, select it in the list of archived teams and then choose Restore team from the ellipsis menu.
Restoring a team makes its conversations and SharePoint site read-write for members.

Archiving a team does not stop it from expiring later, if it comes within the scope of the group expiration policy. The team is still there, albeit in a read-
only state; expiration kicks in once the renewal time of the underlying group is reached and if the group is not renewed, Office 365 will soft-delete the
archived team.

The read-only approach for archiving teams is reasonable. Its biggest advantage is that members continue to enjoy access to channel conversations. A
different approach is outlined in Chapter 14 which uses PowerShell to archive teams by removing everyone but an owner from group membership. The
advantage of this approach is that members lose access to any resource owned by the group, which is what you might want to achieve.

Auditing Teams
In addition to capturing compliance records for individual and channel conversations, Teams generates audit records for many user and administrative
operations. The most common audit record is to record each time a user, including guests, signs-in. Teams captures an audit record for a user sign-on
for every hour in a session. This is because the access token expires after an hour. When the access token is renewed, the user signs in again. The user
does not notice the sign-in happening as token renewal and reconnections happens in the background. The audit record tells you when the user signed
into Teams and some information about what client they used, but it does not capture details of which team or channel they accessed.

Among the administrative activities that Teams captures audit records for are:

Team creation and deletion.

Channel creation and deletion.
Users added or removed from teams.
Settings changed for the organization, team, or channel.
Tab addition and deletion (for a channel).
Connector addition, deletion, and updates.

For example, Office 365 records audit events for the creation and removal of teams and the creation of channels within teams. For instance, when you
use a Teams client to create a new team, the audit log receives an “add group ” event when Azure Active Directory creates the new group followed by
some “update group ” events when it populates the properties of the new group. Finally, you see a “TeamCreated ” event for the new team. Office 365
also records audit events for any alteration of team settings. However, Office 365 does not capture a “TeamCreated ” audit event when you enable an
existing Office 365 group for Teams.

The audit records for team and channel creation capture the names given to the new teams and channels and who created the object. However, the
records for tab creation only tell you that someone created a tab. No information is captured about the content the tab links to. For example, if you
create a tab to link to a YouTube video, the audit record tells you that the tab type is “extension ” but not that it links to a video or what the video is
(apart from third-party apps like YouTube, the extension tab type covers Microsoft applications like Planner and Stream too). No information is
recorded about the content either when someone updates a tab. The lack of data is sometimes explained by the need to protect user privacy, and
sometimes because tenants and Microsoft need to define what information Teams should capture (and why) for audit purposes.

Teams periodically uploads its audit data to the Office 365 audit log. You can interrogate audit records using the audit log search in the Security and
Compliance Center, by running the Search-UnifiedAuditLog cmdlet, or with third-party security products. Techniques for using these tools are
explained in Chapter 21.

Teams and Planner

Figure 13-30: Viewing the set of plans known to Teams

In some cases, Teams uses Office 365 components in much the same way as Outlook or Yammer-based groups do. In others, particularly Planner, it is
quite different. For example, when you work on a plan associated with a channel, task assignees do not receive notification through email. Instead,
team members must check the plan or their email to discover details of their assigned tasks. Selecting the choice to “post to the channel about this
tab” does not post details of task assignments in the channel. Another point to consider during any implementation that includes frontline workers is
that Teams is available to frontline (F1) licenses while Planner is not. The biggest differences are the depth of the integration between Teams and
Planner and the way that Teams manages plans.

Planner creates a default plan when someone first uses the application with an Office 365 Group. Teams is not bound by the same limitation and you
can have multiple plans within a team. When you create a new tab for Planner in a channel, you choose to create a new plan or link to the default plan
if one exists. You can create multiple plans in a channel and work with the plans through Teams or click the Go to website icon in the set of options at
the top right-hand corner of the pane. This launches Planner and loads the tasks for the plan into a browser tab.

To discover the set of plans for the current user, click the More options menu (…) in the navigation pane and then select Planner. Teams then displays
the plans associated with channel tabs in teams that the current user can access. This is a subset of the full set of plans available to the user because it
only includes the plans made known to Teams by being linked to a channel. Because Teams only knows about some plans, Teams does not support the
My Tasks view available in the Planner browser interface that shows all tasks assigned to the user.
Figure 13-31: Accessing the default plan for an Office 365 Group through Teams

Within Teams, the user can opt to see Recent plans (those opened within Teams in the last month) or All (Figure 13-30 ). In this case, we can see that
the GDPR Planning team has three plans, two of which belong to the Audits channel. The tab name can differ from the name of the plan. For instance,
the “Assigned Tasks” tab in the General channel of the GDPR Planning team points to the default plan. Click on a plan to launch Planner and load its
tasks into Teams. At this point, you can rename the tab if you want by selecting the arrow beside the tab and then Rename .

Figure 13-31 shows how a plan appears when opened in Teams. In this case, the plan is the one we use to organize the work for this book. You can see
that the tasks and buckets for the plan are visible and that a conversation has started about a task.

To remove a plan from Teams, open it and then select the arrow next to the tab and then Remove . You now have a choice:

Remove the plan from Teams . This is the default and leaves the plan accessible through the Planner browser interface. If needed, you can add
the plan back to Teams.
Permanently remove the plan : To do this, select the checkbox Permanently delete this plan and all its tasks . This removes the plan data from
the Planner Azure service. The plan is then irrecoverable through any interface.

Unlike removing a plan through the Planner browser interface, removing a plan through Teams does not affect the Office 365 Group or any other
resource attached to the group or team.

Extending Teams
Out-of-the-box, you can extend Teams by adding apps, bots, and connectors to channels. Bots fit into conversations to allow users to interact via chat
with software assistants, usually to answer questions about specific topics. Connectors set up links between channels and network data sources so that
information flows from the data source and appears in the channel like user contributions.

Microsoft has a developer preview program for Teams that focuses on:

Extending the tabs available for a channel to incorporate access to an application.

Adding a new bot to answer questions posed by team members.
Creating a connector to a network data source.

Another aspect that might be of interest to large enterprises who want to stamp corporate branding on applications is to change the active theme for
Teams .

Given the attractiveness of the platform and the number of extensions that have appeared for Slack , it is reasonably safe to predict that developers will
be interested in exploiting Teams. This is especially so as Teams can be used by over 135 million Office 365 accounts today with an almost sure-free
guarantee that the Office 365 user base will expand considerably over the next few years.

Teams Store
Figure 13-32: The Teams Store

The Teams Store (Figure 13-32 ) is the place where you can go to see the apps, tabs, bots, connectors, and other services available for inclusion in
Teams. You can select the target team and channel to install an extension from the Store or go to the team and perform the action there.

Settings to Control Apps

Team owners can always add apps made available in the App Store to their teams. Member permissions for a team control whether individual members
can add and remove apps, tabs, and connectors. At a tenant level, four settings available in the Office 365 Admin Center (Figure 13-33 ) control the set
of apps that are available.

Enable default apps : allows access to apps developed by Microsoft (otherwise called first-party apps). These include apps such as Bing News,
Yammer, Visual Studio Team Services, and Dynamics 365. You can enable or disable individual apps.
Allow external apps in Microsoft Teams : allows access to apps developed by third parties and published in the App Store. If you turn the
switch to On , users can activate any external app in the Store in a team. Otherwise, if the switch is set to Off , you can enable individual apps
such as Trello, Google Analytics, YouTube, Twitter, or the incoming Webhook connector.
Allow sideloading of external apps : sidelining means that you introduce an app to a team by uploading a zip file with the app components
directly to the team. The mechanism is usually used to test apps during development, but it also lets developers build and publish an app for
internal use without giving it to Microsoft for validation and publication in the Teams app catalog. Only team owners or members who are granted
permissions to add or remove apps for the team can sideload an external app.
Enable new external apps by default : when this switch is turned On, users can activate external apps as soon as the app is published in the
App Store.

When you restrict the set of apps available in Teams, the Store filters the set of apps, bots, and connectors it displays to users and team owners. Due to
caching, it can take a little time before enabling or disabling an app is effective within Teams.

Figure 13-33: Tenant-wide settings to control apps

Teams and Connectors

Office 365 connectors create a link between a network data source and a destination within Office 365. In Teams, the destination is a channel within a
team. Connectors work much the same way with Teams as they do with Office 365 Groups (see Chapter 12). The results fetched through the connector
show up as “cards” created as conversations within the target channel. The cards do not hold the full content of an item fetched from a source like
Twitter or an RSS feed. Instead, they hold enough text to let users decide whether they want to discuss the content – or click the link to explore the full
content. Note that only users with an Exchange Online license can create a connector for a channel.
You can use connectors with teams in many ways. For instance, you could have a marketing team that needs to keep an eye on the corporate twitter
feed or whatever appears in the corporate blogs. This would be easy to configure within a channel called “Twitter Feed” or “Corporate Blogs” inside a
the “Marketing Group” team. Another example is how to use Google Analytics to track basic usage patterns for visits of web client to Teams. Many
other different out-of-the-box connectors are available, including the Incoming Webhook connector, which you can use to bring data from custom data
sources into a team or group. Chapter 14 includes an example of how to use the Incoming Webhook connector for this purpose. The example code
explained there also supports Teams.

Because the cards created by connectors appear as messages in channels, it is logical that you add connectors to specific channels. To add a connector,
click the ellipsis menu to the right of the channel name you want to use and select “Connectors.” You can then browse through the gallery of available
connectors and select which one you want to use to bring network data into the channel. Some of the connectors need credentials before they can fetch
content. For instance, the Twitter connector needs the credentials for a valid Twitter account to connect to Twitter. When someone creates or removes
a connector for a channel, Teams notifies members of the fact by posting a system message with details about the connector in that channel such as the
source and download interval. After the connector downloads content and creates messages in the channel, users can comment on them like any other
conversation.

Disabling Connectors
Connectors are included in the sets of default and external apps supported by Teams. As mentioned above, you can enable or disable individual apps
through tenant-wide settings. If the app is a connector, it then becomes available or unavailable to be added to a channel.

If you want to disable the ability to add any connector to a team, you must disable the feature for the underlying group by running a command like:

[PS] C:\> Set-UnifiedGroup –Identity "Marketing Team" –ConnectorsEnabled:$False

Setting the ConnectorsEnabled property for a group disables the ability to add new connectors in any channel in the team. Any attempt to add a
connector results in the error:

Connectors have been turned off for this mailbox by the admin.

Because of the need to synchronize between Teams and Azure Active Directory, it might take a little time to apply or remove a block. Unfortunately, the
implementation of the block within Teams is not particularly graceful as the choice to add Connectors persists in the menu even when a block is in
place for the group (or the organization). To block the creation of connectors for all Teams in the tenant, run the following PowerShell command to
update the organization configuration:

[PS] C:\> Set-OrganizationConfig -ConnectorsEnabled: $False

Existing connectors in use before the implementation of a block continue to run as before. If you want to remove connectors, you must do so manually.
Teams records audit events for the addition or removal of connectors in the Office 365 audit log. The audit records hold details of the type of connector
(for example, RSS) but not the source.

Teams and Bots

Another way of extending the usefulness of a team is to connect it to one or more bots. A bot is a virtual software assistant. In the context of Teams, a
bot is an application that accepts questions about specific topics from team members and responds, all within a conversation like those conducted with
humans. Teams used to have its own T-Bot to answer questions posed by users about Teams, but this bot was replaced by a more comprehensive Help
framework.

Microsoft provides the Bot Framework and documentation to assist developers to build their bots for use with Teams. A sample Teams application
written in C# is available too. To enable a bot for a team, go to the Store and click Bots . You can then browse the set of available bots and decide
which bot to install into a channel within a team. Once installed, team members can interact with the bot. Figure 13-34 shows the Hipmunk bot in
action. The bot responds to a request to find flights by scanning available itineraries and returning the best result.

Figure 13-34: The Hipmunk bot responds to a request to find flights

The Who bot is one of the standard bots included with Teams. Who bot uses the organizational information included from Azure Active Directory and
signals gathered in the Microsoft Graph to answer questions such as “Who works for Kim Akers” or “Who knows about collaboration.” The answer to
the first question comes from the manager-employee data in Azure Active Directory; the answer to the second from information in Teams chats. In both
cases, to find answers, the Who bot executes searches on behalf of the user. To find experts, the bot scans all the teams to which the user belongs to
find messages relating to the topic. The bot then performs some post-processing to score the results, using @mentions as important signals. Each use
is ranked by the number of messages about the topic they post, with their ranking increased for each @mention they receive. The idea here is that if
someone mentions another person in a message about a topic, they do so because they think that person is an expert and might know the answer.
Messages that include multiple @mentions receive a lower scoring boost because these mentions are often used as notify people of something they
should know about. At the bottom of Figure 13-35 , we see a list of the kind of questions that the Who bot can answer. If you chat with Who bot, the
conversations show up as a personal chat.

If you do not want users to interact with bots, you can disable access to bots through the Teams settings in the Office 365 Admin Center.

Figure 13-35: Interaction with Who bot

Can Teams Replace Email?

Some believe that you can reduce the use of email within an organization by replacing email threads with conversations in Teams. Among the
advantages cited for this approach are:

Conversations and documents exist in the team rather than in multiple mailboxes. Thus, users have just one place to find information.
Conversations are in channels while documents (including attachments in email sent to the channel) exist in Files (a SharePoint document library).
Conversations do not fork. It is easy for a small subset of recipients to start off their own conversation after receiving email. This does not usually
happen inside a channel because the conversation stays there and is visible to all team members.
The volume of email declines when some traffic moves to Teams and email evolves to serve different purposes. Internal conversations stay in
Teams while email handles external communications that Teams cannot handle (because no way exists to send outbound email from Teams). If
necessary, people can start internal conversations by sending email to a channel and continued from that point inside the team.

Naturally, the advantages advanced for Teams are aspirational, and no guarantee exists that everyone is willing to change the habits of a lifetime and
move from their favorite email client to Teams overnight. Client is an important distinction here because users typically think about their client and
how they use it instead of email. Outlook desktop is a great case in point as many people have built their working habits around the client.

Figure 13-36 illustrates where Teams scores over email. This channel conversation has 123 replies and there are 53 members in the team. If the
communication happened over email and everyone replied as they did to the conversation, 6,572 messages would circulate among the team. Not only
would people have to spend a lot of time deleting some of the thread messages unread, they’d also have to cope with some out-of-office notifications.
Finally, consider the length of the final message and the impossibility of navigating through all the appended replies, auto-signatures, and other junk to
get to the gist of the conversation. A well-organized threaded conversation in one place is a much better host for fast-paced to-and-fro communications.

Figure 13-36: The end of a long Teams conversation

It is also true that conversations appear in channels can become chaotic, especially when multiple team members contribute to different conversations
over a short period. It can be very easy to lose sight of something important in a flurry of messages. It is also important to understand that Teams is
very much an internal-facing collaboration mechanism. Even with guest users joining in, Teams belong to an Office 365 tenant and the data available to
the members of the teams in that tenant stays in the tenant. Compare this to the way that email serves as the lingua franca for the internet where
anyone with an email server can use it to communicate and collaborate with anyone else. Another important factor to consider is the added
functionality available in email, such as the ability to protect messages and attachments with rights management and encryption, not to mention
features to make it easier for users to organize messages, like rules, categories, flagging items for follow-up, and so on. These features take years to
develop and deploy, and Teams is still very much a young application.

Those who champion Teams should consider that all applications have their flaws. Among those seen in Teams are:

Chaotic, badly organized conversations composed of short, incomplete, and shallow sentences that fail to form complete well-thought out ideas. A
conversation might have many replies from many individuals and be totally devoid of analysis and insight.
Conversations that veer away from the subject at hand because they are hijacked by people who fail to understand the topic.
Conversations dominated by the speedy, the quick-witted, those who have a point of view that they want (and need) to express, and those who
suffer from verbal diarrhea. Some members of a team might choose not to contribute to a fast-paced conversation that, although open to them, is
taking place between others.
Conversations about anything that comes into peoples’ heads that mask the real work that should be done in channels.
Conversations about topics driven by people who have no authority to speak on a subject or no accountability for an outcome. It is truly amazing
how much time can be wasted by so many in such discussions.
A rush to decision based on a feeling that chats should develop quickly.
The fear of losing out if you fail to keep pace with all the conversations that happen in all the channels in all the teams you’re a member of. Apart
from flagging a conversation as important and favoring and following channels, it’s hard to prioritize the firehouse of chats daily, let alone sort out
what might have occurred when you return from a vacation.

Email can suffer from some of the same problems, but because your inbox is under your control, it can be easier to sort, ignore, prioritize, and respond
to what flows into the inbox.

Some organizations will find the transition from inward-circulating email to Teams easy and some will decide that they are best off staying with email
and favor Office 365 Groups as their collaboration platform instead, especially when regulations or legal requirements dictate the use of some of the
compliance functionality presently missing in Teams. Interestingly, it can be easier for users of other collaboration platforms, like Lotus Notes, to move
to Teams than it is to move from Outlook. This is possibly because the other platforms are built around databases instead of email.

Usage reports, such as those available in the Office 365 Admin Center, can show whether any email traffic moves to Teams. However, when you
examine usage data, remember that some traffic handled by Teams might replace conversations previously carried in Skype for Business Online or
Yammer. It is also important to measure after the first burst of activity subsides and people start to use Teams on a consistent basis. You will then know
whether overall traffic is static or growing and what modality is most popular with users.

Migration to Teams
Microsoft has no migration tools to import existing content into Teams in such a way that the migrated information is usable within Teams. Instead, if
you want to import content into Teams, you must do so manually. For example, you can email content to a channel from any application that supports
SMTP, or you can move documents into the folders in the SharePoint document libraries belonging to Teams. Several ISV products are available from
companies such as AvePoint, Metalogix, and ShareGate to import documents from SharePoint on-premises servers, file servers, and other sources, or
you can use Microsoft’s free SharePoint Migration Tool .

If you want to move content from another chat platform like HipChat or Slack to Teams, you can export data from these platforms. The difficulty then
arises in how to import that data into Teams in the form of conversations and associated documents. All trans-platform migrations involve some form of
data manipulation to ensure that the data imported into the target platform is usable. The migration code must parse the information taken from the
source platform and update it to fit the data requirements of Teams. For instance, when the items forming a conversation go into a channel, the items
must be in the correct order. The migration must apply permissions to attachments, and so on. User names are probably different, so some process of
fixing these is necessary as otherwise permissions might not work. The lack of a suitable migration API capable of processing large volumes of data
extracted from a platform like Slack and importing the information into Teams in a usable format is an obstacle as ISVs will wait for Microsoft to
deliver a public API that the ISVs can build migration tools around. Until that point arrives, anyone who wants to move information to Teams is on their
own.

If you only want to extract data from another platform and import it into Office 365 for archive or compliance purposes, several ISVs offer tools
designed for this purpose. For example, 17a-4 LLC DataParser can extract data from HipChat and format it into packages that the Office 365 import
service can process. The imported data goes into mailboxes.

PowerShell
Good as Microsoft is when it comes to writing software, they can’t build everything to suit the unique needs of every company that uses Office 365.
PowerShell can go a long way to solving administration problems and filling the gaps in the standard administration toolset. Our next destination
examines how to use PowerShell to manage Office 365 Groups and Teams. Prepare for some scripting!
Chapter 14: Managing Office 365
Groups and Teams with PowerShell
Tony Redmond

The Power of the Shell

The nature of sharing a massive multi-tenant environment means that Microsoft will never be able to satisfy the
unique requirements of every single Office 365 tenant. This is especially so when it comes to managing objects like
Office 365 Groups. A broad range of policy-driven features are available to control aspects like naming, creation, and
expiry, but you might not like how those features work or want to pay for the extra licenses that they need. You might
decide that you prefer to do things differently, and that the best way to reach your goal is to write some PowerShell.
It’s quick, relatively easy, and many examples of how to do different things with PowerShell are available in
repositories like GitHub, the Microsoft Technical Community, or individual web sites.

This chapter looks at how to deal with some management challenges for Office 365 Groups and Teams with
PowerShell. You can find other examples of using PowerShell to manage groups and teams in other chapters where
those examples illustrate a point in context. Collectively, the full range of examples should help you understand what
is possible and how best to approach a task. Some basic experience of using PowerShell to work with objects like
Exchange mailboxes is assumed in the discussion.

Choosing Cmdlets
The cmdlets to manage Office 365 Groups are in the Exchange Online module. Those for Teams are in a separate
module, as are the cmdlets in the Azure Active Directory module to control Azure Groups. In some cases, you will find
that it is easier to perform an operation by running one cmdlet rather than another, so it is possible that your scripts
will use a mixture of Office 365 Groups, Teams, and Azure Active Directory cmdlets. The choice of which cmdlet to use
is entirely personal and it is perfectly acceptable to use a mixture of cmdlets drawn from different modules to get
work done. The sole warning is that it is usually preferable to manage Office 365 Groups with the cmdlets from the
Exchange module and Teams using those from the Teams module.

Performance
PowerShell command vary from one-liners to very complex scripts spanning many hundreds of lines. If you only use
one-liners and an occasional script, you might not need to worry too much about performance. But once you start with
more complicated processing involving multiple steps or hundreds or thousands of objects, it’s wise to consider how
to make your PowerShell code run faster. The suggestions made here are not specific to Groups or Teams and can also
be applied to other Office 365 objects like mailboxes, distribution groups, or SharePoint sites.

Object Filtering
PowerShell supports server-side and client-side filtering:

Server-side filtering is invoked when you include the –Filter parameter in a command. The value passed to the
parameter is the filter you want to apply. Server-side filtering is preferable because the remote server (for
example, Exchange Online or Azure Active Directory) applies the filter before it returns a set of objects to the
PowerShell session.
Client-side filtering happens when you use the Where-Object (or simply Where or the ? Shorthand command) to
process an available set of objects. Client-side filtering happens after a server returns a set of objects to filter the
objects and extract the objects that you need to work with. Because you must wait for more objects to come from
the server before you can apply the filter, the overall time taken to process objects is invariably longer, and often
by several factors.

For example, the two sample commands shown below end up with the same set of objects (Office 365 Groups that
have more than 10 guest members). We use PowerShell’s Measure-Command cmdlet to tell us how long each
command takes to execute, and the result is obvious. The server-side filter takes less than 0.3 seconds to return the
subset of groups that have more than 10 guest users, while fetching all the groups in the tenant and then applying the
filter to that set takes about 14 times longer. Your mileage will vary depending on factors such as the current server
load, the total number of groups in the tenant and the complexity of the filter (and the property used). However,
server-side filtering is invariably faster and should therefore be used when possible when processing any Office 365
objects – mailboxes, groups, sites, teams, and so on.

[PS] C:\> Measure-Command {Get-UnifiedGroup -Filter {GroupExternalMemberCount -gt 10}}} | Select TotalSeconds

TotalSeconds

------------
 0.2947363

[PS] C:\> Measure-Command {Get-UnifiedGroup | ? {$_.GroupExternalMemberCount -gt 10}} | Select TotalSeconds

TotalSeconds

------------

4.2973367

Filter Properties
Not every property of an Office 365 group supports server-side filtering. The cmdlets written for Office 365 Groups
behave like the other recipient-centric cmdlets available in the Exchange Online module, which share many common
filterable properties . Microsoft’s documentation does not specifically cover Office 365 Groups, so some trial and error
is needed to figure out the filterable properties for groups. Here are some examples:

Filter on the custom attributes (1 to 15 for single-value attributes, 1 to 5 for the extension attributes, which
accept multiple-values).

[PS] C:\> Get-UnifiedGroup -Filter {CustomAttribute3 -ne $Null}

[PS] C:\> Get-UnifiedGroup -Filter {ExtensionCustomAttribute5 -eq $Null}

Filter on the Group member counts (total member count and guest member count).

[PS] C:\> Get-UnifiedGroup -Filter {GroupMemberCount -gt 20}

Filter on the display name, name, or alias of groups.

[PS] C:\> Get-UnifiedGroup -Filter {DisplayName -Like "*Office*"}

[PS] C:\> Get-UnifiedGroup -Filter {Name -Like "*(0FF1CE)*"}

[PS] C:\> Get-UnifiedGroup -Filter {Alias -Like "*Corp*"}

Filter on whether groups are hidden from Exchange address lists.

[PS] C:\> Get-UnifiedGroup -Filter {HiddenFromAddressListsEnabled -eq $True}

Filter on Email addresses:

[PS] C:\> Get-UnifiedGroup -Filter {EmailAddresses -Like "*Office*"}

{PS] C:\> Get-UnifiedGroup -Filter {PrimarySmtpAddress -Like "*Exchange*"}

Filter on the created date for a group, or the date when its properties were last updated.

[PS] C:\> Get-UnifiedGroup -Filter {WhenCreated -lt "01/01/2016 00:01"}

[PS] C:\> Get-UnifiedGroup -Filter {WhenChanged -gt "01/01/2018 00:01"}

The Get-Team cmdlet returns the set of teams accessible to the currently-logged in user. The cmdlet does not support
server-side filtering.

ResultSize
By default, many Office 365 cmdlets, including Get-UnifiedGroup , return up to 1,000 objects. If you have more than
1,000 groups or teams in your tenant, you need to specify the number of objects to return as a value passed to the
ResultSize parameter. Alternatively, you can use Unlimited to instruct the cmdlet to retrieve as many objects as exist.
For example:

[PS] C:\> Get-UnifiedGroup -ResultSize Unlimited

Obviously, the more objects returned, the longer it takes to process a command against those objects. Some of the
examples used in this chapter use the ResultSize parameter and some do not. This should not be taken as a general
recommendation and the right usage depends on how many groups exist in a tenant. In some cases, so many objects
exist that it is best to retrieve them for processing in batches. The custom attributes are useful here because you can
assign one of the attributes to hold a batch number and then fetch objects for that batch using a filter. For example:

[PS] C:\> Get-UnifiedGroup -ResultSize 2000 -Filter {CustomAttribute15 -eq "Batch7"}

Process Data Remotely

In a small to medium tenant, it really does not matter too much if you run a cmdlet like Get-UnifiedGroup without any
filters to return all available groups. However, as you scale up to deal with many hundreds or thousands of groups (or
teams), the time taken to run the command increases. For Exchange Online objects like groups, we can speed things
up by having PowerShell run commands on the remote Exchange server to which our session is connected. Things
speed up because the remote server has local access to the data and is likely to have more resources available to
process the data. To ask the remote server to process a command, we call the Invoke-Command cmdlet.

For example, to retrieve the full set of groups and store them in a variable, we can run:

[PS] C:\> $Groups = (Get-UnifiedGroup -ResultSize Unlimited | Select-Object DisplayName, Alias, GroupMemberCount, WhenCreated}

Or we can fetch the same data via Invoke-Command . In this example, the $Session variable points to the remote
session connected to Exchange Online. The session can be found with a command like:

[PS] C:\> $Session = Get-PSSession -InstanceId (Get-OrganizationConfig).RunspaceId.Guid

Now that we know the session, we can send the command to the server:

[PS] C:\> $Groups = (Invoke-Command -Session $Session -ScriptBlock {Get-UnifiedGroup -ResultSize Unlimited | Select-Object
DisplayName, Alias, GroupMemberCount, WhenCreated})

In both cases, the $Groups variable ends up with the same data. Running Get-UnifiedGroup directly is simpler, but
slower than running it via Invoke-Command . The difference is small (probably 10%) when dealing with 100 groups,
but the performance gap widens as more objects are processed, so this technique is something to consider when you
use PowerShell to process large numbers of groups.

Select Properties
If you run a cmdlet like Get-UnifiedGroup or Get-Team , Office 365 returns a set of properties for all matching objects.
You might need all properties, but it is more likely that you only need a few. To minimize the amount of data that
Office 365 returns and speed up processing, use the Select-Object (or just Select ) cmdlet to pick the properties you
need to process. For example, this command returns four properties for all the Office 365 Groups in the tenant.

[PS] C:\> $Groups = (Get-UnifiedGroup | Select DisplayName, Alias, GroupMemberCount, WhenCreated)

Note that Get-UnifiedGroup returns blank values for some properties unless you explicitly request values to be
returned. To ensure that properties are all filled, use the IncludeAllProperties parameter. This slows processing a little
as Exchange Online must calculate some properties before they can be returned.

[PS] C:\> Get-UnifiedGroup -IncludeAllProperties

Avoid Throttling
Office 365 is a multi-tenant environment and applies throttling to stop users or processes consuming more than their
fair share of resources. When a throttling condition exists for PowerShell, Office 365 signals a “micro delay” warning.
Typically, this only happens when you try to process thousands of objects with resource-intense cmdlets like Get-
MailboxFolderStatistics .

If you run into this condition, you can try to avoid throttling by introducing a short delay between each call to the
resource-intense cmdlet by calling the Start-Sleep cmdlet. In this example, we loop through a set of groups to find out
how many conversation items exist in the mailbox belonging to each group. We report the number and the date the
last conversation occurred, and then sleep for 250 milliseconds.

[PS] C:\> ForEach ($G in $Groups) {

$Conversations = Get-MailboxFolderStatistics -FolderScope Inbox -IncludeOldestAndNewestItems

-Identity $G.Alias | Select ItemsInFolder, NewestItemReceivedDate

Write-Host $G.DisplayName "has" $Conversations.ItemsInFolder "conversations in the Inbox. The last is dated"
$Conversations.NewestItemReceivedDate

Start-Sleep -Milliseconds 250 }

Like most tuning exercises, it might take a couple of attempts to figure out the right delay needed to avoid throttling.

Speed Group Retrieval with Get-Recipient

Sometimes you only need to retrieve a list of Office 365 Groups for processing. In these scenarios, the Get-Recipient
cmdlet is much faster than Get-UnifiedGroup . You can prove this theory by using the Measure-Command cmdlet to
report how fast each cmdlet returns information.

[PS] C:\> Measure-Command {Get-Recipient -RecipientTypeDetails GroupMailbox}

[PS] C:\> Measure-Command {Get-UnifiedGroup}

Get-Recipient doesn’t return any group properties, so you need to access the properties separately to process them.
[PS] C:\> $Groups = (Get-Recipient -RecipientTypeDetails GroupMailbox | Select Alias, DisplayName)

ForEach ($G in $Groups) {

$Site = (Get-UnifiedGroup -Identity $G.Alias).SharePointSiteURL

Write-Host "The URL for the SharePoint site for the Group is" $Site }

However, if you want to update properties, this isn’t a big issue because you will make a call to Set-UnifiedGroup
anyway.

[PS] C:\> ForEach ($G in $Groups) {

Write-Host "Setting value for" $G.DisplayName

Set-UnifiedGroup -Identity $G.Alias -CustomAttribute12 "Some value"}

Get-Recipient supports filtering, but you can only use properties common to all recipient types, like DisplayName, the
custom attributes, or the alias. Note that because Get-Recipient returns all recipient types unless instructed
otherwise, you must include a filter for group mailboxes.

[PS] C:\> Get-Recipient -Filter {DisplayName -Like "*Sanjay*" -and RecipientTypeDetails -eq "GroupMailbox"}

If you need to filter the set of groups using group-specific properties (like the number of guest members or the hidden
from Exchange clients flag), use Get-UnifiedGroup as described earlier. Get-Recipient does not support the group-
specific properties.

In summary, consider using Get-Recipient for faster retrieval of sets of Office 365 Groups for processing, but continue
to use Get-UnifiedGroup whenever you need to filter based on group properties.

Basic Management of Office 365

Groups
The basic cmdlets necessary to work with Office 365 Groups are:

New-UnifiedGroup : Create a new Office 365 group.

Get-UnifiedGroup : Return the properties of one or more groups. As noted earlier, sometimes it is faster to use
Get-Recipient to fetch information about groups.
Set-UnifiedGroup : Manipulate the properties of an Office 365 group.
Remove-UnifiedGroup : Remove an Office 365 group and remove all the content contained in the group mailbox,
calendar, document library, and notebook. Note that you can run the Undo-SoftDeletedUnifiedGroup cmdlet to
restore a soft-deleted group within 30 days of its deletion.
Add-UnifiedGroupLinks : Add a user object to the membership of an Office 365 group. You can add users as a
group owner, a member, or a subscriber.
Get-UnifiedGroupLinks : Return information about the membership of an Office 365 group.
Remove-UnifiedGroupLinks : Remove a member, owner, or subscriber from an Office 365 group.

These cmdlets interact with the properties of group objects stored in Azure Active Directory by writing updates to the
copies of those objects stored in EXODS. Afterwards, a synchronization process pushes them to applications such as
Teams and SharePoint Online. Generally speaking, if you need to update something that is specific to an application,
like the email address for an Office 365 Group, you should use the application cmdlets (in this case, Set-UnifiedGroup
). You can use the Azure Active Directory cmdlets to update group membership and other attributes.

Let’s walk through some examples of these cmdlets in action.

Why does (0FF1CE) appear in some group names? The original implementation of Office 365 Groups was not as
comprehensive as it now is. To upgrade groups created during the initial release, Microsoft executed a background
process to move groups to the new type. The lingering vestige of the transition is that the name reported for older
groups end in “(0FF1CE)”. And yes, the placement of that zero and one in the name is quite deliberate. You cannot
assign a different value to the name of the group because it is used as the key back to Azure Active Directory.
However, this is not usually an issue as users only ever see the display name (DisplayName property).

The New-UnifiedGroup Cmdlet

The New-UnifiedGroup cmdlet creates a new group. This operation can sometimes take a little time to complete
because it creates a new group object in Azure Active Directory followed by synchronization of the new object to
EXODS to force the creation of the group mailbox. Only users whose mailboxes are in Exchange Online can run the
New-UnifiedGroup cmdlet.

In this example, we create a new private group called “Corporate Banking Team”. To create a public group, you set
the AccessType parameter to “Public.” Remember, you can change a group’s access type afterwards if needed. Note
that the DisplayName is a required parameter. In this example, we also set:

The owner (if you do not specify an owner, the user who runs the cmdlet becomes the group owner).
The classification of the group to mark it as private.
AutoSubscribeNewMembers so that members are also added to the group’s subscriber list to receive updates via
email.
Create the membership list. You can add as many members as you like by specifying their alias or primary email
address (or their distinguished name, if you like).
Set the alias of the new group.

[PS] C:\> New-UnifiedGroup -DisplayName 'Corporate Banking Team'

–AccessType Private -RequireSenderAuthenticationEnabled $False -Classification Private

-Owner TRedmond -Members "Kim Akers", "Ben Owens" -AutoSubscribeNewMembers -Alias BankingTeam

The documentation for the cmdlet says that you can add multiple owners when you create a group. This doesn’t work,
but the workaround is to create the group with a single owner and then use the Add-UnifiedGroupLinks cmdlet to add
the extra owners. Before you can add someone as an owner, you must add them as a group member.

Unique Alias for Groups

Running New-UnifiedGroup fails if you try to use an alias already assigned to another Office 365 Group. Although
Office 365 allows mail-enabled objects to have duplicate aliases (or MailNickname ), Groups checks the alias value
when it creates a group to ensure that it is unique and fails if it finds a match with another group. However, New-
UnifiedGroup allows you to create a group with an alias already used for another mail-enabled object, such as a mail
contact.

Because each object has a unique SMTP address, the presence of objects with duplicate aliases does not interfere
with mail flow. But because directory synchronization can use the alias as a unique identifier for objects, having
objects with duplicate aliases can cause problems, including when you synchronize groups in hybrid configurations.
Overall, it is best practice to check whether another mail-enabled object already has the alias that you want to use by
running the Get-Recipient cmdlet. For example, this command checks whether another mail-enabled recipient has the
alias “BankingTeam.”

[PS] C:\> Get-Recipient -Filter {Alias -eq "BankingTeam"}

This command only checks mail-enabled recipients that are not Office 365 Groups, so it won’t detect if another group
already uses the alias. This might not be a problem, but if you want to check other mail-enabled recipient type, you
can use a different form of the command to check against various types:

[PS] C:\> Get-Recipient -RecipientTypeDetails GroupMailbox, SharedMailbox, UserMailbox, MailContact, MailUniversalDistributionGroup,

DynamicDistributionGroup, MailUser, RoomMailbox -Filter {Alias -eq "BankingTeam"}

The Set-UnifiedGroup cmdlet also flags an error if you try to update a group with a duplicate alias.

Valid Classification Needed

Another common issue when running New-UnifiedGroup is to pass an invalid value for the classification parameter.
Groups checks the value that you pass against the set specified in the Azure Active Director policy for Groups and the
cmdlet fails if the value is not in the set.

After Creating a Group

After the creation of a group, we can examine its properties with the Get-UnifiedGroup cmdlet. Some of the properties
that you will see are shown below. Note that if you want to see all available properties for a group, specify the -
IncludeAllProperties switch to force Office 365 to return properties such as the InboxURL (a URL pointing to the
Inbox folder in the group mailbox).

[PS] C:> Get-UnifiedGroup –Identity BankingTeam | Format-List

AccessType : Private

AutoSubscribeNewMembers : True

ConnectorsEnabled : True

Alias : BankingTeam

ManagedBy : {Kim Akers, Ben Owens}

EmailAddresses : {SMTP:BankingTeam@Office365ITPros.onmicrosoft.com}

PrimarySmtpAddress : BankingTeam@Office365ITPros.onmicrosoft.com
Name : Corporate Banking Team_89b6fbd74a

DisplayName : Corporate Banking Team

RequireSenderAuthenticationEnabled: True

MaxSendSize : 35 MB (36,700,160 bytes)

MaxReceiveSize : 36 MB (37,748,736 bytes)

RecipientType : MailUniversalDistributionGroup

Language : en-US

SharePointSiteUrl : https://office365itpros.sharepoint.com/sites/BankingTeam

SharePointDocumentsUrl : https://office365itpros.sharepoint.com/sites/BankingTeam/Shared Documents

SharePointNotebookUrl :

(etc.)

Many of the properties supported by Office 365 Groups are familiar because Exchange distribution groups also
support them. Groups calculates the values of some of these properties automatically and you do not have any control
over their values. Several properties are worthy of comment:

The account that runs the New-UnifiedGroup cmdlet becomes a group owner (the ManagedBy property) and a
group member (the Members property). If you do not want this to be the case, you should remove the account
from these properties afterwards. However, you must ensure that at least one group owner always exists, and
that account must also be a group member.
The recipient type is MailUniversalDistributionGroup , which is the same as an Exchange distribution group.
However, the RecipientTypeDetails property for an Office 365 group returns GroupMailbox . When scanning the
directory for recipients, you can therefore find group mailboxes with the Get-Recipient cmdlet (but not the Get-
Mailbox cmdlet) as follows:

[PS] C:\> Get-Recipient -RecipientTypeDetails GroupMailbox

If you want the scan to check for group mailboxes along with other recipient types, you
can do so by specifying the different mailbox types. For instance:

[PS] C:\> Get-Recipient -RecipientTypeDetails GroupMailbox, UserMailbox, SharedMailbox

The URLs for the SharePoint Online resources are not available at once for a new group. These values become
available after SharePoint Online provisions the resources.
Depending on what email address policies are in force, Exchange Online creates one or more email addresses for
a new group. The first, or primary address (shown by the SMTP prefix in capitals), uses the alias of the group
together with the default SMTP domain for the tenant (as in BankingTeam@Office365ITPros.com ). The second
uses the Office 365 “onmicrosoft.com” domain. If you do not have a vanity domain or do not use this domain as
the default, the “onmicrosoft.com” domain becomes the primary address. If you need to change things later, it is
easy to add other email addresses or completely rewrite the set of email addresses as needed by running the Set-
UnifiedGroup cmdlet. As explained in Chapter 10, multi-domain support for groups can apply different forms of
email addresses to new groups instead of the default.
The language set for the group comes from the tenant. In this case, it is “en-US, ” meaning U.S. English. If you
want users to receive group notification messages in other languages, you can change this value with the Set-
UnifiedGroup cmdlet.
The values of the MaxSendSize and MaxReceiveSize properties control the largest message size that the group
mailbox can send and receive. You cannot change these values.
The AutoSubscribeNewMembers property controls whether new members added to the group will receive email
notifications for new conversations. In other words, they join the subscribers list. The default is $False . You can
set this to $True with the Set-UnifiedGroup cmdlet.
The RequireSenderAuthenticationEnabled property works in the same way as a regular distribution group. When
set to $True , the group will only accept messages from accounts known to the tenant. Set the property to $False
when creating the group or with the Set-UnifiedGroup cmdlet afterwards if you want to allow external users to
be able to send email to the group.

Creating Office 365 Groups with Azure Active Directory Cmdlets

Because Azure Active Directory treats Office 365 Groups as just another form of group to manage, you can use the
New-AzureADMSGroup cmdlet from the Azure Active Directory PowerShell module to create a new Office 365 Group.
When you create a new group in Azure Active Directory, synchronization of the new object must happen with
Exchange Online and SharePoint Online before you can access group resources. Although it is better to create Office
365 Groups through OWA or Outlook, if you want to use the Azure Active Directory PowerShell module, use a
command like this.

[PS] C:\> New-AzureADMSGroup -Description “Banking Team” -DisplayName “Banking Managers and Others” -MailEnabled $true -
SecurityEnabled $true -MailNickname “BankingGroup” -GroupTypes “Unified”
Creating a Dynamic Office 365 Group
You can’t create a dynamic Office 365 by running the New-UnifiedGroup cmdlet. Instead, you must use the New-
AzureADMSGroup cmdlet. For example:

[PS] C:\> New-AzureADMSGroup -DisplayName "Sales people in France" -Description "Dynamic group containing all sales people based in
France" -MailEnabled $True -SecurityEnabled $True -MailNickname SalesPeopleFrance -GroupTypes "DynamicMembership", "Unified" -
MembershipRule "(User.Department -eq ""Sales"" and User.Country -eq ""France"")" -MembershipRuleProcessingState "On"

Groups uses the value passed in the MailNickname parameter to create the SMTP address of the group (in this
case, SalesPeopleFrance@Office365ITPros.com) and the group’s alias. If the alias is not unique, Groups adds a
four-digit suffix to make it unique.
The MailEnabled and SecurityEnabled parameters are both set to true, which is what you would expect for an
Office 365 group.
Specifying Unified in the GroupTypes parameter creates an Office 365 group. You must include both
DynamicMembership and Unified in GroupTypes to create a dynamic Office 365 group.
MembershipRuleProcessingState determines whether Azure Active Directory evaluates group membership on an
ongoing basis to find the most up-to-date member set. If you are in the middle of a migration, you can pause
processing (set the value to Paused) to stop the evaluation of group membership against AAD. In this case, the
most recent member set is used.

Because the queries used to figure out the membership of a dynamic group can become quite complex, in most cases,
it is easier to use the GUI in the Azure Active Directory portal to create dynamic groups.

Outputting a List of Groups

After creating some groups, you might want to have a list or those group. This command creates a CSV file listing the
alias, display name, group owner, access type (private or public), the counts for group members and external
members, and the creation date, and then calls Excel to open the file. You can then sort and order the information as
you need.

[PS] C:\> Get-UnifiedGroup -ResultSize Unlimited | Sort DisplayName | Select Alias, DisplayName, ManagedBy, AccessType,
GroupMemberCount, GroupExternalMemberCount, WhenCreated | Export-CSV -Path C:\temp\TenantGroups.CSV -NoTypeInformation -Encoding
Ascii

[PS] C:\> Start Excel C:\temp\TenantGroups.CSV

In this example, we limit the set of groups to report on to those created within the last 90 days.

[PS] C:\> Get-UnifiedGroup |? {$_.WhenCreated -gt (Get-Date).AddDays(-90)} | Format-Table DisplayName, WhenCreated

A variant of the above which uses server-side filtering for better performance is:

[PS] C:\> Get-UnifiedGroup -Filter ([scriptblock]::create("WhenCreated -gt ' $((Get-Date).AddDays(-90)) '"))

Many variants exist for reporting groups. For example, you might want to look for groups or teams that are inactive.
As explained later in this chapter, we can do this by looking for signs of activity in the conversations stored in group
inboxes, team compliance records, or SharePoint documents.

Setting Group Properties

The Set-UnifiedGroup cmdlet manipulates group properties. In this example, we use it to update some of the
properties of our newly created group. The example changes the default language for group notifications, changes the
email address list, tells Office 365 that we want new members to automatically join the subscribers list, and sets up a
MailTip for the group. Just like any other mail-enabled object, the MailTip will be displayed by Outlook and OWA when
a user addresses a message to the group.

PS: C:\> Set-UnifiedGroup –Identity BankingTeam –Language 'fr-FR' –PrimaryEmailAddress 'BankingTeam@Office365ITPros.com' -

AutoSubscribeNewMembers $True –MailTip 'Corporate Banking is great fun' –DisplayName 'Fun with corporate banking'

The language selected for a group does not affect user contributions. Instead, the choice of group language only
affects the information contained on the bottom of notification messages.

Another interesting switch is the ability to allow group members to only have read access to the calendar. Group
Admins continue to have read-write access. For example:

[PS] C:\> Set-UnifiedGroup –Identity "Senior Managers" –CalendarMemberReadOnly:$False

Setting Email Addresses

Like any other mail-enabled recipient, an Office 365 group has at least a primary SMTP address used by Exchange to
route email to the group. In addition, a group can have a number of secondary proxy addresses that Exchange can
also use in routing. The difference between a primary and secondary address is that the primary address is stamped
on ongoing messages from the group and is therefore the reply address for the group. However, Exchange can route
messages using any of the addresses assigned to a group. To set the full set of addresses for a group, run the Get-
UnifiedGroup cmdlet:

[PS] C:\> Get-UnifiedGroup -Identity BankingTeam | Select -ExpandProperty EmailAddresses

smtp:BankingTeam@Office365itpros.com

SMTP:CorporateBanking@Office365ExchangeBook.com

SPO:SPO_b4f8244d-d4ee-46b8-96d5-21d2b6c44c81@SPO_b662313f-14fc-43a2-9a7a-d2e27f4f3478smtp:BankingTeam@office365itpros.onmicrosoft

smtp:BankingTeam@office365itpros.onmicrosoft.com

This group has a primary SMTP address (SMTP is capitalized), two proxy SMTP addresses, one of which is for the
service domain (the last in the list), and a proxy address used by SharePoint Online (SPO). To change the primary
SMTP address, run the Set-UnifiedGroup cmdlet and pass the new address in the PrimarySMTPAddress parameter.
You must specify an address in one of the domains belonging to the tenant:

[PS] C:\> Set-UnifiedGroup -Identity BankingTeam -PrimarySMTPAddress BankingGroup@Office365itpros.com

Setting a new primary address demotes the previous primary address but keeps it as a secondary proxy address.
Therefore, in this example, we start with three SMTP addresses and end up with four. If you need to rewrite the set of
addresses for a group, use the EmailAddresses parameter. For example, here we replace any existing addresses with
two. The first is the primary because it is prefixed with “SMTP:” Exchange Online assumes that an address is SMTP if
you don’t specify its type:

[PS] C:\> Set-UnifiedGroup -Identity BankingTeam -EmailAddresses "SMTP:BankingTeam@Office365itpros.com",

"CorporateBanking@office365itpros.com"

To add a new address without affecting existing addresses, use this syntax:

[PS] C:\> Set-UnifiedGroup -Identity BankingTeam -EmailAddresses @{Add="BankingGroup@Office365itpros.com"}

Similar syntax is used to remove an address. In this case the command will not work because we are trying to remove
the primary address for the group. To remove the primary address, you need to first assign a new primary address to
the group and then remove its old primary address.

[PS] C:\> Set-UnifiedGroup -Identity BankingTeam -EmailAddresses @{Remove="BankingTeam@Office365itpros.com"}

You can add or remove several addresses at one time by specifying the addresses in a comma-separated list.

Setting up Private Groups

Some groups exist to share sensitive information between their membership. It might be the case that you do not
want these groups to be generally known or accessible to non-members, so you can take steps to prevent users
knowing about their existence. The first step is to make sure that the group is private. It does not make sense to have
public access to a sensitive group and making the group private prevents any chance that non-members will discover
documents belonging to the group through a search performed with SharePoint or Delve. You set the access type for a
group by editing its properties or by running the Set-UnifiedGroup cmdlet:

[PS] C:\> Set-UnifiedGroup –Identity “Senior Managers” –AccessType Private

Next, we can hide the group’s existence from address lists to prevent it appearing to users who browse the GAL or
another address list to look for groups that they might like to join. This is an important step to take for groups that
might have sensitive or controversial words in their name, such as “HR Task Group on Layoff Planning” or “Merger
with the Contoso Corporation”. To hide a group, run the Set-UnifiedGroup cmdlet to set the
HiddenFromAddressListsEnabled property as follows:

[PS] C:\> Set-UnifiedGroup –Identity “Senior Managers” –HiddenFromAddressListsEnabled:$True

Hiding a group from the address lists is an Exchange Online setting and only applies to clients that connect to
Exchange. For instance, if you hide a group, it will still appear in Stream or Planner but not in OWA or Outlook.
Members of a hidden group will not be able to see it if they browse an address list. However, they can access the
group through a client. For example, if they expand the Groups list in Outlook 2016, they can see the hidden group
because this list shows all groups to which the user belongs. The usual approach is to have members of hidden groups
mark them as favorites so that the groups are easily accessible.

Groups Hidden from Exchange

As explained in Chapter 13, Teams uses a different property to hide the groups that it creates from Exchange clients
like Outlook and OWA. If the HiddenFromExchangeClientsEnabled property for a group is $True, Exchange clients do
not try to open the group, even if the user is in its membership. If $False, the group is available to Exchange clients.

[PS] C:\> Set-UnifiedGroup -Identity GroupForTeams -HiddenFromExchangeClientsEnabled

Because HiddenFromExchangeClientsEnabled is a switch, you do not need to pass $True if you want to enable it, but
when the time comes to disable the switch, you do have to pass $False.

[PS] C:\> Set-UnifiedGroup -Identity GroupForTeams -HiddenFromExchangeClientsEnabled:$False

Hidden Membership
The New-UnifiedGroup cmdlet supports the HiddenGroupMembershipEnabled switch, which conceals the
membership of a group from everyone except group members and tenant administrators, who can always see and
amend the membership using the Office 365 Admin Center, EAC, or PowerShell. The
HiddenGroupMembershipEnabled switch is only available when you create groups through PowerShell and you
cannot change the hidden status of membership afterwards by running Set-UnifiedGroup . The intention is to block
casual viewing of group membership for sensitive groups like those who are working on special projects or (as needed
by law in some countries), the students in a certain class. An example of creating a group with hidden membership
appears below. Remember to set the group access type to private.

[PS] C:\> New-UnifiedGroup -Alias QTX -HiddenGroupMembershipEnabled -DisplayName "Secret Planning Team" -AutoSubscribeNewMembers -
AccessType Private

To list the groups with hidden membership, use the command:

[PS] C:\> Get-UnifiedGroup | ? {$_.HiddenGroupMembershipEnabled -eq $True} | Format-Table DisplayName, Notes, AccessType

If you create a group with hidden membership, you might also want to hide its presence from Exchange address lists
so that it doesn’t show up in the GAL or OAB. This doesn’t stop existing group members being able to access the
group, but it does stop other users seeing the group through the “Discover” feature. To hide a group from the
Exchange address lists, use the command:

[PS] C:\> Set-UnifiedGroup -Identity Secret Planning Team" -HiddenFromAddressListsEnabled:$True

Teams cannot use groups with hidden membership. The presence of the flag blocks their inclusion in the set of groups
presented to an owner to be potentially enabled for Teams.

Controlling Who Can Send Email to a Group

Like Distribution Groups, you can control the individual users and groups who can send messages to an Office 365
group. To do this, use the AcceptMessagesOnlyFromSendersOrMembers parameter to create the set of individual
senders and groups from whom the group will accept email. You might want to do this to protect sensitive groups
from unwanted communications. For instance, you might want to allow external users to email the group but restrict
this access to some specific email addresses.

The list of individual senders can include mailboxes belonging to the tenant, mail contacts, mail users, and guests.
Valid groups include distribution groups, dynamic distribution groups, and Office 365 Groups. By default, the
parameter value is $Null , meaning that anyone can send messages to the group. Exchange will reject messages from
external senders if the RequireSenderAuthenticationEnabled flag for the group is set to $True .

The following example creates a set of allowed senders for a group using a mixture of email addresses, mailbox
names, and the group itself. Adding the group to its allowed sender list means that Exchange delivers messages sent
by members to the group. You might want to prevent members sending messages to a group, but this is not usually
the desired outcome.

[PS] C:\> Set-UnifiedGroup -Identity "Banking Communications"

-AcceptMessagesOnlyFromSendersOrMembers Flay@outlook.com, "Ben Owens", "Banking Communications"

When Exchange Online creates the allowed sender list for the group, it resolves the addresses specified against Azure
Active Directory and writes the values into three group properties:

AcceptMessagesOnlyFromDLMembers : Stores any reference to the groups on the allowed sender list.
AcceptMessagesOnlyFrom : Individual senders on the allowed sender list
AcceptMessagesOnlyFromSendersOrMembers : The complete list of senders and groups allowed to send to
the group.

We can see these properties with the Get-UnifiedGroup cmdlet:

[PS] C:\> Get-UnifiedGroup -Identity "Banking Communications" | Format-List accept*

AcceptMessagesOnlyFrom : {flay_outlook.com#EXT#, Ben Owens}

AcceptMessagesOnlyFromDLMembers : {BankingCommunications_a6275ce562}

AcceptMessagesOnlyFromSendersOrMembers : {flay_outlook.com#EXT#, Ben Owens,

BankingCommunications_a6275ce562}

As you can see, the addresses used for guests are obvious because of their unique format.

Managing Who Can Edit Group Calendar Items

In some cases, you might not want to allow all the group members to be able to edit items posted to the group
calendar. The most common situation is in education when a class shares a group but only the teacher should be able
to add or update calendar items. You can set the CalendarMemberReadOnly property for a group to restrict add and
update access to group owners. For example:

[PS] C:\> Set-UnifiedGroup -Identity Classroom1 -CalendarMemberReadOnly $True

Currently, the feature only restricts access to group calendar items through OWA.

Using Exchange Online Cmdlets with

Office 365 Groups
Because the Get-UnifiedGroup cmdlet returns an identity suitable for use as the input needed by some of the other
cmdlets used to interact with mailboxes, it follows that you can pipe the identity or a set of identities to these cmdlets
to retrieve some information about group mailboxes. For example, you can use a command like this to discover the
number of messages in conversations stored in group mailboxes:

[PS] C:\> Get-UnifiedGroup | Get-MailboxStatistics | Format-Table DisplayName,

ItemCount, LastLogonTime -AutoSize

DisplayName ItemCount LastLogonTime

----------- - -------- -------------

Accounting Department 6 29/07/2017 13:51:34

Advertising 69 29/07/2017 13:51:28

Ask HR! 15 29/07/2017 13:51:36

Budget Planning 5 29/07/2017 13:51:28

Bug Reports 7 29/07/2017 13:51:31

Exchange MVP discussions 1515 29/07/2017 13:51:25

Human Resources Group 13 29/07/2017 13:51:24

The last logon time shown for each group seems interesting. In reality, it is of little value because no one logs onto
group mailboxes. In fact, although in this instance, the date of last access reported for the group mailboxes all fall into
much the same timeframe, in practice you will see some variation and discover that the last logon date often reflects
background processing by a mailbox assistant rather than the last time a human connected to a group mailbox.

Using the Get-MailboxFolderStatistics cmdlet, we can also see that items accumulate in the Inbox folder (conversation
items), the calendar folder (the shared calendar), and Sent Items (items posted to the group by some clients). All the
other default folders that you would expect to see in a user mailbox are also present.

PS C:\> Get-MailboxFolderStatistics –Identity BankingTeam | Format-Table Name, ItemsInFolder

-AutoSize

Other interesting folders that you see in a group mailbox include Files (information about the items in the SharePoint
team site belonging to the group), Team Chat (compliance records captured for team conversations, if the group is
team-enabled), Junk Email (potential spam and other malware received by the group, which might include false
positives), and the folders in the Recoverable Items structure holding items needed for eDiscovery.

Managing Group Membership

A newly created group is a lonely place because it has no group members apart from the group owner. We can fix the
problem by running the Add-UnifiedGroupLinks cmdlet to add some members to the group. In this example, we add
Kim Akers to the group.

[PS] C:\> Add-UnifiedGroupLinks -Identity BankingTeam -Links Kim.Akers@Office365ITPros.com

-LinkType Members

In this instance, the user principal name specifies the new member, but you can also use a mailbox alias or display
name. The important thing is that the provided value is resolvable to a unique account. Although you can add
unlicensed accounts to an Office 365 Group, users will not be able to access group resources unless they have the
right license. For instance, you cannot access the files in a document library unless you have a license to access
SharePoint Online.

To add a manager (owner) to the group, we change the LinkType to “Owners”. A user must be a member of the group
before they can become an owner. To add a subscriber to the group, change the LinkType to “Subscribers.” Members
who are in the subscribers list receive email updates for new messages sent to conversations. Logically, a mailbox
must be a member of a group before they can become a subscriber.

Office 365 throttles PowerShell processing to make sure that no one process can adversely affect another. If you want
to add a large set of members to a group at one time, it is always both more efficient and less liable to throttling to
include multiple users in a single call to Add-UnifiedGroupLinks. The easiest way to do this is to create an array of the
users that you want to add to the group and then use the array as an input to the cmdlet. In this example, we use a
variable to hold the mailboxes in the membership of a distribution group. The aliases for each of the accounts in the
array can become an input for the Add-UnifiedGroupLinks cmdlet to add the users to the group.

[PS] C:\> $Members = (Get-DistributionGroupMember –Identity MyDL |

Where-Object {$_.RecipientTypeDetails –eq 'UserMailbox'})

[PS] C:\> Add-UnifiedGroupLinks –Identity MyNewO365Group –LinkType Members –Links $Members.Name

Another example scans for all user mailboxes in the tenant not marked with a value in one of the customized
attributes. Again, the resulting array becomes the input to the cmdlet:

[PS] C:\> $Users = (Get-Recipient -RecipientPreviewFilter {RecipientTypeDetails -eq "UserMailbox"

-and CustomAttribute3 -ne "N"}

[PS] C:\> Add-UnifiedGroupLinks -Identity O365Group -LinkTypeMembers -Links $Users.Name

Creating an All Users Group

It is common for organizations to have an “All Users” distribution group to allow for the distribution of company
communications to everyone in a tenant. We can mimic the same approach with Office 365 Groups by creating a
group holding all the mailboxes in the tenant. The steps are simple:

Create a variable holding the details of all user mailboxes in the tenant.
Create the group with the New-UnifiedGroup cmdlet.
Populate the membership of the group with Add-UnifiedGroupLinks. First, we add users as members of the group
and then add them as subscribers.

We give the variable ($Mailboxes) holding the mailbox data to the cmdlet, which treats it as an array because it holds
multiple objects. To avoid confusion, we tell Exchange Online to use the distinguished name property for each object
in the array to add that user to the membership. Here is the code:

[PS] C:\> $Mailboxes = Get-Mailbox –RecipientTypeDetails 'UserMailbox'

[PS] C:\> New-UnifiedGroup –Alias AllMailboxes –DisplayName 'Company Communications'

[PS] C:\> Add-UnifiedGroupLinks –Identity AllMailboxes –LinkType Members

–Links $Mailboxes.DistinguishedName

[PS] C:\> Add-UnifiedGroupLinks –Identity AllMailboxes –LinkType Subscribers

–Links $Mailboxes.DistinguishedName

[PS] C:\> Get-UnifiedGroupLinks –Identity AllMailboxes –LinkType Members | Format-Table DisplayName

PowerShell does not flag an error if a user already exists in a member list. The user who runs the New-UnifiedGroup
cmdlet automatically joins the newly created group as its owner. You can add more owners by running the Add-
UnifiedGroupLinks cmdlet. Because we created this group manually, you must add new members as the need arises.

Later in this chapter, we describe a script to create a team for all employees. Similar steps are used, adjusted for
cmdlets in the Teams PowerShell module.

Checking Group Membership

To examine the membership of a group, we can use the Get-UnifiedGroupLinks cmdlet. The value passed to the
LinkType parameter is either Members, Owners, or Subscribers to tell the cmdlet what membership information to
display. For example:

[PS] C:\> Get-UnifiedGroupLinks –Identity BankingTeam –LinkType Members

Name RecipientType
---- -------------

TRedmond UserMailbox

Kim Akers UserMailbox

Sometimes we might want to know how many groups someone belongs to. One way to approach this problem is to
loop down through all the groups in the tenant and check whether the user is in the membership list. It’s slow, but
effective, as shown in this code, which asks for a user to check and then goes to work. The output is a variable
($Groups) holding the display name and alias for each group that is usable as an input to another function or to create
a CSV or text file.

[PS] C:\> $Mailbox = Read-Host "Enter User to check"

$Alias = Get-Mailbox -Identity $Mailbox

If ($Alias -ne $Null) {

$Groups = (Get-UnifiedGroup | ? { (Get-UnifiedGroupLinks $_.Alias -LinkType Members | ForEach {$_.Name}) -contains $Alias.Name} |
Select DisplayName, Alias) }

Write-Host $Alias.DisplayName "is a member of" $Groups.count "Groups"

$Groups

Enter User to check: Kim Akers

Kim Akers is a member of 100 Groups

DisplayName Alias

----------- -----

Ignite 2016 Ignite2016

Managers Managers

To check what groups someone is an owner of, simply change the link type in the code from “Members” “Owners.”
However, it is faster to use the Get-AzureADUserOwnedObject cmdlet to fetch information about group owners direct
from Azure Active Directory. For example, to see the set of groups owned by a user, you could use the code shown
below. The set returned by Get-AzureADUserOwnedObject includes security groups in addition to Office 365 Groups,
so that’s the reason for filtering the results.

[PS] C:\> $Mailbox = Read-Host "Enter User to check"

$CheckUser = (Get-Mailbox -Identity $Mailbox | Select DisplayName, Alias, ExternalDirectoryObjectId)

If ($CheckUser -ne $Null) {

$Groups = ( Get-AzureADUserOwnedObject -ObjectId $CheckUser.ExternalDirectoryObjectId | ? {$_.MailEnabled -eq $True} | Select

DisplayName, Description)}

Write-Host $CheckUser.DisplayName "is the owner of" $Groups.count "Groups"

$Groups

A much faster approach to finding out how many groups a member belongs to is to use the distinguished name of the
mailbox as a filter for the Get-Recipient cmdlet. The first parameter ensures that the cmdlet only returns group
mailboxes, while the filter checks the membership of each group to see if the member exists. Apart from not having to
call a separate cmdlet to check group membership, this is a good example of how server-side filtering speeds up
processing.

[PS] C:\> $Mailbox = Read-Host "Enter User to check"

$DN = Get-Mailbox -Identity $Mailbox | Select -ExpandProperty DistinguishedName

$Groups = (Get-Recipient -RecipientTypeDetails GroupMailbox -Filter "Members -eq '$DN'" | Select DisplayName, Alias)

Write-Host $Mailbox "is a member of" $Groups.count "Groups"

$Groups | Select DisplayName, Alias

Checking if Someone is in a Group

No off-the-shelf cmdlet exists to allow you to check whether a user is already a group member. For small groups, it is
possible to check by iterating through the membership to look for a match, but once group membership grows past a
hundred or so entries, that kind of processing becomes very slow. One way to check is to use code like that shown
below to compare the object identifier for a user against an array holding the set of object identifiers that make up the
group membership. This is rough and ready code that does not include error checking.

[PS] C:\> $Check = (Read-Host "Enter User Principal Name to check")

$User = (Get-AzureADUser -ObjectId $Check)

$Obj = $User.ObjectId

$CheckGroup = (Read-Host "Enter name of group to check")

$TargetGroup = (Get-UnifiedGroup -Identity $CheckGroup)

$GroupId = $TargetGroup.ExternalDirectoryObjectId

$GroupMembers = (Get-AzureADGroupMember -ObjectId $GroupId | Select ObjectId)

If ($GroupMembers -Match $Obj) {

Write-Host $User.DisplayName "is part of" (Get-AzureAdGroup -ObjectId $GroupId).DisplayName}

You could also use a modified version of the code used to discover the set of groups to which someone belongs. The
downside is that this code only works for users with mailboxes.

[PS] C:\> $Mailbox = Read-Host "Enter User to check"

$GroupName = Read-Host "Enter Group to check"

$DN = Get-Mailbox -Identity $Mailbox | Select -ExpandProperty DistinguishedName

$Group = Get-UnifiedGroup -Identity $GroupName | Select Alias, DisplayName

$Groups = (Get-Recipient -RecipientTypeDetails GroupMailbox -Filter "Members -eq '$DN'" | Select DisplayName, Alias)

If ($Groups -Match $Group.Alias) {

Write-Host $Mailbox "is in group" $Group.DisplayName }

Else { Write-Host $Mailbox "is not in group" $Group.DisplayName }

Dealing with Users and Guests

We can do a little better by building a hash table from the group membership and then check a user against the hash
table. This is more efficient and faster than using a simple array and it allows us to deal with groups that include
guest members. In this example, we create a hash table from the membership of the All Employees group. The table
consists of keys (user display names) and values (their email addresses), and looks something like this:

Name Value

---- -----

Sanjay Patel Sanjay.Patel@office365itpros.com

Tony Redmond Tony.Redmond@office365itpros.com

Jack Healy Jack.Healy@office365itpros.com

Imran Khan Imran.Khan@office365itpros.com

Guest users have the string “#EXT#” in their name, so we need to do some processing to normalize the names in the
table for tenant users and guests. If you want to exclude guests, comment out the lines that process these members.
After we build the table, the next thing is to ask for an SMTP address to check for in the group and then look for that
address in the hash table. If the code finds a match in the table, we report success.

[PS] C:\> $GroupMembers = @{}

# Populate the array with current group membership

$OrgName = “@”+ (Get-OrganizationConfig).Identity

$TargetGroup = "ExchangeGoms"

Get-UnifiedGroupLinks -LinkType Member -Identity $TargetGroup | % {

$User = $_.Name.tostring()
 # Handle guests

if ($User -like "*#EXT#*") {

$GuestUPN = $User+$OrgName

$GuestUser = Get-AzureADUser -ObjectId $GuestUPN -ErrorAction SilentlyContinue

$GroupMembers.Add($GuestUser.DisplayName, $GuestUser.Mail) }

else {

# local mailbox

$LocalUser = $Email = (Get-Mailbox -Identity $User)

$GroupMembers.Add($LocalUser.DisplayName, $LocalUser.PrimarySmtpAddress) }

$Check = (Read-Host "Enter SMTP address to check")

If ($GroupMembers.Values -Match $Check) {

Write-Host "User" $Check "is in the group" $TargetGroup }

Still another approach to the problem is to create a hash table holding the groups to which someone belongs and then
check against that table. The code is simpler because you assume that the user you check for has a mailbox.

[PS] C:\> $Groups = @{}

$Check = (Read-Host "user to check")

$CheckUser = (Get-Mailbox -Identity $Check)

$Dn = $CheckUser.DistinguishedName

$TargetGroup = "GDPR Planning"

# Populate the array with current group membership

Get-Recipient -Filter "Members -eq '$Dn'" -RecipientTypeDetails GroupMailbox | % {

$GroupName = $_.Name

$Group = $_.DisplayName

$Groups.Add($Group, $GroupName) }

If ($Groups.ContainsKey($TargetGroup) -eq $True) {

Write-Host "User" $CheckUser "is in the group" $TargetGroup }

The hash table used to hold details of the groups that someone belongs to is probably smaller too as an individual user
is unlikely to be member of 2,500 groups. However, apart from not supporting guests, the downside is that you must
create a hash table for every user you want to check instead of using a single table holding group membership that
you can check every candidate member against.

Now that we know how to check whether a user is already a member of a group, we can apply the technique when we
add a new employee to the Company Communications group. Of course, in this case, we check for a False result
because we only want to add the user if they are not already a member.

[PS] C:\> Check = (Read-Host "Enter user to check")

# Check that the user has a mailbox and can be a group member

$CheckUser = (Get-Mailbox -Identity $Check -ErrorAction SilentlyContinue)

If ($CheckUser -ne $Null) {

# Go ahead and add - if they are not a group member

If ($GroupMembers.ContainsKey($CheckUser.DisplayName) -eq $False) {

Add-UnifiedGroupLinks -Identity $TargetGroup -LinkType Member -Links $CheckUser

Write-Host $CheckUser "added to" $TargetGroup }

 Else {

Write-Host $CheckUser "is already a member of" $TargetGroup }

Removing Members from Groups

To remove a member from an Office 365 group, run the Remove-UnifiedGroupLinks cmdlet and follow some simple
rules:

Only a group owner or a tenant administrator can remove a user from a group.
You must specify the link type of the list from which you want to remove the user – Owners, Members, or
Subscribers.
If a user is a group owner, you must remove their owner status first and then remove them as a group member.
If a group member has subscribed to the group for email updates, you do not have to remove this link separately
as the removal of their membership also removes their subscription.

Exchange Online caches link information to improve performance so changing the links for a group might take some
time before the change is fully effective across a tenant. As an example, here is how to remove Kim Akers as a
member of the Banking Team group:

[PS] C:\> Remove-UnifiedGroupLinks –Identity BankingTeam –LinkType Members –Links 'Kim Akers'

As discussed earlier, it is possible that you might want to remove a certain member from all groups in a tenant. For
example, you discover that some group owners invite a guest from a competitor company to join their groups. Another
example is where you want to put a user account into a state where their mailbox receives no further communications
and their account cannot access information held in groups. The code to check for and remove users from groups is
straightforward.

First, here is some code to check how many groups a user belongs to. The $CheckValue variable holds a valid mailbox
name (the user to check), because this is what the Get-UnifiedGroupLinks cmdlet reports for each member. The code
checks that the input is a valid mailbox before going ahead to check group membership and report the number of
groups the user belongs to.

To handle the removal of guest accounts, we could adapt the code to support the input of values such as
“France_outlook.com#EXT#, ” or we simply remove the guest account from Azure Active Directory as described
earlier in this chapter.

[PS] C:\> $CheckValue = (Read-Host "user to check")

$Mbx = (Get-Mailbox -Identity $CheckValue -ErrorAction SilentlyContinue)

If ($Mbx -ne $Null) {

Write-Host "Fetching details of groups…"

$DN = $Mbx | Select -ExpandProperty DistinguishedName

$Groups = Get-UnifiedGroup -Filter "Members -eq '$DN'"

Write-Host $Mbx.DisplayName "found in" $Groups.Count "Groups"}

Else { Write-Host $CheckValue “is not a valid mailbox name” }

The next step is to remove the membership link from the groups where the user is present. This code asks the
administrator if they want to go ahead and remove the user from the groups. Upon confirmation, the code first checks
whether a user is a group owner and removes that link before it removes the member link. If you want to process
guests, you do not need to check owner lists because guests cannot become group owners.

[PS] C:\> $NextStep = Read-Host $User.DisplayName "found in" $Groups.Count "Groups. Do you want to remove the user from the groups?"

If ($NextStep.SubString(0,1).ToUpper() -eq "Y")

Write-Host "Removing" $User.DisplayName "from Office 365 Groups..."

ForEach ($G in $Groups)

# Have to remove them as an owner first

$Owners = Get-UnifiedGroupLinks -Identity $G.DistinguishedName -LinkType Members

If ($Owners -like $CheckValue)

 {

Write-Host "Removing" $User.DisplayName "as owner of group" $G.DisplayName

Remove-UnifiedGroupLinks -LinkType Owners -Links $CheckValue -Identity $G.DistinguishedName

-Confirm:$False -ErrorAction SilentlyContinue

# Now remove their membership link

$Members = Get-UnifiedGroupLinks -Identity $G.DistinguishedName -LinkType Members

If ($Members -like $CheckValue)

Write-Host "Removing" $User.DisplayName "as member of group" $G.DisplayName

Remove-UnifiedGroupLinks -LinkType Members -Links $CheckValue -Identity $G.DistinguishedName

-Confirm:$False -ErrorAction SilentlyContinue

}}

Although this code does an effective job of cleaning up member and owner links for a selected user, it might have the
side-effect of leaving some groups without any owners. We will handle that issue shortly.

Another way to handle the task is to remove the guest account from Azure Active Directory. Removing the guest
account instantly removes the user from all membership lists.

Checking Groups with Guests

Another common reason to examine groups is to figure out how many groups have guests and how many guests are in
the membership of each group. Remember, it doesn’t matter whether guests were added through Groups, Teams, or
Planner – the guests all end up in the same membership. We can carry out our task by checking the membership count
data held for each group. In this case, we fetch all the groups in the tenant and sort them by total member count. We
then output the total member count, the count of guests, and a computed value for the number of users in the tenant.

[PS] C:\> Get-UnifiedGroup -Filter {GroupExternalMemberCount -gt 0} | Sort-Object GroupMemberCount -Descending | Format-Table -
AutoSize DisplayName, GroupMemberCount, GroupExternalMemberCount, @{Name=”Tenant Users”; Expression = {$_.GroupMemberCount -
$_.GroupExternalMemberCount}}

DisplayName GroupMemberCount GroupExternalMemberCount Tenant Users

----------- ---------------- ------------------------ ------------

Exchange's Grumpy Old Men 62 57 5

Office 365 Discussion 61 55 6

Office 365 Engage 2017 Speakers 42 41 1

Company Communications 28 0 28

The member counts recorded in group properties should only ever be treated as guidance. The actual number might
be off by 1 or 2 in some groups before the background process which updates these numbers catches up with recent
changes. If you want to be absolute about member counts, you must check the individual members in each group.

Removing Unwanted Guests

Even with an Azure B2B Collaboration allow or deny list in place (see Chapter 12), it is possible that some unwanted
guests already exist in group memberships because group owners added them before you implemented the block, or
because you forget to add a domain to the blocked list. To fix the problem, you can scan the memberships of groups
with guests to check where guests come from. Because Groups and Teams share common membership, this code
reports guests added through Teams too. This script does the job and tells us how many guest memberships exist, how
many groups have guests, and which group has the largest number of guests. Make sure that you change the
reference to @tenantname in the code with your actual tenant name before running the script.

[PS] C:\> $Groups = (Get-UnifiedGroup -Filter {GroupExternalMemberCount -gt 0} | Select Alias, DisplayName, SharePointSiteURL,
GroupExternalMemberCount)
If ($Groups.Count -gt 0) {

Write-Host "Processing" $Groups.Count "groups with guest members"

$Report = @()

$NumExt = 0

$LargestGroup = $Null

$LargestGroupNum = 0

ForEach ($G in $Groups) {

Write-Host "Processing" $G.DisplayName

$Users = Get-UnifiedGroupLinks -Identity $G.Alias -LinkType Members

ForEach ($U in $Users) {

If ($U.Name -Match "#EXT#" -and $U.Name -NotLike "*teams.ms*") {

$NumExt++

$CheckName = $U.Name + "@ tenantname.onmicrosoft.com"

$User = (Get-AzureADUser -ObjectId $CheckName).DisplayName

$ReportLine = [PSCustomObject][Ordered]@{

Email = $U.Name

User = $User

Group = $G.DisplayName

Site = $G.SharePointSiteURL }

$Report += $ReportLine }

If ($G.GroupExternalMemberCount -gt $LargestGroupNum) {

$LargestGroupNum = $G.GroupExternalMemberCount

$LargestGroup = $G.DisplayName}

Write-Host $NumExt "guest user memberships found in" $Groups.Count "groups"

Write-Host "Largest external group is" $LargestGroup "with" $LargestGroupNum "guests"

The script creates an ordered array of the guest users found in group membership. It is easy to sort the array and look
at the data in whatever way you like. For example, you could sort by user to be able to see what groups each user
belongs to.

[PS] C:\> $Report | Sort User | Format-Table User, Group, Site -AutoSize

U ser Group Site

--- - ---- ----

Ailbhe Smith (Hotmail) Office 365 Tenant Health https://tenant.sharepoint.com/sites/office365Health

Ian Byrne Dynamic Passion https://tenant.sharepoint.com/sites/DynamicPass

I an Byrne Exchange Trades https:// tenant.sharepoint.com/sites/exchangetrades

Bulk Removal of Guests

If you find problematic guests in group memberships, you can remove guest accounts selectively or in bulk. This is not
something to do without thinking (or testing) as removing a guest account ends any access it has to SharePoint and
OneDrive for Business resources in the tenant as well as its membership of Office 365 Groups and Teams. Here is how
to remove an individual guest account.
[PS] C:\> Get-AzureADUser -SearchString "Flowers_outlook.com#EXT#" | Remove-AzureADUser -Force

As an example of bulk removal, this code looks for guest accounts belonging to a certain domain (in this case,
Badpeople.com) and removes them from Azure Active Directory.

[PS] C:\> $Users = (Get-AzureADUser -Filter "UserType eq 'Guest'" -All $True | Select DisplayName, ObjectId)

[PS] C:\> Foreach ($U in $Users)

{ If ($U.UserPrincipalName -Like "*BadPeople.com*") {

Write-Host "Removing"$U.DisplayName

Remove-AzureADUser -ObjectId $U.ObjectId }

Alternatively, if you do not want to remove the guest account from the tenant, you can run the Remove-
UnifiedGroupLinks cmdlet to remove the guest from the membership of any groups or Teams (running the Remove-
TeamUser cmdlet has the same effect) to which they belong. Later examples in Chapter 14 explain how to use this
cmdlet.

Synchronizing Office 365 Groups with Security Groups

You might have a set of security groups that you use to control access to information and want to use the same
membership of those security groups with Office 365 Groups or Teams. No synchronization exists within Office 365
today to keep the membership of a security group aligned with an Office 365 group or vice versa, but it’s reasonably
easy to do with PowerShell.

Let’s take an example where you have a security group and want to have an equivalent Office 365 group (or team). In
this case, the “eDiscovery Admins” security group is the master and the “eDiscovery Administrators” Office 365 is the
slave. Synchronization is done from master to slave. Assuming the Office 365 group already exists, we fetch the
membership of the security group and add any user account found there to the Office 365 group.

[PS] C:\> $O365Group = (Get-UnifiedGroup -Identity "eDiscovery Administrators")

$SecurityGroup = (Get-AzureADGroup -SearchString "eDiscovery Admins")

# Grab list of security group members

$SecurityGroupMembers = (Get-AzureADGroupMember -ObjectId $SecurityGroup.ObjectId | Select UserPrincipalName, Membertype)

# Populate the Office 365 Group with the members of the security group

ForEach ($i in $SecurityGroupMembers) {

If ($i.UserType -eq "User") {

Add-UnifiedGroupLinks -Identity $O365Group.Alias -LinkType Member -Links $i.UserPrincipalName }

This process doesn’t handle nested groups, so if the security group includes a nested group, you’ll have to extract the
membership from that group and process those entries.

After we write the security group membership into the Office 365 group, we need to do some added processing to
fully synchronize the two groups. The security group is the master and some other members might have been
previously added to the Office 365 group who do not belong to the security group. To complete synchronization, we
need to compare the two memberships and remove anyone found in the Office 365 group who doesn’t exist in the
security group. Again, this is a straightforward operation:

[PS] C:\> $GroupMembers = (Get-UnifiedGroupLinks -Identity $O365Group.Alias -LinkType Member)

# Check if any Office 365 group members do not exist in the security group

ForEach ($i in $GroupMembers) {

$Member = (Get-Mailbox -Identity $i.Name)

If ($SecurityGroupMembers -Match $Member.UserPrincipalName)

{Write-Host $Member.DisplayName "is in security group" }

Else
 { Write-Host "Removing" $Member.DisplayName "from Office 365 group because they are not in the security group" -
ForeGroundColor Red

Remove-UnifiedGroupLinks -Identity $O365Group.Alias -Links $Member.Alias -LinkType Member -Confirm:$False}

Write-Host "Current Membership of" $O365Group.DisplayName

Get-UnifiedGroupLinks -Identity $O365Group.Alias -LinkType Member

To be fully effective, you would need to set up a periodic batch job to synchronize the two memberships. Some more
robust error checking and handling of nested groups would improve the script too.

Flagging Unowned Groups

All groups should have at least one owner. If groups end up lacking owners, it means that no one except
administrators can add new members or perform other group management functions and that administrators must
process group expiry notifications. The following PowerShell commands flag any groups that are missing owners.

[PS] C:\> $BadGroups = (Get-UnifiedGroup -Filter {ManagedBy -eq $Null})

[PS] C:\> $BadGroups | Format-List DisplayName, WhenCreated, AccessType

[PS] C:\> Write-Host $BadGroups.Count “groups have no owners”

Another interesting approach to the reporting of Office 365 Groups is downloadable from the TechNet Gallery . In this
case, a PowerShell script checks the creation, deletion, and updates to groups within a tenant. The script can then
email the resulting report to an administrator. Like all PowerShell code, you are free to adapt it to the needs of your
tenant.

Another way of looking at group ownership is to figure out what groups a certain user owns. You can do this by
running the Get-Recipient cmdlet to return a list of group mailboxes and using a filter on the ManagedBy property to
extract the groups owned by the person that you want to check. The only complicating factor is that you must use the
user’s distinguished name in the filter. For example, this PowerShell finds all the groups where Kim Akers is an owner.

[PS] C:\> $Dn = (Get-Mailbox -Identity "Kim Akers").DistinguishedName

[PS] C:\> Get-Recipient -RecipientTypeDetails GroupMailbox -Filter "ManagedBy -eq '$Dn'" | Format-Table DisplayName

Another way is to ask Azure Active Directory for information about the objects it believes someone owns:

[PS] C:\> $ObjectId = (Get-Mailbox -Identity "Kim Akers").ExternalDirectoryObjectId

[PS] C:\> Get-AzureADUserOwnedObject -ObjectId $ObjectId | Select DisplayName, Description

Checking for Unused Groups

The group expiration policy gives tenants an automated method to age out groups after a set period. However, you
might want to check how active groups are before they become candidates to expire to understand the reasons why
users create groups and what happens to those groups in the months afterwards. To achieve this goal, we can use
PowerShell to find any groups where no activity has occurred recently. Depending on their use, groups display
different signs of activity. A group used by a team who build their work around Outlook is likely to use conversations
heavily while another team might use Teams for their communications. Table 14-1 describes how users interact with
groups and the consequent dependency on the two primary workloads.

Type of group Files stored in SharePoint Items stored in Exchange

Online Online

Office 365 Groups: documents Yes N/A

Office 365 Groups: conversations and shared N/A Yes

calendar

Yammer Groups: documents Yes N/A

Yammer Groups: discussions and shared N/A Yes (calendar)
calendar

Group created by Microsoft Planner Yes Yes (discussions)

Group created by Microsoft Teams Yes Yes (calendar and compliance)

Table 14-1: Exchange and SharePoint usage in various group types

You can check the potential underuse of groups used for document generation and management by looking for
evidence of SharePoint file activity in the site as recorded in the Office 365 Audit Log (Chapter 20). After connecting a
PowerShell session to SharePoint Online and Exchange Online, the code shown below checks to see whether any audit
records generated by SharePoint file operations over the last 90 days exist for the site belonging to each group. If no
audit records for SharePoint file operations for a site are available, it’s reasonable to assume that not much document-
centric activity has taken place in the site and it is therefore potentially obsolete.

[PS] C:\> $WarningDate = (Get-Date).AddDays(-90)

$Today = (Get-Date)

$Groups = Get-UnifiedGroup

$ObsoleteGroups = 0

ForEach ($G in $Groups) {

If ($G.SharePointDocumentsUrl -ne $Null)

$SPOSite = (Get-SPOSite -Identity $G.SharePointDocumentsUrl.replace("/Shared Documents", ""))

Write-Host "Checking" $SPOSite.Title "..."

$AuditCheck = $G.SharePointDocumentsUrl + " /*"

$AuditRecs = 0

$AuditRecs = (Search-UnifiedAuditLog -RecordType SharePointFileOperation `

-StartDate `$WarningDate -EndDate $Today -ObjectId $AuditCheck `

-SessionCommand ReturnNextPreviewPage)

If ($AuditRecs -eq $null)

Write-Host "No audit records found for" $SPOSite.Title `

"-> It is potentially obsolete!"

$ObsoleteGroups++

Else

Write-Host $AuditRecs.Count "audit records found for " $SPOSite.Title `

"the last is dated" $AuditRecs.CreationDate[0]

}}

Else

{
 Write-Host "SharePoint has never been used for the group" $G.DisplayName

$Obso leteGroups++

Write-Host $ObsoleteGroups "obsolete group document libraries found out of" $Groups.Count "checked"

Groups that are email-centric need a different approach because we need to track the activity level for conversations.
In this case, we use the Get-MailboxFolderStatistics cmdlet to check the date and time for the last item (conversation)
added to the Inbox. If that conversation occurred within the last year, all is well, and we can go on. If not, we flag the
group as potentially obsolete. Checking the date of the last item is a simple sign of activity. We can improve it by
checking how many items are in the Inbox. If just a few items exist, it is a sign that the group processed the first
notification message sent following its creation and a couple of other conversations, and then gently fell into decay.

[PS] C:\> $Groups = Get-UnifiedGroup

$BadGroups = 0

$WarningDate = (Get-Date).AddDays(-365)

ForEach ($G in $Groups) {

Write-Host "Checking Inbox traffic for" $G.DisplayName

$Data = (Get-MailboxFolderStatistics -Identity $G .Alias -IncludeOldestAndNewestItems -FolderScope Inbox)

If ($Data.NewestItemReceivedDate -le $WarningDate)

Write-Host "Last conversation item created in" $G.DisplayName "was" `

$Data.NewestItemReceivedDate "-> Could be Obsolete?"

$Bad Groups++

Else

Write-Host $G.DisplayName "has" $Data.ItemsInFolder `

"conversation items amounting to" $Data.FolderSize

Write-Host $BadGroups "Obsolete Groups found out of" $Groups.Count

We can take the same approach to check whether Teams are active. Offices 365 saves records of channel
conversations in the Conversation History\Team Chat folder. This means that we can check the usage of a team-
enabled group by looking at the newest item in Conversation History (and its subfolders). Note that if you only want to
process team-enabled groups, use the Get-Team cmdlet to create a collection of teams and then process each group.

$TeamCheckDate = (Get-MailboxFolderStatistics -Identity $G.Alias -IncludeOldestAndNewestItems -FolderScope

ConversationHistory).NewestItemReceivedDate

If ($TeamCheckDate -le $WarningDate)

Write-Host "Last Teams conversation created in" $G.DisplayName "was" `

$TeamCheckDate "-> Might be Obsolete?" }

Unfortunately, you cannot use the same approach for plan-enabled groups because Planner does not capture records
recording tasks or buckets.

These code examples use different methods to find potentially underused groups. It would not take a lot of added
effort to add some code to combine the results and make a better decision as to which groups really are obsolete and
those that only use either SharePoint Online or Exchange Online. An example script that combines checks of both
SharePoint Online and Exchange Online to find and report potentially obsolete Office 365 Groups is available in the
TechNet Gallery .

You can use the “Office 365 Groups activity” report available in the Office 365 Admin Center to identity inactive
groups based on the last activity date, but the nice thing about approaching the problem through PowerShell is that
you have a chance to tailor the assessment and the output to meet your needs, such as including details of group
classification, access type, or using one of the custom properties that can be assigned to groups to hold information
about its current status.

One problem shared by both the Office 365 Groups activity report and any attempt to detect inactive groups via
PowerShell is that the focus is on activity created in Exchange and SharePoint. Groups that use Yammer for
conversation storage might show up as inactive because discussions in these groups occur in areas that PowerShell
cannot detect.

Blocking Guest Access to Classified

Groups
Chapter 12 discusses how to use a policy to disable guest access for a selected group. If you use classifications in your
tenant, you probably have a classification like “Company Confidential” or “Secret” to mark groups (or teams) that hold
sensitive information. In some cases, you might want to make sure that these groups do not have guest users. Another
example is when a school wishes to block guest access to any group used for student communications.

It’s easy to solve the problem with PowerShell if a suitable classification is assigned to sensitive groups. With the
classifications in place, we can scan for those groups and then block them all for guest access. In this example, the
first step is to create a set of groups classified as “Confidential.” The code then loops through each group to discover
whether group-specific policy settings are in place. If so, the code updates the settings to block guest access. Groups
that don’t have a policy setting are controlled by the tenant policy, so the first step is to create policy settings for the
group. We can then update the setting to block guest access.

[PS] C:\> $GroupTemplate = (Get-AzureADDirectorySettingTemplate | ? {$_.DisplayName -eq "Group.Unified.Guest"})

$Groups = (Get-UnifiedGroup -ResultSize Unlimited | Where {$_.Classification -eq "Confidential"})

ForEach ($Group in $Groups) {

$GroupSettings = Get-AzureADObjectSetting -TargetType Groups -TargetObjectId

$Group.ExternalDirectoryObjectId

if($GroupSettings) {

# Policy settings already exist for the group - so update them

$ GroupSettings["AllowToAddGuests"] = $False

Set- AzureADObjectSetting -Id $GroupSettings.Id -DirectorySetting $GroupSettings

-TargetObjectId $Group.ExternalDirectoryObjectId -TargetType Groups

Write-Host "External Guest accounts prohibited for" $Group.DisplayName

Else

# Settings do not exist for the group - so create a new settings object and update

$S ettings = $GroupTemplate.CreateDirectorySetting()

$Settings["AllowToAddGuests"] = $False

New-AzureADObjectSetting -DirectorySetting $Settings -TargetObjectId

$Group.ExternalDirectoryObjectId -TargetType Groups

Write-Host "External Guest accounts blocked for"$Group.DisplayName

}
The update performed in the code occurs in Azure Active Directory. Synchronization to the workload directories must
happen before clients learn about the new status for a group and the block is effective.

To check that the block is effective, we can create a list of the groups blocked from having guest members. To do this,
run the Get-UnifiedGroup cmdlet to check the AllowAddGuests property, which is $False for blocked groups. For
example, this command reports the display names and classification for all blocked groups. Remember that the block
is effective for all clients that populate group membership, including Teams.

[PS] C:\> Get-UnifiedGroup | ? {$_.AllowAddGuests -eq $False } | Format-Table DisplayName, Classification

Archiving Inactive Office 365 Groups

As explained above, over time, it’s likely that some groups will become inactive as the original need wanes or member
interest fades. It’s good management practice to review groups on a regular basis to understand their usage and to
identity potentially inactive groups with an eye to doing something with them. After you’ve found some inactive
groups, you can keep these groups online and available, let Office 365 expire and then remove the groups, or put
them into a state where the group content still is available for compliance purposes and an administrator can
reactivate the group if needed.

Projects tend to have a natural lifetime. For example, let’s assume that you spin up a group to support the planning
and coordination of a financial project. After the project finishes, the group holds conversations about project issues,
the calendar of meetings, and all the documents related to the project. Depending on the compliance regime that
applies to the company operations, you might have to keep this information might for an extended period. Unlike
mailboxes, which administrators can put into an inactive state to preserve their content in a state that is inaccessible
to users, we need a different approach to archive group content.

Conceptually, the steps to archive a group are easy. The aim is to put the group into a state where we hide the content
from general view while the content stays indexed and discoverable for compliance purposes.

Add a new group owner. Ideally, this should be a special compliance administration account instead of a tenant
administrator. Remember, before you can add an owner, they must first be a member of the group.
Remove all owners from the group’s membership list.
Remove all users from the group’s membership list.
Ensure that the group is private so that its documents stay invisible to Delve or other searches.
Set RequireSenderAuthenticationEnabled property to $True to stop Exchange Online accepting any external
email sent to the group. Guest members are not subject to this check as they can always email a group until their
membership is removed.
Assign a new primary SMTP address to the archived group and remove the old address to stop the delivery of
new messages from users inside the tenant.
Hide the group from Exchange clients so that it does not appear in address lists.
Optionally, use some of the custom properties available for group mailboxes to record some information about
the group’s archived status as well as information about who archived the group.

Although Microsoft doesn’t offer an out-of-the-box method to archive Office 365 Groups (Teams offers the option to
archive a team by setting it to read-only mode – see Chapter 13), we can do the job with PowerShell. Here’s a script to
archive a group based on the steps outlined above.

[PS] C:\> $CheckGroup = Read-Host -Prompt "Enter alias of group to archive"

$AGroup = (Get-UnifiedGroup $CheckGroup -ErrorAction SilentlyContinue)

If ($AGroup) {

Write-Host "Archiving" $AGroup.DisplayName -ForegroundColor Yellow

} Else {

Write-Host $CheckGroup "group not found - terminating"

Return }

# Get lists of current owners and members

$CurrentOwners = (Get-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Owners | Select Name)

$CurrentMembers = (Get-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Members | Select Name)

# Add a new owner - this is the address of the account that will continue to access the group

$AdminAccount = "Compliance Administrator"

Add-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Members -Links $AdminAccount

Add-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Owners -Links $AdminAccount

# Remove the other members and owners

ForEach ($O in $CurrentOwners) {

Remove-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Owners -Links $O.Name

-Confirm:$False}

ForEach ($M in $CurrentMembers) {

Remove-UnifiedGroupLinks -Identity $AGroup.Alias -LinkType Members -Links $M.Name

-Confirm:$False}

# Create SMTP Address for the archived group

$OldSmtpAddress = $AGroup.PrimarySmtpAddress -Split "@"

$NewSmtpAddress = $OldSmtpAddress[0] + "_archived" + "@" + $OldSmtpAddress[1]

$AddressRemove = "smtp:"+$AGroup.PrimarySmtpAddress

$ArchiveInfo = "Archived " + (Get-Date) + " by " + $ O365cred.username

# Update Group properties

Set-UnifiedGroup -Identity $AGroup.Alias -AccessType Private -RequireSenderAuthenticationEnabled

$True -HiddenFromExchangeClientsEnabled -CustomAttribute1 "Archived"

-CustomAttribute2 $ArchiveInfo -PrimarySmtpAddress $NewSmtpAddress

Set-UnifiedGroup -Identity $AGroup.Alias -EmailAddresses @{remove=$AddressRemove}

Write-Host $AGroup.DisplayName "is now archived and" $AdminAccount "is the new group owner"

A short time after the script runs, the group will disappear from clients. The exact time depends on the client. It is
fastest for OWA because that client reads information from the online directory. It is slowest for Teams because of the
need to synchronize the membership changes with the Teams directory. Because we remove people from the group
membership, they lose access to any resource available to the group like Planner or SharePoint.

Group Reactivation
To reactivate the group, all we need to do is to add a new owner, who can then add new members to allow them
access to group resources. We should also reverse some of the settings made to group properties to make it available
to clients. The email address probably doesn’t need to be changed and you don’t need to change the setting to require
email authentication unless you want to allow external users who are not guests to be able to email the group. The
following command is probably enough:

[PS] C:\> Set-UnifiedGroup -Identity ArchivedGroupAlias -AccessType Private

-HiddenFromExchangeClientsEnabled:$False -CustomAttribute1 $Null -CustomAttribute2 $Null

Tracking Archived Groups

As mentioned above, an optional but useful step is to record information about the archival of a group. Two custom
properties store the status and information about when the archival occurred together with the user name of the
account that ran the script. The user name comes from the credentials used to sign onto Exchange Online with the
PowerShell session and the variable used will be different, depending on how you connect. Marking archived groups
in this manner makes it easier to find the groups if needed. Here’s how we can find the groups archived with the
script:

[PS] C:\> $Output = @{Expression = {$_.DisplayName}; label = "Group name"; width=30}, @{Expression = {$_.CustomAttribute2}; Label =
"Archived on and by"}

[PS] C:\> Get-UnifiedGroup -Filter {CustomAttribute1 -eq "Archived"} | Format-Table $Output

Group name Archived on and by

---------- ------------------

Sales Department team Archived 07/15/2017 15:14:01 by Tony.Redmond@office365itpros.com

Nikon Camera Club Archived 09/09/2017 23:15:12 by Administrator@office365itpros.com

Office 365 Archived 01/10/2018 16:18:47 by Administrator@office365itopros.com

Working with the Group Expiration Policy

Version 2.0.0.137 of the Azure Active Directory PowerShell module (or later) includes cmdlets to work with the group
expiration policy. Two sets of cmdlets are available.

The *-AzureADMSGroupLifecyclePolicy cmdlets manipulate the expiration policy settings.

The *-AzureADMSLifecyclePolicyGroup cmdlets are used when you want the expiration policy to process selected
groups rather than apply to every group in the tenant. These cmdlets add or remove groups to the policy to bring
them under the scope of the policy.

Because all Office 365 Groups in a tenant can come within the scope of the group expiration policy, you should be
aware that the settings you apply through the policy affect all applications that use Groups. In other words, the
expiration of a group affects Teams, Planner, Power BI, and other applications that use Groups to control their
membership.

To retrieve the settings for the current expiration policy, you run this command:

[PS] C:> Get-AzureADMSGroupLifecyclePolicy | Format-List

Id : 23036082-ac71-49ed-a8fd-24db5ad38379

GroupLifetimeInDays : 1000

ManagedGroupTypes : All

AlternateNotificationEmails : Tony.Redmond@office365itpros.com

To update a policy setting, run the Set-AzureADMSGroupLifecyclePolicy cmdlet. In this case, we set the expiration
interval to 750 days and update the backup group owner (the person who receives notifications about expiring groups
when no group owner is available).

[PS] C:\> Set-AzureADMSGroupLifecycePolicy -id (Get-AzureADMSGroupLifecyclePolicy).Id -GroupLifeTimeInDays 750 -

AlternateNotificationEmails Ben.Owens@Office365itpros.com

As an example of applying the policy to a selected set of groups instead of every group in the tenant, here is how to
make the policy only process groups classified as “Confidential.” We need to create a collection of groups marked as
such and then add each group to the policy. The following code does the trick, if you set the policy to process selected
groups rather than all. Note that you cannot assign different expiration periods to individual group. All groups
covered by the policy use the same expiration period.

[PS] C:\> $Groups = (Get-UnifiedGroup | ? {$_.Classification -eq "Confidential"}

$PolicyId = (Get-AzureADMSGroupLifecyclePolicy).Id

ForEach ($G in $Groups) {

Add-AzureADMSLifecyclePolicyGroup -GroupId $G.ExternalDirectoryObjectId -id $PolicyId }

Although you can access this information in the Azure portal, Microsoft does not include a cmdlet to return a list of
groups subject to the policy. To create a set of groups subject to the policy, you must check each group in the tenant
using the Get-AzureADMSLifeCyclePolicyGroup cmdlet to confirm that its ManagedGroupTypes property is not null.
Thus, to find the groups within the scope of the expiration policy, we can do something like this:

[PS] C:\> $Groups = (Get-Recipient -RecipientTypeDetails GroupMailbox)

ForEach ($G in $Groups) {

$Status = $Null

$Status = (Get-AzureADMSLifecyclePolicyGroup -Id $G.ExternalDirectoryObjectId).ManagedGroupTypes

If ($Status -ne $Null) {

$Days = (New-TimeSpan -Start $G.WhenCreated -End (Get-Date)).Days

Write-Host "Group:" $G.DisplayName " status:" $Status "date created:" $G.WhenCreated "days old:" $Days }

To check on the progress of renewal, you can look at the last renewal date and calculate the next renewal date based
on the lifecycle period set in the policy. To do this, we take the code we use for checking whether a group is subject to
the policy and add some lines to calculate the renewal date and report what we find. If you see a minus figure
reported for “days before expiration,” it means that this is a group recently added to the policy that has already
expired.
[PS] C:\> $LifeCycle = (Get-AzureADMSGroupLifeCyclePolicy).GroupLifeTimeInDays

$Report = @()

$Today = (Get-Date)

$GroupsinPolicy = 0

Write-Host “Finding Groups to check…”

$Groups = (Get-UnifiedGroup | Select DisplayName, ExternalDirectoryObjectId, WhenCreated)

Write-Host $Groups.Count “found. Now checking expiration status.”

ForEach ($G in $Groups) {

$Status = $Null

$Status = (Get-AzureADMSLifecyclePolicyGroup -Id $G.ExternalDirectoryObjectId).ManagedGroupTypes

If ($Status -ne $Null) {

$Days = (New-TimeSpan -Start $G.WhenCreated -End $Today).Days

$LastRenewed = (Get-AzureADMSGroup -Id $G.ExternalDirectoryObjectId).RenewedDateTime

$NextRenewalDue = $LastRenewed.AddDays($Lifecycle)

$DaysLeft = (New-TimeSpan -Start $Today -End $NextRenewalDue).Days

$GroupsInPolicy++

$ReportLine = [PSCustomObject][Ordered]@{

Group = $G.DisplayName

Created = $G.WhenCreated

AgeinDays = $Days

LastRenewed = $LastRenewed

NextRenewal = $NextRenewalDue

DaysLeft = $DaysLeft

$Report += $ReportLine }

Clear-Host

Write-Host "Total Groups in tenant:" $Groups.Count "Total Groups covered by expiration policy:" $GroupsInPolicy

$Report | Select Group, @{n="Last Renewed"; e= {$_.LastRenewed}}, @{n="Next Renewal Due"; e={$_.NextRenewal}}, @{n="Days before
Expiration"; e={$_.DaysLeft}}

Total Groups in tenant: 157 Total Groups covered by expiration policy: 18

Group Last Renewed Next Renewal Due Days before Expiration

----- ------------ ---------------- ----------------------

Office 365 for IT Pros 1 2-Mar-18 2:07:16 PM 31-Mar-20 2:07:16 PM 749

Corporate Accounting (Billing) 2 4-May-17 9:33:29 AM 13-Jun-19 9:33:29 AM 457

Ben Owens Reports 2 8-Sep-16 12:58:54 PM 18-Oct-18 12:58:54 PM 219

The New Hydra Project Team 02-Nov-16 9:38:47 PM 22-Nov-18 9:38:47 PM 254

Hydra Due Diligence 29-Nov-17 11:11:31 AM 19-Dec-19 11:11:31 AM 646

European Office 365 Engage 23-Jan-17 3:06:05 PM 12-Feb-19 3:06:05 PM 336

Managing Teams with PowerShell
The Teams PowerShell module is still a preview version (0.9.6 is the current version available from the PowerShell
Gallery ). In terms of the administrative control over Teams, the module has improved steadily from the first release
and is now capable of handling most of the administrative operations available in the Teams and Skype for Business
Online Admin Center. Cmdlets are available to create teams (including a team for an existing Office 365 group), list all
teams in the tenant, update settings for teams, and create new channels. Because the module is still evolving, some
change is inevitable before the final release. Even with this caveat, you can depend on the current release for
production use.

Before you can use the Teams module, you must download it from the PowerShell gallery and then install the module
on your PC. To update to the latest release, use PowerShell as an administrator and run the following commands to
remove any earlier version and download and install the latest module.

[PS] C:\> Uninstall-Module -Name MicrosoftTeams

[PS] C:\> Install-Module -Name MicrosoftTeams -Repository PSGallery

After installing the module, you run the Connect-MicrosoftTeams cmdlet to connect to the Teams service. You can then
check that you have the latest module installed by running the command:

[PS] C:\> Get-Module |? {$_.Name -eq "MicrosoftTeams"} | Select Version

Version

-------

0.9.6

It is often the case that cmdlets from other modules are needed to fill in gaps in the Teams module, so in most cases
you will connect your session to Exchange Online and Azure Active Directory. Documentation for the Teams cmdlets is
available online .

Permissions
Your account must have permissions to be able to run some of the Teams cmdlets.

To create a new team: Tenant administrator, Teams service administrator, or be allowed to create new Office 365
Groups.
To change team settings, including updating team membership: Tenant administrator, Teams service
administrator, or owner of the target team.
To list team settings, including team membership: member of the team.

Listing Teams
To list the teams in a tenant, run the Get-Team command without any other parameter. You should always use Get-
Team to return a list of teams instead of fetching a set of groups with Get-UnifiedGroup and then trying to figure out
which groups are team-enabled. Because it does not return the large set of properties processed by Get-UnifiedGroup
, Get-Team is much faster.

[PS] C:\> Get-Team

The reason why Get-Team is so fast at retrieving a list of teams is partially explained by the fact that it returns just
three values for each team:

Group identifier.
Display Name.
Description.

The group identifier is the most important value. It is a GUID that uniquely identifies a team and is used as the input
to most of the other cmdlets in the Teams module. In addition, you can use the group identifier to fetch information
about the underlying group from Azure Active Directory or the mail-enabled properties using the Office 365 Groups
cmdlets.

The list of teams is returned in order of creation date. To view the list in another order, use the Sort-Object cmdlet to
sort by the DisplayName, Description, or GroupId properties.

[PS] C:\> Get-Team | Sort DisplayName

To list the teams that the currently logged-in user is a member of, specify the User parameter and pass their full User
Principal Name:

[PS] C:\> Get-Team -User Kim.Akers@Office365itpros.com

You cannot use this command to fetch details of the teams that another user belongs to. If you want to find the teams
that a user belongs to, use Get-Team to fetch a list of teams and then Get-TeamUser to discover whether the user is a
member of each team.

For example, to get information from Azure Active Directory about the team membership, input the group identifier as
the object identifier for Get-AzureADGroupMember :

[PS] C:\> Get-AzureADGroupMember -ObjectId 129bfa64-3d11-4d6f-b9a9-6c20ec03bd8d

To use the group identifier with the Office 365 Groups cmdlets, pass it as the identity. For example, here’s how to get
the same membership using Get-UnifiedGroupLinks :

[PS] C:\> Get-UnifiedGroupLinks -LinkType Member -Identity 129bfa64-3d11-4d6f-b9a9-6c20ec03bd8d

Combining Teams with Other Modules

The limited properties returned by Get-Team means that you often need to use other cmdlets, including those in other
modules, to solve problems. As noted above, the identifier returned in the properties can be used with the Office 365
Groups cmdlets. In this example, we use Get-Team to populate a variable holding details of the teams in a tenant and
then use the Get-UnifiedGroup cmdlet to fetch some interesting properties about the underlying group, and the Get-
TeamUser cmdlet to fetch details of the team owners.

[PS] C:\> $Report = @()

$Teams = Get-Team | Sort DisplayName

Write-Host "Reporting" $Teams.Count "teams"

ForEach ($T in $Teams) {

# Fetch information about the team - mostly by getting it from the Unified Group Object

Write-Host "Processing" $T.DisplayName

$G = (Get-UnifiedGroup -Identity $T.GroupId | Select GroupMemberCount, GroupExternalMemberCount, PrimarySmtpAddress, AccessType,

Classification, WhenCreated)

$Owners = (Get-TeamUser -Role Owner -GroupId $T.GroupId | Select Name)

$OwnerNames = $Null

$First = $True

ForEach ($O in $Owners) {

If ($First -eq $True) {

$OwnerNames = $O.Name

$First = $False}

Else {

$OwnerNames = $OwnerNames + ", " + $O.Name }}

$ReportLine = [PSCustomObject][Ordered]@{

Team = $T.DisplayName

Email = $G.PrimarySmtpAddress

Members = $G.GroupMemberCount

Guests = $G.GroupExternalMemberCount

Owners = $OwnerNames

Access = $G.AccessType

Created = $G.WhenCreated

Classification = $G.Classification }

$Report += $ReportLine }

$Report | Export-Csv -NoTypeInformation c:\temp\Teams.csv

Filtering Get-Team
The Get-Team cmdlet does not include a Filter parameter to select a subset of teams, probably because the cmdlet
returns such a limited set of properties. However, you can trade some of the performance by applying a client-side
filter to the display name or description properties to find specific teams. For example:

[PS] C:\> Get-Team | ? {$_.DisplayName -Like "*Office*"} | Format-Table DisplayName, Description

We can combine a filtered lookup for a specific team to return its identifier and use that value to fetch information
from Azure Active Directory or the Office 365 group:

[PS] C:\> $Group = (Get-Team | ? {$_.DisplayName -eq "Office 365 Questions"} | Select GroupId)

[PS] C:\> Get-UnifiedGroup -Identity $Group.GroupId

The lack of good filtering for Get-Team doesn’t matter as much when your tenant only has a few hundred teams. It
becomes more of an issue as the number of teams increases.

Creating New Teams

You can create a new team in two ways: either create a brand-new team from scratch or team-enable an existing
Office 365 group. If you add a new team from scratch, you also need to add some users.

The New-Team cmdlet applies in both cases. In this example, we create a new team from scratch and set its access
type to Private and apply one of the classifications defined for the tenant. To specify the owner of the new team, pass
their User Principal Name in the Owner parameter:

[PS] C:\> $TeamId = (New-Team -DisplayName "Planning Co-Ordinators" -Alias Coordinators -Description "Team for the folks who make
sure we turn up in the right place at the right time" -AccessType Private -Classification Confidential -Owner
James.Ryan@office365itpros.com)

The $TeamId variable holds the GUID for the newly-created team. We can use the GUID to identify the newly created
team when we go to create members and owners. When you specify an owner for a new team, New-Team adds them
both as a member and as an owner, but when you add new owners, you must decide whether they are an owner or an
owner/member. Someone who is an owner can access the team and participate in discussions. However, if they are not
also a member, this causes problems for applications like Planner which check that someone is a member rather than
an owner before allowing access to resources. SharePoint also has an issue with owners who aren’t members as these
people cannot search for documents (using SharePoint or Delve) stored in the site belonging to the team.

It is therefore best to follow the rule to add new owners both as an owner and as a member unless you have good
reason to do otherwise.

[PS] C:\> Add-TeamUser -GroupId $TeamId.GroupId -User Brian.Weakliam@Office365itpros.com -Role Member

[PS] C:\> Add-TeamUser -GroupId $TeamId.GroupId -User Donald.Vickers@Office365itpros.com -Role Owner

[PS] C:\> Add-TeamUser -GroupId $TeamId.GroupId -User Donald.Vickers@Office365itpros.com -Role Member

After the team membership is populated, you can proceed to amend group settings and create some channels – or
consider posting a welcome message to the General channel in the new team.

Posting a Welcome Message to a New Team

A new team begins with no content. Sometimes it’s nice to give members of the team some guidance about the
purpose of the team and how to use it. PowerShell is used for administrative operations and the Teams module doesn’t
include a cmdlet to create and post messages to a Teams channel, but this can be done by calling the Graph API from
PowerShell.

The basic requirement is to register a new app with Azure to act as the entry point for read-write access to Teams via
the Graph. The details of how to register an app are explained in this blog post . When you have an app, you can use it
to authenticate with the Graph and then post Graph commands via PowerShell. In this example, we have created a
new team and added the logged-in user to the team (otherwise they won’t be able to post to a channel). The identifier
for the new team is stored in the $GroupId variable. We find the identifier for the General channel in the target team,
use an array to compose some HTML text that we want to post, convert the array to JSON, and then post the message
to the channel using the Graph.

[PS] C:\> $DisplayName = "Project Condor"

$GroupId = (Get-Team |? {$_.DisplayName -eq $DisplayName}).GroupId

$Description = (Get-Team |? {$_.DisplayName -eq $DisplayName}).Description

$MessageLine = "Team description:" + $Description

$ChannelId = (Get-TeamChannel -GroupId $GroupId | ?{$_.DisplayName -eq "General"}).id

$apiUrl = "https://graph.microsoft.com/beta/teams/$Groupid/channels/$ChannelID/chatThreads"

# End up with something like https://graph.microsoft.com/beta/teams/41fd1ff0-099b-46b9-8b55-

e49c8e378383/channels/19:ba56cf956b0a47d991cb953906f605ee@thread.skype/chatThreads

$body = @{

"rootMessage" = @{

"body" = @{

"contentType" = 1;

"content" = "

<h1>Welcome to $DisplayName</h1><p><b><i>Welcome to your new Team.</b></i></p> <p>We have some basic rules of Teams etiquette that we
would like you to follow to make everyone's life easier: </p>

<ul><li>Always start a new topic with a <strong>message subject</strong>. </li><li>When you have something to say about a topic,
<strong>reply to that topic</strong> and don't start another thread.</li><li>Use <strong>@mentions </strong>to bring something
important to the attention of individuals, teams, or a channel.</li></ul><p>$MessageLine</p>"

}}

$bodyJSON = $body | ConvertTo-Json

$Msg = Invoke-RestMethod -Headers @{Authorization = "Bearer $accessToken"} -Uri $apiUrl -Method Post -Body $bodyJSON -ContentType
'application/json'

Figure 14-1 shows the welcome message generated by the code.

Figure 14-1: A welcome message for a new team generated by a Graph call through PowerShell

This code demonstrates the principle of what needs to be done to post to a team using PowerShell and needs some
more work before it can be incorporated into the workflow to create a new team.

Creating a Team from an Exiting Office 365 Group

To create a new team based on an existing Office 365 group, run New-Team and pass the identifier (GUID) for the
Office 365 group in the Group parameter. For example:

[PS] C:\> New-Team -Group (Get-UnifiedGroup -Identity "BoardMembers").ExternalDirectoryObjectId

You can only team-enable an Office 365 group if your account is an owner of the group, a global administrator, or a
Teams Service administrator.

Controlling the Alias

The Alias parameter (also known as MailNickname ) for the New-Team cmdlet controls the generation of the SMTP
primary address for a new team, or rather, the underlying Office 365 group. The alias must be unique. If you don’t
pass a value to the Alias parameter, Teams generates unique values for the Name , Alias , and PrimarySMTPAddress
properties (when the Teams client creates a new team, it creates the alias based on the team’s display name). For
example, here’s what Teams generates for a new team as viewed with the Get-UnifiedGroup cmdlet when no value is
passed for Alias :

Name : msteams_3e8d41_245f9292-29ca-4773-9240-de5fcee03853

Alias : msteams_3e8d41

PrimarySmtpAddress : msteams_3e8d41@office365itpros.com

The name given to the team is composed of “msteams,” an underscore, and the team’s Azure Active Directory object
identifier. The alias is derived from the name and the primary SMTP address uses the alias and the default tenant
domain. Users only ever see the display name of a team, so having such a value in its name doesn’t make any
difference.

If you use Teams messaging for communication, you probably aren’t concerned about the email address assigned to
the group, so generating automatic values for these properties might not be an issue, but if you want to control the
email address assigned to a new team, include the name you want to use in the Alias parameter. For instance, if I pass
“Ignite2019” in the Alias, the following values are used:

Name : Ignite2019_bb966263-a263-4731-8454-afbd7be22b81

Alias : Ignite2019

PrimarySmtpAddress : Ignite2019@office365itpros.com

Make sure that any alias you pass to New-Team is unique as the cmdlet will fail if you try to use a value that already
exists for another object in the tenant.

Creating an All-Employees Team

Teams includes the ability to create automatically-maintained org-wide teams, but only up to a limit of 1,000 accounts,
which is lower than the 2,500-member limit for a team. Fortunately, PowerShell enables us to create an all-employees
team with a membership composed of everyone in the tenant who has a mailbox.

The group identifier is needed to add members to a team. Usually, we would fetch a group identifier using either the
Get-UnifiedGroup or Get-AzureADGroup cmdlets, but in this case, we put the result of the call to the New-Team
cmdlet in a variable so that the identifier is available for later use. Another interesting aspect is how we use Get-
Recipient to fetch a set of current user mailboxes because it’s the fastest method to get this data. You could also use
Get-Mailbox or Get-AzureADUser to create a set of input members. In all cases, remember to fetch the right property
to get the user’s primary email address as this is needed to identify the user when you add them to the team – if you
use Get-Mailbox , the property is WindowsEmailAddress ). No attempt is made to check whether a user has a valid
license to use Teams before we add them to the membership. This is a refinement that could be easily added by using
the cmdlet and commands described in Chapter 5.

[PS] C:\> $Tenant = (Get-OrganizationConfig).Name

$Tenant = $Tenant.Split(".")[0]

$TeamName = "All-Employees ("+$Tenant+")"

$NewTeam = (New-Team -DisplayName $TeamName -Alias AllEmployeesTeam -Description "All-employee team" -AccessType Public)

$Mailboxes = (Get-Recipient -RecipientTypeDetails UserMailbox -ResultSize Unlimited)

$Owner = (Get-TeamUser -GroupId $NewTeam.GroupId -Role Owner)

# Add all the mailboxes except the owner, who's already there

ForEach ($M in $Mailboxes) {

If ($M.WindowsLiveId -ne $Owner.User) {

Add-TeamUser -GroupId $NewTeam.GroupId -User $M.WindowsLiveID -Role Member }

Set-UnifiedGroup -Identity AllEmployeesTeam -HiddenFromExchangeClientsEnabled

Set-TeamMemberSettings -GroupId $NewTeam.GroupId -AllowCreateUpdateChannels $False

-AllowDeleteChannels $False -AllowCreateUpdateRemoveTabs $False

-AllowCreateUpdateRemoveConnectors $False -AllowAddRemoveApps $False

The last two commands in the script hide the new team from Exchange clients and restrict the ability of team
members to add or remove channels, connectors, and tabs from the team as it’s unlikely that you would want to allow
members to change the structure of an all-employees team. The resulting team is a skeleton that needs to be built out
with channels, tabs, and apps. In addition, you should restrict posts to the general channel and add some owners to
moderate what is often a busy forum. Finally, if you add new employees to the company after running the script, you
must add them to the membership manually.

If you have more than 2,500 employees, Yammer might be a better solution for an all-employees forum.

Viewing Team Settings

A set of cmdlets is available to view and update the various settings that control how members interact with a team.
The cmdlets that deal with team settings mimic what’s available through the client user interface. First, the Get-
TeamMemberSettings cmdlet reveals what members can do to the structure of the team (add channels, etc.):
[PS] C:\> Get-TeamMemberSettings -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4

AllowCreateUpdateChannels : True

AllowDeleteChannels : True

AllowAddRemoveApps : True

AllowCreateUpdateRemoveTabs : True

AllowCreateUpdateRemoveConnectors : True

Use the Get-TeamMessagingSettings cmdlet to see the settings that control whether users can edit or remove
messages and use team and channel mentions in conversations. Your account must be listed as an owner of the team
before you can update its settings:

[PS] C:\> Get-TeamMessagingSettings -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4

AllowUserEditMessages : False

AllowUserDeleteMessages : False

AllowOwnerDeleteMessages : False

AllowTeamMentions : True

AllowChannelMentions : True

To see the settings that control guest functionality within in a team, use the Get-TeamGuestSettings cmdlet:

[PS] C:\> Get-TeamGuestSettings -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4

AllowCreateUpdateChannels AllowDeleteChannels

------------------------- -------------------

False False

To see the “fun stuff” settings for a team, use the Get-TeamFunSettings cmdlet:

[PS] C:\> Get-TeamFunSettings -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4

AllowGiphy GiphyContentRating AllowStickersAndMemes AllowCustomMemes

---------- ------------------ --------------------- ----------------

True moderate True False

Updating Team Settings

You can update the settings for a team using the Set-TeamMemberSettings , Set-TeamMessagingSettings , Set-
TeamGuestSettings , and Set-TeamFunSettings cmdlets: for example:

[PS] C:\> Set-TeamMessagingSettings -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -AllowTeamMentions $False

It is possible that you might not be able to change some settings because of restrictions set at the tenant level.

Team owners can run Set-TeamPicture cmdlet to update the image used for the team in listings. You can use PNG,
JPEG, or GIF images, but make sure to size the image appropriately (100 x 80 pixels works well). Only owners can
update the picture for a team. For example:

[PS] C:\> Set-TeamPicture -GroupId 34d68904-9d7c-4ef7-b715-eed283e80243 -ImageFile c:\temp\TeamFile.jpg

Channels
To return the channels in a team, use the Get-TeamChannel cmdlet:

[PS] C:\> Get-TeamChannel -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4

Id DisplayN ame Description

-- ----------- -----------

534ebe88-8236-4e68-acd4-6935b126fdca General What’s happening in Office 365

f1dc5034-b5d5-4d01-b97b-14f2d37dfc31 The EHLO blog Posts from the EHLO blog

d286c712-6d6a-406f-84fb-74981e9276c2 MVP Tweets Tweets from MVPs

e8d93767-9017-4050-9782-6fd0eda4073c News from Petri.com News from Petri.com

c265e899-6905-4824-9c8a-95d86b0df585 Group Email Email sent to the group

To change a channel setting, you pass the group identifier and the current channel name to the Set-TeamChannel
cmdlet. You cannot update the settings of the General channel.

[PS] C:\> Set-TeamChannel -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -CurrentDisplayName "News from Petri.com" -NewDisplayName
"Petri.com News" -Description "RSS feed from Petri.com"

The New-TeamChannel cmdlet adds a channel to a team. For instance:

[PS] C:\> New-TeamChannel -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -DisplayName "Bug Tracking" -Description "A place to track
bugs as we discover them in our favorite product"

The Remove-TeamChannel cmdlet is available to remove a channel from a team.

Viewing Team Membership

The Get-Team cmdlet returns a list of the teams to which the logged-in user belongs. The cmdlet only returns three
properties, the most important being the group identifier, or GroupId , which we use with most of the other cmdlets.
This is the key to find the group in Azure Active Directory.

[PS] C:\> Get-Team

GroupId DisplayName Description

------- ----------- -----------

8836e3ca-6531-402a-ac5f-b71c83d6affa Mail Filter Patent Discussing the Mail Filter patent

84fe8ec1-d85f-4564-a786-1f7bedd9862c Engineering Testers People who do testing for engineering

72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 Office 365 Chat What’s happening in Office 365

For example, to list the membership of a team, we use the Get-TeamUser cmdlet. This example outputs the display
name (Name), role, and Azure Active Directory account (User) for each member. Guest members are clearly marked.

[PS] C:\> Get-TeamUser -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 | Format-Table Name, Role, User

Name Role User

---- ---- ----

Tony Redmond owner Tony.Redmond@ office365itpros.com

Jeff Guillet member Jeff.Guillet@ office365itpros.com

Administrator member Administrator@office365itpros.com

Vasil Michev (MVP) guest Vasil@Michev.info#EXT#@Office365itpros.onmicrosof...

You can output just the people with a specific role by including the -Role parameter and passing either Member or
Owner:

[PS] C:\> Get-TeamUser -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -Role Owner

Get-TeamUser can return the membership of any team if you know the group identifier, which you can find out from
Azure Active Directory. For example:

[PS] C:\> Get-TeamUser -GroupId (Get-AzureADGroup -SearchString "A random group").ObjectId

Alternatively, use the Get-UnifiedGroup cmdlet to retrieve the group identifier:

[PS] C:\> Get-TeamUser -GroupId (Get-UnifiedGroup -Identity Hydra).ExternalDirectoryObjectId

Get-TeamUser returns the group membership for any kind of Azure Active Directory group as the cmdlet does not
check that the target group is an Office 365 group. This proves that Get-TeamUser is simply another way to call the
Get-AzureADGroupMember cmdlet. Likewise, the other -TeamUser cmdlets are proxies for the underlying -
AzureADGroupMember cmdlets.

Adding and Removing Team Members

The Add-TeamUser cmdlet adds a new member or owner to a team. Although Teams and Groups share a common
membership list, you do not have to add an owner as a member to a team first as you do with the Add-
UnifiedGroupLinks cmdlet. The -Role parameter tells Add-TeamUser what type of member the user is.

[PS] C:\> Add-TeamUser -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -User "Don.Vickers@Office365itpros.com" -Role Member

[PS] C:\> Add-TeamUser -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -User Tony.Redmond@office365itpros.com -Role Owner

Adding individual members to a team soon becomes a pain when there are more than a couple of new members to
process. A simple loop through source data is a good way to add multiple users, as in this example which adds all the
members of an email distribution list to a team. The code needs to check whether users in the distribution group are
already members of the team, so we create a hash table holding the team membership and use that to check a user’s
status. Building the table is a one-time operation that consumes some resources but checking an item against a hash
table is much faster than an iterative loop through team membership. The key to the hash table is the display name of
the mailbox or guest; the value is their email address and includes both local and guests. If you do not need to include
guests, comment out the lines that create these entries. The table looks like this:

Name Value

---- -----

Robert Wille (MVP) rw@ outlook.de

Paul Cunningham (Mr Exchang... Paul.Cunningham@office365itpros.com

Stanislav Buldakov (MVP) sbuldakov@outlook.com

Mahmoud Magdy (MVP) mamagdy@outlook.com

We can use the email address of mailboxes retrieved from the distribution group to check against the hash table and
figure out whether a member of a source distribution group member is already part of the team.

[PS] C:\> $TargetGroupId = "8836e3ca-6531-402a-ac5f-b71c83d6affa"

$SourceDL = "GDPR Working Group"

#Create hash table to look up existing team members

$TeamMembers = @{}

Get-TeamUser -GroupId $TargetGroupId | % {

$User = $_.User.tostring()

# Handle guests

if ($User -like "*#EXT#*") {

$GuestUser = Get-AzureADUser -ObjectId $User

$TeamMembers.Add($GuestUser.DisplayName, $GuestUser.Mail) }

else {

# local mailbox

$Email = $_.User

$User = $_.Name

$TeamMembers.Add($User, $Email) }

$DLUsers = Get-DistributionGroupMember -Identity $SourceDL

ForEach ($U in $DLUsers) {

$CheckUserEmail = (Get-Mailbox -Identity $U.Name).PrimarySmtpAddress

If ( $TeamMembers.ContainsValue($CheckUserEmail) -eq $False)

 { Add-TeamUser -GroupId $TargetGroupId -User $CheckUserEmail -Role Member }

The Remove-TeamUser cmdlet removes a member from a team. You do not have to specify the user’s role.

[PS] C:\> Remove-TeamUser -GroupId 72ee570e-3dd8-41d2-bc84-7c9eb8024dd4 -User "Don.Vickers@Office365itpros.com"

A tenant administrator or an account holding the Office 365 User Management Administrator role can add a member
to any Office 365 group (and team) by fetching the group identifier with the Get-UnifiedGroup cmdlet and using it
with Add-TeamUser to update membership for the selected team. This works even if an Office 365 group is not team-
enabled. For example:

[PS] C:\> Add-TeamUser -GroupId (Get-UnifiedGroup -Identity "Stock Market Club").ExternalDirectoryObjectId -User
Donald.Vickers@office365itpros.com -Role Member

Because a team always has an underlying Office 365 group, you also can use the cmdlets that work against Office 365
groups to view or modify team membership. However, any changes made through cmdlets like Add-UnifiedGroupLinks
take extra time to be effective within Teams as the change must synchronize from Azure Active Directory to the Teams
directory, so overall it is best to use the Teams cmdlets to manipulate team membership with PowerShell.

Checking for Active Teams

Administrators often ask for a way to know what groups are enabled for Teams or the other Office 365 resources
available to groups. The Get-UnifiedGroup cmdlet returns the basic properties of groups, including the number of
members, the URL for SharePoint resources, and whether connectors are enabled, but it doesn’t tell you if the group
is team-enabled or uses Planner. The ProvisioningOption property returned for groups by the Get-UnifiedGroup
cmdlet tells us whether a group is created using an Exchange client (including PowerShell, because the New-
UnifiedGroup cmdlet is part of the Exchange Online module) or Yammer but gives inconsistent results for Teams. You
can figure out whether an Office 365 group has an associated team by comparing lists of teams and groups, but that is
an impossible task in most tenants. And anyway, the advent of a version of the Teams PowerShell module with an
updated Get-Team cmdlet that can return all the teams in a tenant makes messing around with provisioning options
unnecessary and means that you should always use Get-Team to find the team-enabled groups in a tenant. See the
description of Get-Team later in this chapter.

A group being team-enabled is one thing. Figuring out if the team is active is another. One approach to test for team
activity is to use the Get-MailboxFolderStatistics cmdlet to check the group mailbox to see if the compliance records
are in the Team Chat folder. As explained earlier in this chapter, the Team Chat folder is a sub-folder of the
Conversation History folder and holds compliance records generated for messages in all channel conversations within
a team. If the folder in a team’s group mailbox holds some items, it is a good sign that the team has been active.

While a simple check for a certain folder name works, we need to do more. First, the name of the Team Chat folder
varies depending on the language configured for the group mailbox, so we cannot simply check for a certain name. A
further complication is that the mailbox language can change without renaming the default folders. Fortunately, the
value assigned to the folder type is consistent, and we can check that it is “TeamChat,” so we can use that.

This script makes a collection of all teams in the tenant and runs Get-MailboxFolderStatistics for each team to check
its activity level. A report is generated with details of each team, including the last time something happened in a
channel and the number of conversations per day. It would be easy to extend the script to flag teams where no activity
has happened in the last 30 days or the number of conversations per day is under a certain number.

[PS] C:\> Write-Host "Fetching list of teams..."

$Teams = (Get-Team)

Write-Host "Starting to process" $Teams.Count "teams"

$Count = 0

$Report = @()

ForEach ($T in $Teams) {

$G = Get-UnifiedGroup -Identity $T.GroupId | Select Alias, ManagedBy, WhenCreated, GroupMemberCount

$Test = (Get-MailboxFolderStatistics -Identity $G.Alias -FolderScope ConversationHistory -IncludeOldestAndNewestItems)

$TimeSinceCreation = (Get-Date) - $Test.CreationTime[1]

If ($Test.ItemsInFolder[1] -eq 0) {

Write-Host "No Teams compliance records found for” $T.DisplayName

$ChatsPerDay = 0 }

ElseIf (($Test.FolderType[1] -eq "TeamChat") -and ($Test.ItemsInFolder[1] -gt 0)) {

 Write-Host $T.DisplayName "is active"

$Count++

$ChatsPerDay = $Test.ItemsInFolder[1]/$TimeSinceCreation.Days

$ChatsPerDay = [math]::round($ChatsPerday,2)}

$ReportLine = [PSCustomObject][Ordered]@{

Alias = $G.Alias

Name = $T.DisplayName

Owner = $G.ManagedBy

Members = $G.GroupMemberCount

WhenCreated = $G.WhenCreated

ChatCount = $Test.ItemsInFolder[1]

LastChat = $Test.NewestITemReceivedDate[1]

DaysOld = $TimeSinceCreation.Days

ChatsPerDay = $ChatsPerDay}

$Report += $ReportLine

Write-Host $Count "of" $Teams.Count "are active Teams"

$Report | Export-CSV c:\temp\TeamsReport.csv -NoTypeInformation

The output of the script is a CSV file, which can be opened with Excel to format the data to meet your requirements.
Several properties are included in the report that are extracted from the Office 365 group for a team. If you want to
include extra group properties, you can add them to the Select statement piped from the call to Get-UnifiedGroup and
include the properties in the output line generated for each group.

The Get-MailboxFolderStatistics cmdlet is expensive in terms of the system resources it consumes, so running it to
check hundreds of groups is impractical if you intend to create the report very regularly. The solution explored here is
imperfect and flawed, but it shows what is possible by combining Teams cmdlets with cmdlets from other Office 365
modules.

Using Cmdlets from Other Modules with Teams

As we’ve just seen, other cmdlets in the Exchange Online or Azure Active Directory modules are useful when dealing
with Teams. For instance, every Office 365 group has twenty custom attributes for tenant-specific use. Fifteen
(CustomAttribute1 through CustomAttribute15 ) support single-value properties and five (ExtensionCustomAttribute1
through ExtensionCustomAttribute5 ) support multi-value properties. You can populate these attributes with whatever
values you need. For instance, you might want to add a department name to each team to track how many teams each
department uses. In this example, we use the first single-value attribute to hold a department name.

[PS] C:\> Set-UnifiedGroup -Identity Team1 -CustomAttribute1 Marketing

Updating a multi-value attribute uses a different syntax:

[PS] C:\> Set-UnifiedGroup -Identity Team1 -ExtensionAttribute1 @{Add="IT"}

When the attributes are set, you can easily retrieve just the teams for a specific department or generate reports
sorted by department. For example, to process the teams that belong to the IT department, we can do something like
this to fetch a list of groups stamped with a departmental value and then extract the membership for each team.

[PS] C:\> $Teams = (Get-Recipient -RecipientTypeDetails GroupMailbox -Filter {ExtensionCustomAttribute1 -eq "IT"})

ForEach ($Team in $Teams) {

Write-Output $Team.DisplayName

Get-TeamUser -GroupId $Team.ExternalDirectoryObjectID

Write-Output "" }

Office 365 Connectors

Connectors allow you to link a feed from a cloud data source to an Outlook group or a Microsoft Team. The idea
behind a connector is that it creates a link to bring information from the source to the attention of group members,
who can consume the information through conversations in a group or as a chat within a team. In the case of Office
365 Groups, members who have subscribed to the group receive notifications via email. Connectors create their items
as conversations in the group mailbox. Each item is represented by a card containing a snapshot of information
fetched from the source. Just enough information is in the card to let the reader know what is happening. If they want
to read more, they can consult the original source, usually through a hyperlink included in the card, to access the full
content.

Connectors support an extensive set of network sources, including those featured on project activities (Trello, Asana,
and Wunderlist), customer relationships (Salesforce, Dynamics 365, and Zendesk), news (Bing News, Twitter, and RSS
feeds), and developer tools (GitHub and Visual Studio). In addition, an “Incoming Webhook” connector is available as
a generic link to allow developers to fetch data from other services to an Outlook group. Programmers can use the
webhook to create a link to a group for a company-specific system or some other network data source for which a
connector does not exist. Both Office 365 Groups and Microsoft Teams support connectors. The Connectors created
for Office 365 Groups store their cards as conversations while those linked to Teams creates cards as messages within
channels.

Actionable Messages
“ Actionable messages ” leverage a subset of the connectors available for groups and Teams. Many inbound messages
have a link to bring the user to a web site to conduct a transaction such as responding to an inbound customer
request or to decide how to process an item. The idea behind actionable messages is that you can go ahead and
execute whatever needs to happen from within the message because its content has the necessary commands. Clients
like Outlook 2016 and Teams can interpret the commands and make them available to the reader. In the same way as
a connector creates cards within a group, items from a network source flow from the source to a destination.
Actionable messages store their cards in the user’s inbox instead of a group mailbox. In both cases, cards show
content extracted from the network source. In the case of actionable messages, the user can interact with the content
because the cards can have buttons, dropdown lists, date pickers, and text input boxes.

At Build 2018, Microsoft announced Adaptive Cards as the next step forward from actionable messages. Adaptive
cards are available for Outlook, Teams, Cortana, and the Windows Timeline view. For more information, see the
Adaptive Cards web site or watch this Ignite 2018 session .

Adding a New Connector for an Office 365 Group

To create a new connector for a group, select “Connectors” from the Group settings menu (gear icon) in OWA. You
can then select the type of connector and give the information necessary to authenticate to the data source (usually by
giving a user name and password known to the data source) and fetch information. Some sources, like an RSS fee, do
not need authentication, and others, like a Twitter feed, need you to configure parameters to select items from the
feed. For example, Figure 14-2 shows a Twitter connector configured to capture tweets from selected individuals plus
any sent with the #Office365 hashtag. The connector uses a Twitter account to authenticate to the service and that
the options available for this connector allow cards created from digests (like a weekly collections of tweets) or
individual tweets.

You can configure a group with multiple connectors of the same type and multiple connectors for different sources to
gather information from multiple places into a single group. There are many interesting ways that you might consider
using connectors with Office 365 Groups. For instance, you might set up connectors to link to a corporate Twitter feed
or RSS feed so that group members automatically see updates about important corporate initiatives without having to
check out a blog post or use their own Twitter account. In addition, the group then serves as an index for tweets or
blog posts and because the items are in the group mailbox, they are discoverable and easily searched.

A key factor to understand is that the current form of connector fetches notifications and similar signals with the
intention that group members then react to the information that shows up in group conversations. For instance, you
might see some information about a new blog post that prompts you to access the blog to read the entire content.
Using connectors as a notification mechanism for group members is valuable because people then do not have to keep
on checking with various sites to see if new information is available, but the card-based items that turn up in
conversations are incomplete and therefore cannot be used for compliance or record-keeping purposes.

Take the example where you want to connect the corporate Twitter account to a public group that is available to
everyone in the company. People who access the group will see a note of every tweet sent from the corporate account,
but the content shown in the card for each tweet is incomplete. It might exclude some data and it misses context in
terms of any conversation that might have occurred in Twitter. Therefore, the connector is useful for letting members
know that tweets exist, but they must go to Twitter to get the full picture.
Figure 14-2: Configuring a Twitter feed for an Outlook group connector

Using the Incoming Webhook Connector

Connecting a Twitter or RSS feed to a group is a good use for a connector. Being able to connect your own data source
to a group (or to a channel in a team) is perhaps even more interesting. Microsoft makes this possible with the
Incoming Webhook connector, which receives incoming HTTP requests holding JSON-format payloads and routes the
information to a destination group or team. The “webhook address”, generated by Office 365 when the Incoming
Webhook connector is linked to a group or team, is used by programs to find the connector for future communications.
When the group receives inbound information, it uses the information to create a new card. If received by a team, the
team creates a new conversation in the target channel. All we need to do to connect a data source to a group or team
is:

1. Find the data source to use.

2. Decide on the destination – an Outlook group or a team.
3. Create an Incoming Webhook connector for the destination.
4. Extract data from the source and format the data as a JSON payload.
5. Post the data to the webhook address.

Any data source is acceptable. The important thing is to be able to extract relevant data from the source and format it
for submission to the connector. You can use most programming languages to extract, format, and send the data to the
connector. This example uses the Office 365 Service Communications module for PowerShell to extract details of the
last known service incident for the tenant and post details of the incident to the target group or team channel. Before
you can use the Service Communications module, you must install it from the PS Gallery onto your workstation:

[PS] C:\> Find-Module O365ServiceCommunications | Install-Module

Creating an Incoming Webhook Connector
To create the connector, select the group that you want to use and access it from OWA. Select Manage Connectors
from Group settings (gear icon), and then Incoming Webhook from the list of available connectors. Click Add to
start creating the connector and then add:

The name of the connection.

An image file if you do not want to use the default icon for the connector. OWA seems to ignore the custom image
while Outlook displays it.

Click Create when ready to go ahead. The connector is then created along with its webhook address, which is in the
form:

https://outlook.office365.com/webhook/f47702b9-d4a9-414a-9056-d6b0230ebfa9@b662313f-14fc-43a2-9a7a-
d2e27f4f3478/IncomingWebhook/6e57a36d0e654b0a8bffa6336b3d81a7/eff4cd58-1bb8-4899-94de-795f656b4a18

You can copy the webhook address to the clipboard using the option available in the form (Figure 14-3 ) so that it is
available for copying into the code that creates and sends the payload. If you forget to save the webhook address, you
can always retrieve it by selecting the connector and editing its properties. Click Done to complete the creation
process.

Figure 14-3: Creating a new Incoming Webhook connector

Creating payloads for an Incoming Webhook connector

We want to send information about service incidents to the group. To collect the information to insert into the payload,
we use PowerShell to fetch the incident data. This code grabs service incident data for the last two days.

[PS] C:\> Import-Module O365ServiceCommunications

$O365ServiceSession = New-SCSession -Credential $O365Cred

$Incidents = Get-SCEvent -EventTypes Incident -PastDays 2 -SCSession $O365ServiceSession | Sort StartTme -Descending | Select-Object
Id, Status, StartTime,

@{n='ServiceName'; e={$_.AffectedServiceHealthStatus.servicename}},

@{n='Message';e={$_.messages[0].messagetext}}

Creating the payload means that we take the data and format it properly. In doing so, it is important to remember the
following points:

It can take some time and practice to learn how to use PowerShell to create the JSON content. It is best to start
with a basic card and then gradually build up the full content for the cards you want to create.
Posting to the connector depends on proper formatting of the payload. You might be able to convert your payload
to JSON successfully using the ConvertTo-JSON cmdlet only to find that the Invoke-RestMethod cmdlet rejects
the payload when processing it for submission to the connector. All of which leads to hours of fun debugging
payloads.

This code takes details about the most recent service incident and creates a suitable payload in a PowerShell variable.
The different elements that we want to report are formed into name-value pairs.

[PS] C:\> $InfoBit = ConvertTo-Json -Depth 4 @{

Text = "Office 365 Service Update"

Title = "New Office 365 Service Information at " + (Get-Date -Format U)

Summary = $Incidents.ServiceName[ 0] + " incident"

sections = @(

@{

facts = @(

@{

name = "Incident Number"

value = $Incidents.ID[ 0]

},

@{

name = "Affected Service:"

value = $Incidents.ServiceName[ 0]

},

@{

name = "Current Status:"

value = $Incidents.Status[ 0]

},

@{

name = "Start date:"

value = Get-Date $Incidents.StartTime[ 0] -Format u

},

@{

name = "Information:"

value = $In cidents.Message[0]

)
}

Submitting the Payload

The Invoke-RestMethod cmdlet sends the payload to the connector, identified by the webhook address stored in the
$InfoFeed variable.

[PS] C:\> Invoke-RestMethod -ContentType "Application/Json" -Method Post -Body $InfoBit -Uri $InfoFeed

If everything goes well, the cmdlet will respond with 1 (one) to show that the post was successful. And of course, a
new card should appear in the target group or team channel (Figure 14-4 ). This is a relatively simple example of the
information that a programmer can include in a card. Other applications might want to include elements such as a
hyperlink to bring users to a location where they can get more detailed information about an item. It is also possible
to include graphics. For a full explanation about how to format the JSON payload for an Office 365 connector, see the
Office 365 Connectors API Reference .

Of course, this example is incomplete. It has no error checking (such as making sure that at least one service incident
exists), does not check whether a card already exists for an incident, nor does it include a delay before checking for
new incidents. Some additional code will easily address these deficiencies and create an operational-quality script .
The point made here is that it is very easy to use the Incoming Webhook connector to deliver data from a source to a
group or team.

Figure 14-4: Details of an Office 365 service incident posted to a team channel

Disabling Connectors
The wide range of available connectors make it easy to see how Office 365 Groups can serve as a one-stop fulcrum of
inbound information collected from different cloud sources. However, if you do not want to use connectors, you can
disable them for a specific group or for the entire tenant. To disable connectors for an individual group, run the Set-
UnifiedGroup cmdlet to set the ConnectorsEnabled property to $False . Disabling a connector for a group also
disables it for the team associated with the group, if a team exists for the group.
[PS] C:\> Set-UnifiedGroup –Identity "Office 365 for IT Pros" –ConnectorsEnabled:$False

If you want to disable connectors for a complete tenant, run the Set-OrganizationConfig cmdlet to set the
ConnectorsEnabled property to $False . This blocks connectors for both Groups and Teams.

[PS] C:\> Set-OrganizationConfig–ConnectorsEnabled:$False

Disabling connectors on a tenant-wide basis naturally takes precedent. You cannot block connectors for a tenant and
then selectively enable connectors for some groups. Unfortunately, apart from opening each group to see whether it
has any configure connectors, there is no quick way to scan all the groups within a tenant to find which groups have
connectors and to the sources for the connectors.

Plans
People make plans all the time. Office 365 offers several different ways to manage tasks, including a complete team-
oriented task management application called Planner. Let’s go and plan our way to happiness.
Chapter 15: Planner
Tony Redmond

Lightweight Planning
Microsoft Planner is a lightweight task-oriented planning application whose big advantage and unique selling point is the depth of its integration with other Office 365
workloads, predominantly achieved through a symbiotic relationship with Office 365 Groups.

One way of thinking about Planner is that it is Microsoft’s version of the popular and widely used Trello application. Trello has been around for a while and the two
applications share many visual similarities in layout and visual clues, albeit with slightly different naming conventions. For instance, Trello lists are Planner buckets. See
this page for a comparison between Planner and Trello. In terms of other competition, Planner needs to look no further than its big brother, Project Online, other
Microsoft offerings like SharePoint Online, and third-party offerings for Office 365 task management, like ActionSpace or Tasks In A Box . It might be in competition
with Excel because people often use spreadsheets for basic project and task management.

To Do Lists
Many of us do our very best to organize our own lives for the better. Writing up to-do lists and crossing off completed tasks from the list helps us to prioritize and get
things done. Almost every system designed to help people with office-oriented tasks has included some form of to-do list alongside email and a calendar. The “Personal
Information Manager” (PIM) segment was very popular in the early days of PCs and persists to this day. Every version of Outlook since its introduction include a Tasks
subsystem to allow users to note and track things that they must do. Indeed, if you select Tasks from the Office 365 Portal, you go to Outlook Tasks.

Microsoft bought the popular Wunderlist application in 2015. However, instead of releasing Wunderlist to Office 365, Microsoft built a new app on top of the Office 365
infrastructure, called Microsoft To-Do, which reached general availability in October 2017. Microsoft describes To-Do as a “simple and intelligent to-do list app that
empowers users to keep track of and focus on the things they need to get done.”

To-Do is supported for both Office 365 and Microsoft consumer accounts. The To-Do mobile (for iOS or Android) and browser clients use the Outlook Tasks API to store
task info in the Tasks folder of either an Outlook.com or Exchange Online mailbox. If users create a new list with the app, the to-do items for that list go into a sub-folder
under Tasks named for the list. Tasks created using Outlook.com or Exchange Online synchronize with the To-Do service. By default, Microsoft enables To-Do for users
with eligible Office 365 subscriptions . You can control access To-Do within the tenant through the Services & Add-ins section of the Office 365 Admin Center.
Alternatively, you can control access for individual users by editing the options bundled in the Office 365 license assigned to the user to remove or allow access to the
application.

Planner and Project

Project is Microsoft’s headline project and task management application. Originally launched for DOS in 1984, the capabilities of the Project application have expanded
dramatically since. The latest iteration is composed of Project clients and servers, the latter being available in on-premises and cloud versions. Project is a powerful and
sophisticated time and task management application that is much beloved by professional project managers. It can handle the complexities involved in very large
projects such as the design and construction of major infrastructure projects in most industries. Like many highly functional software packages, learning to use Project
can be overwhelming and it can take a long time to become proficient in its use. Those who have mastered Project will probably consider Planner’s approach to be
simplistic and deficient in terms of charting, dependencies, and other areas. However, it is worth underlining again that Microsoft does not intend Planner to take on the
kind of complex, heavy-duty planning requirements that Project takes in its stride. In short, Project is the right tool to use when the need arises to coordinate multiple
milestones, dependencies, and complicated schedules. If needed, you can connect tasks managed in Project Online with Planner . However, you cannot transfer tasks
from Planner to Project or convert a plan into a project or vice versa.

An obvious gap exists between the intensely personal nature of something like Outlook Tasks or To Do and the powerful but complex capabilities available in Project.
This is the middle ground that Planner tries to cover (Table 15-1 ). Designed to be easy to use and need both a minimum of administrator involvement and user training,
Planner supports teams that come together to work on common tasks (or goals). A team leader can use Outlook Tasks to manage the work assigned to the different team
members, but no one else in the team would be able to see or contribute to the planning activity. You could use Project for the job, but it might be like using a
sledgehammer to crack an egg. In short, Planner lends itself to the coordination of work done by ad-hoc or persistent teams that have a strong focus on sharing and
collaboration rather than structure and rigorous project discipline. Although no connection exists today between individual tasks created in Outlook and an item entered
in a plan managed by Project, it is conceivable that Microsoft might create links between the products in their task and project management portfolio in the future.

Individual Team Company

Outlook Tasks Office 365 Planner Project Server

Microsoft To-Do SharePoint Team Sites Project Online

Table 15-1: The continuum of task management from to-do items to projects

Planner, Office 365 Groups, and Applications

Planner has a dependency on Office 365 Groups. Every Office 365 group can have a corresponding plan and a plan belongs to that group. Originally, a group could only
have a single plan – the default plan, but this situation changed when Teams introduced the ability to have multiple plans. A team can have multiple channels, all of
which use an Office 365 group to share a common team membership. However, each channel inside a team can have multiple plans, all of which belong to the team.

Group-enabled SharePoint team sites can also create multiple plans, all of which are associated with the team site. You can add these plans (or the default plan for the
group) as links accessible from the home page of the team site or embed one or more plans in site pages using the Planner web part, deciding whether the web part
displays boards or charts. The Planner hub does not display the name of the underlying team site that a plan belongs to, so it is wise to include the name of the site when
naming a plan.

Some organizations might be using SharePoint Online team sites (classic sites, rather than the group-enabled variety) to coordinate tasks, especially if they are coming
from an on-premises environment where classic team sites are the only choice. Over time, as these sites move to become group-enabled, they can consider moving to
Planner instead to take advantage of the development effort Microsoft puts into Planner.

Ideally, a user needs to have an Exchange Online mailbox to enjoy the full functionality available in Planner, particularly the close connection with the underlying Office
365 Group. If a hybrid user has an on-premises mailbox, they are unable to add a comment to a task using OWA or Outlook. Marking plans as favorites is also a feature
that does not work properly when a user does not have a cloud mailbox. However, a hybrid user whose mailbox is on-premises can post comments to tasks through the
Planner application.

Planner Services
Planner uses Azure Data Services to host its data in table storage/blob storage, encrypted using Azure Storage Service Encryption. The infrastructure used by Planner is
Autopilot, a datacenter infrastructure stack and service management layer used by Microsoft for its datacenter operations. The implementation supporting the Planner
services is called “Pilotfish.” Microsoft has deployed the service on an Autopilot cluster inside every Office 365 datacenter region, co-located alongside Azure. Audits of
Pilotfish confirm its compliance with the Service Organization Controls Report SOC 1 , SOC-2, SOC-3, and ISO 27001 standards.

Planner Clients and Integrations

Planner stores the metadata for plans, including information describing the tasks and buckets that make up each plan, in an Azure data service. The links to an Office
365 Group for membership, conversations, and file storage, are part of the plan metadata. Programmatic access to Planner data is available through the Microsoft
Graph.

Like many other Office 365 applications, Planner’s first client was browser-based, followed by a mobile app (for iOS and Android ). No desktop client currently exists for
Planner, but Microsoft has an iCalendar connector to link tasks assigned to a user to their Outlook calendar (see the section later in this chapter). In the meantime, a
third-party add-in is available in the Office Store from iGlobe that makes Planner appear as an Outlook resource (just like public folders). You can navigate between
tasks and view their details. Another tool from the same company gives a reporting capability for Planner.

Planning Basics
You can access the Planner application with your normal Office 365 credentials by:

Clicking the Planner icon in the Office 365 app launcher.

Navigating to the Planner Hub at tasks.office.com .
Selecting Planner from the menu bar of an Office 365 Group when working with group conversations in OWA.

It is not possible to access Planner through the Outlook 2016 integration with Office 365 Groups. You can access Planner from Groups through OWA or Yammer and a
Planner mobile app is also available.

The “All Plans” section in the Planner Hub list all the Office 365 Groups to which the user belongs, even if these groups have no plan information associated with them.
Planner is a web application that uses a streamlined interface composed of a navigation pane to the left and a details pane to the right. The user interface accommodates
a range of screen sizes and form factors and resizes elements to fit the available space. Figure 15-1 shows how the Planner Hub displays Office 365 Groups as Plans in
the navigation pane. Office 365 Groups that you mark as favorites show up as a favorite plan. Likewise, if you select another plan as a favorite, the groups in the
favorites section of Outlook 2016 and OWA will include the associated group. The “My tasks” link displays tasks assigned to the user from all plans.

To work with a plan, you click the name of an existing group or create a new plan. The writing team for this book uses an Office 365 group to coordinate the book
components and writing tasks since we started the project, so it seemed natural to use Planner as a more structured way of outlining and tracking the many tasks that
go into the production of the book alongside the group library that holds the source documents and the shared OneNote notebook that records the “to-do” list and other
information about the book. As it happens, creating the plan involved creating tasks based on the existing to-do list.

Figure 15-1: The Planner Hub

Creating New Plans

To create a new plan, click New plan . Creating a new plan through the browser interface also creates a new Office 365 group. However, if a plan is created in a group-
enabled application (Teams or SharePoint) that supports multiple plans, no new group is created because one already exists.

If a group creation policy is in force, Planner “trims” its user interface to remove the choice to create a new plan when the user is not allowed to create new groups (see
Chapter 12 for more information). Tenant administrators can always create new Office 365 Groups, but Planner removes the choice to create a new plan if a tenant
administrator is not a member of the group allowed by policy to create new groups. The workaround is to either include all tenant administrators in the group defined in
the policy or to create a new group-through another client (for example, OWA, Outlook, or PowerShell) and then access the group through Planner.

Office 365 Groups created by Planner are the same as any other group and are fully-provisioned with a SharePoint team site and other resources. Figure 15-2 shows the
interface to collect information about a new plan. If a group naming policy is in force, the names given to new plans will be adjusted. For example, if the policy settings
mandate that new groups get a prefix of “O365GRP-,” the display name for the group created for the plan shown in Figure 15-2 is “O365Grp-GDPR Planning.” The
exception is for plans created by tenant administrators, who do not come under the control of the naming policy.

After creating a new plan, the next thing to do is to add some members. Click Members in the menu bar and add tenant users and guest users to the membership.
Remember that these people are now in the group membership and therefore have full access to the other resources belonging to the group.
Figure 15-2: Creating a new plan

Tasks and Buckets

Tasks form the basic building block for a plan. Each task describes a piece of work needed for the plan along with other information such as the person assigned the
work. Following the user interface design used in other Office 365 applications such as Delve, cards display information about tasks. You can customize the cards to
display different information about the tasks such as a graphic, expanded description, or even a checklist item.

Planner organizes tasks into “buckets” and arranges the buckets on a “board” (the analogy is to pin post-it slips with details of tasks to a board). Each plan starts off
with a “To Do” bucket. You can rename this bucket and create as many others as you need. Think of a bucket as a collection of tasks that constitute a major section of a
plan and you can even use buckets to coordinate several plans within a single overarching plan. In this scenario, each plan, or sub-plan, becomes a bucket. If you remove
a bucket from a plan, you also remove all of the tasks in the bucket.

Tasks belong to an individual plan and cannot move from one plan to another (yet – this feature might come). If you need to have a task in a plan, you must create it
there. You can group tasks for display in five ways:

Buckets : This grouping allows you to view the tasks belonging to major parts of the plan. Some people use buckets to group tasks by importance, some to divide
up a project into logical pieces of work, and some to show task status by having buckets with tasks that are completed, on-hold, not started, or whatever other
status you would like to use.
Assigned to (person) : This grouping displays the tasks assigned to each person. By default, the view only shows tasks that are in progress or not started. You can
add the completed tasks to the list by scrolling to the bottom of the list and clicking “Show Completed.” Figure 15-3 shows an example of this view.
Progress : This view shows tasks in the three stages supported by Planner – Not Started, In Progress, and Completed.
Labels : This view groups tasks into the set of six labels that you can assign to tasks. The names given to the labels are arbitrary and can vary from plan to plan.
Due date : Tasks not started or in progress are arranged according to the date set for their completion as:

Late.
Today.
This week.
Next week.
Future.
No date.

Figure 15-3: Tasks for a plan viewed by progress

Planner starts off by listing tasks in an arbitrary order and there is no way to select a preferred sort order such as by due date. You cannot assign a certain priority to a
task to increase its important over another, but you can move tasks up and down within the grouping to create whatever priority order you feel is best. You can also use
the six colored labels that are available for tasks to convey their relative importance. Once set, lists keep the order of tasks. You can also drag and drop tasks between
buckets to make sure that they show up in the right place. A nice touch in the UI is that when you click the icon for a group member (listed in the top right-hand corner),
Planner highlights the tasks assigned to that person.

You can see that buckets are primary elements of the Planner user interface. For this reason, unless you use a screen that is very wide, it is wise not to have more than
five or six buckets per plan. Some people have run plans with 25 buckets, but the user interface becomes cluttered at this point and it can be difficult to find an
individual task. If you think you will need to use more than six buckets to organize tasks, perhaps it is best to split them across multiple plans.


Arranging buckets : You can display the tasks for a plan in different groupings. If you use buckets to arrange tasks into segments of a plan, you can group the tasks
by bucket and then drag and drop the buckets into whatever order makes sense to create an overall view of the plan. You cannot reorder the other grouping (by
progress or by assignee) in this manner.

Changing Plan Settings

The options available in the ellipsis menu […] in the menu bar for a plan are:

Links to other group resources. These choices open new browser tabs.

Conversations : Opens OWA to view email conversations in the group mailbox.

Members : Opens OWA to view group membership. Group owners can add, modify, or remove group members here.
Files : Opens the document library in the SharePoint team site belonging to the group.
Notebook : Opens the shared OneNote notebook belonging to the group.
Sites : Opens the home page of the SharePoint team site belonging to the group.

Open in Microsoft Teams opens the Teams desktop client (if available) in the General channel of the team belonging to the group. This choice only appears for
groups that are team-enabled.
Remove from favorites : This choice appears if the plan is in the list of favorites. Otherwise, you will see “Add to favorites.” If you mark a plan as a favorite, the
group also appears in the list of favorite groups in OWA or Outlook.
Copy link to plan : Copies a URL for the plan to the Windows clipboard. The link is in the form shown below. When pasted into a browser, it opens the plan:

https://tasks.office.com/redmondassociates.org/en-US/Home/PlanViews/knwuDbAxtE2A7iYVAEJ-vpYAFE4L

Add plan to Outlook calendar : Allows plan owners to publish plan data as an iCalendar feed. Anyone inside or outside the organization can add the iCalendar
feed to their calendar to add information about tasks in the plan.

Plan Settings : This choice allows plan owners to:

Change some of the properties of the underlying Office 365 Group rather than anything else associated with the plan with the Edit plan link. For instance, you can
choose to make the plan public or private.
Change how notifications are sent for the plan (Figure 15-4 ). Plan owners control whether Planner sends email notifications to users when they are assigned
tasks or when someone marks a task as complete. Individual users can choose what other notifications to receive by checking Follow this plan’s group in your
inbox . If you choose to follow a plan, you can receive notifications for everything that happens in a plan. The messages received by plan members include a link to
open Planner and interact with the relevant task.
While the notification setting chosen in Plan settings controls email notifications received by a user for a specific plan, the Notifications setting on the Planner
cogwheel menu apply for all plans. Here a user can choose whether to receive notifications when someone else assigns them a task or when a task is late, due
today, or due within the next week.
Remove the group with Delete this plan (a Delete group link also appears under Edit plan ). Be careful. If you remove a plan, you do not just remove its tasks
and other metadata., Instead, removing the plan also forces Office 365 to remove the underlying Office 365 Group and all its resources, including the SharePoint
team site and group mailbox. If you remove a plan accidentally, you can recover it and its group using the procedure described in Chapter 12.

Stop following plan in Inbox/Follow plan in Inbox : Allows the user to control whether they receive email updates for task assignments and completions. When
a member chooses to stop following the plan, they leave the subscribers list for the group. This means that they also stop receiving copies of other conversations in
the group mailbox. Conversely, if they follow a plan, they become a member of the subscriber list for the group.

Figure 15-4: Plan settings

Plans can span hundreds of tasks, and you can assign all those tasks if you like to a single user (or leave them unassigned). The largest plan I have used included over
500 tasks (you can assign up to 300 tasks to an individual user within a single plan). Obviously, a user can have multiples of this in terms of open tasks spread across
different plans.

It is possible that you make a mess when setting up a plan and want to start over. A complete bucket can be selected by clicking its name and then Delete Bucket from
the ellipsis menu. If you want to remove the complete plan, go to the ellipsis menu for the plan, select Edit Plan and then Delete Plan . As noted above, deleting a plan
also deletes the associated Office 365 group and all its resources.

Private or Public
A plan can be private, in which case only group members have access to its content, or public, where any user in the tenant can join the plan. The access type for a plan
depends on the privacy setting for the underlying Office 365 Group. If you change the plan’s access type, you change the access type for the group and its team (if
enabled). To change the privacy setting, edit the properties of the plan or run the Set-UnifiedGroup cmdlet. For example, to set a group (and its plan) to be private:

[PS] C:\> Set-UnifiedGroup -Identity "My Big Plan" -AccessType Private

Because access to a plan depends on its group, changing a plan’s access type might have some unexpected consequences. In the case of changing a plan from private to
public, the intention might be to reveal details of a plan to invite comments from outside the project team on the set of tasks defined for the project. This is laudable, but
perhaps the project team does not want to expose the other shared group resources (documents, conversations, calendar, and notebook) to all and sundry. Some thought
is therefore necessary to figure out whether a plan should be public or private when it is created or if you edit a private plan to change its status to public.
Closing a Plan
In the current implementation of Planner, the only way to close a plan is to abandon it. There is no way to archive tasks from a plan to start afresh and start a new
planning cycle as you might do at the start of a new fiscal year. You must clear out old tasks manually by removing before you create any new tasks. Alternatively, you
could create a new bucket called “Archive” and move all the completed tasks to it before you start work on the next stage of a plan.

You cannot “reset” a plan by moving all completed tasks into an accessible archive where they can stay for records purposes for a set period. Even better, it would be
good if administrators could assign a policy to plans to govern their lifecycle and conduct automatic archiving.

Of course, archival is not just a matter of simply moving tasks to some other repository because tasks can have associated documents and comments stored as
conversations in the Office 365 Group. Removing this data without thought could have unforeseen effect on the underlying group. It is reasonably easy to copy the
documents held in the group library but more problematic to copy the conversations (comments) held in the group mailbox. Even if you write some code to extract the
necessary information from the group mailbox, the challenge still exists to link everything together in a way that you can reliably rebuild a plan if needed.

Archiving inevitably brings compliance to mind. AS discussed in several chapters in this book, Office 365 includes a comprehensive suite of data governance technology
designed to work across the service. Although some notable gaps still exist, Office 365 Groups and Teams now support the data governance framework, but Planner is
an outlier. Office 365 indexes the documents and comments stored in Office 365 Groups to make sure that they are discoverable, but Office 365 should also be able to
index and search the tasks and the metadata for plans. Finally, although group creation and updates (like adding a new member to a plan) generate Office 365 audit
records, plan actions such as creating a new bucket or task creation, update, assignment, or completion are not.

Charts
The Charts view (Figure 15-5 ) presents another way of looking at the tasks in a plan with the screen divided into a chart view of the complete plan and a list of selected
tasks to the right-hand side of the screen. In this case, tasks are listed by bucket, but you can apply any of the filters available in Planner (see later section) to display
matching tasks.

With just three charts available (status, bucket, and members), Planner’s ability to present graphic views of tasks is limited. A simple embellishment would be to allow
graphs to chart information for one or more selected buckets rather than a plan. Another obvious enhancement is the addition of a timeline view as people often
measure the progress of plans by date. It would be very convenient to be able to drag and drop tasks along a timeline and have their due dates adjusted to match.
Microsoft has committed to consider a timeline view in the future but have said that they are unlikely to add more complex or project-oriented charts such as Gantt or
burndown charts to Planner as these are more suitable for a product like Project.

Figure 15-5: Tasks shown through the Charts view

Schedule View
Schedule View (Figure 15-6 ) presents open tasks in a calendar, which is a natural way for many people to think about the priority order for their work. The view is like
OWA’s calendar and lists all open tasks in the plan, unless the view is filtered (see below). You can add new tasks into the calendar or drag and drop existing tasks to
assign them new due dates. Clicking a task reveals its full details. Tasks that do not yet have an assigned due date are shown in the right-hand pane. You can drag and
drop these tasks into the calendar to fit them into the right order within the plan.

Figure 15-6: Planner’s schedule view

Task Filters

Figure 15-7: Applying a filter to tasks

Because plans can span hundreds of tasks, Planner supports filters for the Board and Schedule views to allow users to focus on different collections of tasks. Apart from
being able to group tasks in buckets, you can filter by:

Due date : Select Late, Today, This week, Next week, Future, or No date.
Label : Select one of the six label names used for tasks within the plan.
Assignment : Select a task member to see the tasks assigned to them.

As shown in Figure 15-7 , you can combine one or more values for the different filters to highlight specific tasks. For example, you could opt to see tasks due this week
and next week for two group members that have a label called “Important.” When a filter applies, Planner only displays tasks matching the filter.

Creating and Managing Tasks

Creating a new task is very straightforward. Click the plus sign and enter details of the task such as the bucket it belongs to, the due date, and the person to whom you
assign the task. You can also select an existing task and copy elements of the task (such as the description, checklist, attachments, and labels) to create a new task. To
copy a task, click the ellipsis […] menu when editing the task that you want to copy, select the elements to copy, and give the copied task a new name.

Although each task must have a name, the names do not have to be unique. You can create as many tasks as you want with the same name and assign them all to the
same person, which might be a little confusing. Task names can be up to 254 characters long. If you add more, Planner will trim the name back to the limit. Special
characters are fully supported and a task name such as “1"£$%^&*()_+{}[}:;@'~#><,.?/|\ ” is possible, even if it doesn’t mean very much to anyone. On the upside,
plenty of opportunity exists to dream up some interesting code names for tasks.

Task Assignment
Planner originally limited task assignment to a single person but now you can assign a task to more than one person, including a mixture of group members and people
outside the group. Figure 15-8 shows that we have assigned three group members to a newly created task. The membership of the group populates the drop-down list,
which divides into those assigned to the task and those who are not. If you prefer, you can assign a task to someone by dragging their picture from the set of members
displayed at the top of the group and dropping it onto the task.

To assign someone from outside the group to a task, enter their name or email address in the text box shown in Figure 15-8 . If you enter the name of someone outside
the group, Plans add that person to the group as a new member. This is logical but remember that the new member then gains access to all the resources available to
the group, including all the items in the group document library. In other words, you cannot add someone to a plan without also granting them group membership,
something that might or might not be desired. And because Office 365 Groups allow the same level of access to all members, there’s no ability to allow someone to have
view-only access to a plan.

When you assign a task to a group member, Planner sends a notification to that user to tell them that they the assignment exists. Planner also starts a conversation for
the task in the group mailbox. You can control if Planner sends notifications for new-created and completed tasks by editing the plan properties (click the […] menu and
select Plan settings ). Comments added to the task become part of this conversation. Office 365 adds the comments to the conversation in the group mailbox and
circulates copies of the comments to anyone in the group who has previously commented on the task.

If needed, you can leave tasks unassigned until the right person is available to take on a task. As already noted, if someone leaves a group, their tasks stay assigned to
them until someone reassigns the tasks to another team member. No bulk reassignment function is available to move a set of tasks to someone else within one plan or
across all plans in a tenant, such as when someone leaves the company and the need arises for others to take over this work. For this reason, it is sensible to check
whether an individual has any assigned tasks when they leave the company.
Figure 15-8: Assigning a task to group members

Office 365 Groups and Planner support external access by guest users, which means that these users appear in membership lists and you can assign them tasks. In fact,
a group owner can add any account from the tenant directory, including guest users, to the group membership by assigning them to a task. There’s no doubt that
Planner makes it easy for group owners to populate group membership in this way, but the owner is not warned that they have just added a guest to the plan and its
underlying group, which means that they might not realize that the guest now has access to all the group resources.

Like other Office 365 applications that use Office 365 Groups to manage their membership, everyone in a group shares equal access to the plan. Anyone can create or
edit a task. Anyone can assign a task to someone else in the group or reassign a task that they have received to someone else. And they can remove a task or update the
progress of a task. It’s all very self-liberating and empowering if you’re prepared for this mode of working. If you need more hierarchical form of management where
only defined members can add, change, or assign tasks, then you need to use traditional project management software such as Project Online. Another option is to use
the “Project” template for SharePoint Online to create a site to hold tasks for a project. Out of the box, this will not deliver the same kind of user interface that Planner
provides, nor will it have the same close connection to an Office 365 Group, but because it is based on SharePoint Online, it is possible to customize and organize the
site to meet your requirements.

Building Out Tasks

The detail for newly created tasks is sparse with just enough information gathered to be able to populate the card. This might be enough for some plans but not for
others. To build out a task, select and click it to reveal the task dialog box (Figure 15-9 ), which allows you to complete the task metadata.

Figure 15-9: Details of a task

You can add the following information to a task:

Start and end date.

Task status (in progress, not started, complete).
The assignee.
 A free-form text description of the task. The description can be up to 800 characters long.
Attachments (files or web links).
Checklist items.
Six colored tabs or labels to mark the task in whatever way works best for the plan. You can use these labels to filter or group tasks.

You cannot change the metadata available for tasks. In other words, you cannot create another piece of information for users to complete to describe or categorize a
task.

The task name is the default information shown in the card, but if you prefer, you can use other elements as the preview displayed in task lists. For instance, it can be
easier to pick out something with a graphic image, so Planner allows you to use graphic files added as attachments to tasks as the preview.

Checklist items (four of which are in Figure 15-9 ) allow task owners to note activities and things to do for a task. In one way, you could consider checklist items to be
sub-tasks. However, checklist items have no formal connection to the outcome of a task because you can mark a task as complete even if many checklist items are
unfinished. The idea here is that a checklist item really is not a formal activity that must be done for a task to finish. Instead, it is a kind of free-form reminder of
something that should happen, but a task can complete without everything being checked off. Unlike comments, checklist items are stored along with the rest of the
plan metadata and are not circulated to group members.

You can add the following attachments to a task:

A file from your computer. This includes files stored in any network location available to the PC, such as the OneDrive and SharePoint sites synchronized to the PC.
A file from the SharePoint document library associated with the group that owns the plan.
A URL.

Planner stores files uploaded as attachments for a task in the Documents folder of the group document library (you cannot select a different folder). You cannot upload
the same file twice as it would cause a duplicate file in the folder, but once you upload a file to the library, you can attach it to multiple tasks. The task shown in Figure
15-9 has one attachment, a URL (link) to a web site. If you try to create a link attachment that points to a link already attached to the task, Planner overwrites the
original link. You can attach up to nine files or links to a task.

In the task shown in Figure 15-9 , one of the attachments is a JPEG graphic file. To make this graphic the preview, it is a matter of clicking the ellipsis menu and
selecting “Set as preview.” Cards for tasks that have graphic previews (Figure 15-10 ) take up more screen real estate than text-only cards, but sometimes you want to
highlight a specific item and a graphic can be an effective way of doing this. Checklist items or the free-form text description of the task can also be the preview. You can
add up to 20 checklist items to a task.

The different elements of a task show up in the user interface for the task cards too. The visual hints that you can add to cards give a quick insight into the tasks within a
plan, bucket, or assigned to a specific user. Obviously, there is a big difference between a card that uses a simple line of text to describe a task when compared to a card
that uses a graphic file for the preview plus some populated tabs or labels. What you add to a card in terms of comments, attachments, and checklist items help plan
members understand the full context of a task, including the task owner and their progress towards completion. A lot of information about a task is therefore available at
a glance.

You cannot customize the items available for a task by adding new fields for users to complete or removing components like the checklist. Unlike Project, Planner tasks
have no dependencies on each other. You cannot link one task to another in any way except by making sure that they are arranged in display order so that one follows
another as needed. This situation is problematic for some, but it reflects the general “let’s make it simple” design philosophy followed throughout Planner.

Figure 15-10: Using graphics to highlight tasks

Task Status and Tabs

Planner allows a task to be in just three states – not started, in progress, and completed. There is none of the precision and exactitude of measuring a task to be say 87%
complete. On the other hand, the simplicity is appealing as it is easy to know whether a task has started, is being worked on, or has finished.

You can also assign tasks with one or more of the six colored labels available in a plan. Labels are specific to a plan and you cannot change the colors used or lock the
names given to the labels. Because you can use labels to group or filter tasks, labels are a useful way to flag high-priority tasks or those that need to be flagged because
they are experiencing some problems or need specific attention from someone. As an example of how to use labels, the plan used to track progress for this book has
labels named after different areas of major functionality (Figure 15-11 ).
Figure 15-11: Adding labels to a task

You can assign whatever name you want to the labels and the names are the same for all tasks in a plan. However, because everyone in a group shares equal access to a
plan, anyone can decide that they want to change a label and update it, which then affects all the tasks that have the renamed label. This aspect of the user interface
encourages freedom of operation but seems a little odd in the context of an application designed to help people organize work. For instance, if I assign “Important” to be
the red label, I am not sure that I want someone else to rename it to “Can be ignored” without any hindrance.

Managing tasks is a matter of updating their status as the work unfolds. Tasks are completed, and their status can be set as such. Some tasks need to have their dates
adjusted to reflect changing circumstances, new tasks are added and perhaps some are removed. Eventually, everything will come together and all the tasks on a board
will be complete. And then, when all the tasks on all the boards are complete, the plan is complete.

Comments and Conversations

While the work is ongoing to complete tasks, it is likely that some communication between the group members is needed and that’s where conversations come in. Or
rather, “comments” that group members make about tasks, for this is how Planner refers to them.

As tasks are created and assigned to people, conversation items are created in the group mailbox. People who choose to follow the group Inbox receive these items in
their mailbox, meaning that when you create a new plan for an existing group that has a large set of existing subscribers, a certain amount of message traffic is likely to
be generated as tasks are created, assigned, and perhaps reassigned. The email notifications help group members keep track when they and others have been assigned
items as well as knowing when tasks are complete without having to go to the group mailbox to see what has been happening with the plan.

Figure 15-12 shows that a task has five comments and a new comment is being prepared. The editor used to compose text for comments is rudimentary and does not
support common text formatting shortcuts such as CTRL+B to bold selected words. If you want to emphasize text in comments, you will have to use another client.

Figure 15-12: Comments for a task

Except for the comment about the new task being created, the comments are posted to the group mailbox to form a threaded conversation for the task. Unlike task
notifications, group members don’t receive copies of comments in email unless they have commented on a task. Members who receive comments in email can either
reply directly to the message, or they can use the link embedded in the message to open Planner and go to the task and add their comment there. Posts added to the
conversation through email appear as comments for the task. And, as you’d expect, if you delete a conversation, all the comments for the associated task disappear.

The differentiation between comments and checklist items is that a comment is shared with all group members, even if they never go near the plan (because they
receive comments in email). Checklist items are visible to anyone who looks at a task and are intended to be a to-do list for the task owner. Checklist items cannot be
shared between tasks.

Behind the scenes, Planner uses the REST-based API for Groups to post comments to the group mailbox that is associated with the plan. When a user contributes to a
group conversation, the item is submitted through the Exchange Online transport system to apply transport rules before content shows up in conversations. For
example, a rule might block someone from posting sensitive data such as credit card information. Another might stop the posting of any text holding an offensive term.

Printing and Exporting Planner Data

Planner includes no method to print information about a plan such as a list of tasks assigned to a user or a schedule view of tasks due in the coming week. This is a
much-requested feature that is somewhat handled by the ability to send tasks to Outlook (see below), from where users can print the information as needed. Planner
also cannot export data in Excel or other formats.

Synchronizing Tasks to Outlook Calendars

Outlook calendar synchronization is automatically enabled for all Office 365 tenants that have Planner as part of their subscription (to disable the feature, follow the
instructions in this article ). When a user wants to connect Planner to Outlook, they select My Tasks in the navigation pane and click the ellipsis menu to reveal the
choice to Add “My Tasks” to Outlook calendar . Click the button and then select Publish . Planner generates an iCalendar link to the All Tasks view. The link looks
something like this:

https://tasks.office.com/b662313f-14fc-43a2-9a7a-d2e27f4f3478/Calendar/User/Qi6exSZyQkC737py0An6HpYAAZ1X?t=0_95716443-a6d2-425b-a936-
27fc76a889be_2018-05-10T12%3a29%3a13.4330492%2b00%3a00
Now click Add to Outlook . Planner launches OWA at the Calendar subscription window, copies the iCalendar link from Planner and names the calendar “Planner-My
tasks.” Click Save to continue. OWA creates a new calendar folder in the user’s mailbox and populates it with details of their “Not Started” and “In Progress” tasks
fetched from Planner. You can’t filter the tasks as the connector is configured to fetch all open tasks assigned to the user.

Synchronization is one-way from Planner to Outlook. New items do not synchronize at once because Outlook refreshes Planner data via the connector every three to four
hours. You can’t force synchronization to happen. The information synchronized to the calendar for a task includes:

Date: Planner items are scheduled for all day calendar slots as Planner bases task assignment on days rather than hours. If a task has a due date, the calendar item
is scheduled for that day. If it has both start and due dates, the item is scheduled for that period.
Location: None added as Planner does not capture this data.
Progress: The status of the task in Planner.
Checklist: a note about how many checklist items exist and are complete.

Calendar entries created through the connector do not tell you what plan or bucket within a plan a task belongs to, but each entry has a link to Planner to bring the user
back to the original task, where they can make whatever changes are necessary and have those changes synchronized back to Outlook later. Although the user cannot
edit the calendar entry, they can add a reminder.

If you prefer not to use the iCalendar connector, you can also link Outlook to Planner using several standard Flow templates published by Microsoft. Flow is good at
inserting items into a calendar. It is less successful at tracking changes made to tasks such as completing tasks or synchronizing changes made to a task like changing
its name, due date, or status. This is probably because the set of triggers supported by Planner for the connector available to Flow only covers task creation, assignment,
and completion, and not task updates.

Planner Guests
Planner supports the Azure B2B Collaboration model for guest access. As explained in Chapter 12, to join an Office 365 group, external users receive and redeem an
invitation. During the redemption process, if the external user doesn’t already have a guest account, one is created for them in the tenant directory. The same
membership as used for Office 365 Groups and Teams controls access to Planner, and a guest can be added to a group through Groups, Teams, or Planner . Once a guest
user is a group member, their membership automatically allows access to the plans associated with the group, including any plans created in channels belonging to a
team.

To access Planner, guest users must specify the service domain of the tenant to which they want to connect. Planner doesn’t have the ability to switch guests between
tenants, so the connection is always to a specific tenant. For example, to access plans in the office365itpros.com tenant, a guest connects to the URL:

https://tasks.microsoft.com/office365itpros.onmicrosoft.com

When Planner processes this connection, it asks the guest to authenticate. After a successful authentication, the Planner browser client displays the set of plans
available to the guest. These plans include those created before Planner supported external access.

Planner presents a tailored user interface to guest users by removing choices they cannot use (Figure 15-13 . Although guests can create and edit tasks, change task
status, and add checklist items, they cannot add comments to tasks because they do not have a mailbox in the tenant. As you might expect, guests cannot create new
plans because they cannot create new Office 365 Groups. Likewise, they cannot remove a plan. Guests can edit a plan’s name, but they cannot change other plan
settings. Finally, guests cannot browse the tenant to look for new plans (and groups) to join.

Figure 15-13: The Planner interface for a guest user

As noted earlier, if a guest user already exists in the tenant directory, a group owner can add them to a plan by assigning them a task.

Planner PowerShell
Microsoft does not publish a PowerShell module for Planner. The only processing that is possible with PowerShell is for the Office 365 Groups and group membership
used by Planner. See Chapter 14 for more information.

Phones and Meetings

As we’ve seen. Planner is integrated with Teams. But while some will use Teams to plan, many will use it for meetings. Our next chapter is all about the voice and video
aspects of Teams and how the new Microsoft Phone System makes it easy for companies to collaborate. On to 16!

Chapter 16: Teams Meetings and Cloud Voice

Ståle Hansen

Skype vs Teams
Microsoft is transitioning Skype for Business Online (SfBO) to Teams. If you create a new Office 365 tenant today with less than 500 users, Microsoft enables Teams only
mode for the tenant and users are forced to use Teams. If needed, a tenant can set the policy back to Skype. Setting Teams as a default for small tenants makes sense
from an adoption point of view. Why train users on a client and platform that is in maintenance mode?

Rolling out Teams for shared files, chat, meetings and presence is an opportunity for cultural change in your organization. You can change how people interact, move
conversations from email to Teams chat and teach your users team productivity philosophies, a step that means that more than IT needs to be involved in the project.
Microsoft has advanced many arguments to support this matter and you can check out their thinking along with some material to help the adoption of Teams at
https://aka.ms/TeamsAdoption and join the Office 365 Champions program .

To succeed with Teams adoption, you should therefore move to TeamsOnly mode as soon as possible to ensure that your users work with a single cohesive platform.
Does this mean that Skype for Business Online is going away? No, not really, but we see many Lync and Skype for Business server deployments that just use the main
meetings and chat features that can easily move to Teams. Some organizations use telephony as well, but most users just use the applications to make and receive calls
while a subset might use advanced features such as call center or recording. However, we also see a trend where these features are outsourced to other cloud based
specialized solutions, leaving the Skype client as just an endpoint. These are scenarios where it makes sense to move users to Office 365, and if you are rolling out
Teams, move Teams Only mode. Microsoft released Skype for Business Server 2019 with supportability until 2025, so Skype for Business is not going away just yet, but
all innovation around meetings and calling will happen in Teams.

This chapter lays out the case for moving to Teams as quickly as an organization can and explains the logic behind that position. We discuss topics such as
interoperability, licensing, client calling and meetings, audio and video recording, plans to allow clients to interact with the PSTN, and achieving high levels of call
quality. Any deployment of Teams that wishes to use Teams as a platform for voice and video communications should take these factors into account.

Moving from Skype to Teams

SFBO users can be moved to Teams by assigning the TeamsOnly policy to selected accounts or to everyone by making the change for the organization. Today, this update
can only be done via the Skype for Business Online PowerShell module. In the future you will be able to manage this choice in the Teams and Skype admin center as
well.

If you have an on-premises Lync Server 2013 or newer deployment today, you must configure a hybrid setup . With Lync Server 2013 you will need to move your users to
Skype for Business Online and then assign the TeamsOnly policy to users in a two-step procedure. The move request is started from one of the 2013 Front-End servers
and looks like this

[PS] C:\> Move-CsUser -Identity ken.meyer@contoso.com -Target sipfed.online.lync.com

Then you log on to Office 365 using the Skype for Business Online PowerShell module (see Chapter 4) to run the following command

[PS] C:\> Grant-CSTeamsUpgradePolicy -Policyname UpgradeToTeams -Identity ken.meyer@contoso.com

If you do not specify an identity, you will set the policy on a tenant wide level and move all your users to TeamsOnly, so make sure you use the command to set the
intended scope (user-specific or tenant-wide).

If you have a Skype for Business Server 2015 CU8 or Skype for Business Server 2019 hybrid setup, then you can run the following command and move users directly to
Teams

[PS] C:\> Move-CsUser -Identity ken.meyer@contoso.com -Target sipfed.online.lync.com -MoveToTeams

Skype and Teams interoperability

Figure 16-1: The Skype for Business client after you have been moved to Teams

After a user account is moved to TeamsOnly , the Skype for Business client might still be active on their computer. The client is still used as a meeting join client because
the Teams client cannot join a Skype meeting natively. When Teams user is invited to a Skype for Business meeting from colleagues still using Skype for Business Online
or on-premises in a hybrid setup, the Skype for Business client will launch and join the Teams user into the meeting. This is the same behavior when you get an external
Skype for Business meeting invite.

You can still chat with Skype for Business users when you are in TeamsOnly mode. Only 1-to-1 chat in plain text is supported. If the Skype for Business user adds another
modality such as audio, video, application sharing, or adds another participant, a link to the Skype for Business meeting is posted in the Teams user chat. Clicking this
link will launch the Skype for Business meeting join experience using the Skype for Business client. The Teams user cannot escalate a 1-to-1 Skype for Business chat to
audio, video, desktop sharing, or add extra participants today. The Teams user can invite Skype for Business users to Teams meetings, then the Skype for Business user
needs to join using a Teams client or at least a browser.

These restrictions mean that the Skype for Business client is still needed after moving to TeamsOnly , but only as a meeting join client. You will be able to see it in the
system tray, but if you open it, you will get the message that you have been moved to Teams as shown in Figure 16-1 . You still have a button to start Skype for Business,
this will open the meeting tab and conversation history tab only.

The first time you log on to the Teams client after moving to TeamsOnly , all internal contacts are copied from Skype for Business to Teams. External contacts will also
be copied over in a future update, but today you need to search for contacts through the Teams search box to communicate with external federated users. Federation
with Skype for Business users works the same way as for internal Skype for Business users. If the party you want to federate with has a Lync or Skype for Business on-
premises deployment without hybrid, the Teams client will not be able to communicate in that scenario. This means that a hybrid deployment is the minimum
requirement for Teams federated conversations today.

Licensing Requirements for Skype in Teams

As explained in Chapter 13, Teams uses Office 365 Groups for its membership and identity service. Teams requires that users are assigned a Skype for Business Online
license for meetings and Cloud Voice. Skype for Business Online has two standalone plans in Office 365, which define the features available to users. The major
difference between plan 1 and plan 2 is that uses with plan 1 cannot schedule or do ad-hoc conference calls, but they can join a conference call scheduled by a plan 2
user. Plan 1 users are also restricted to joining desktop sharing sessions. Plan 1 is the equivalent to a Skype for Business Server Standard CAL. It supports 1-to-1
features such as chat, presence, audio, and video internally within your tenant and in external calls. Today, only the Office 365 F1 plan includes the Skype for Business
plan 1. All other Enterprise plans use the Skype for Business plan 2, even Office 365 Business Essentials and Premium. Table 16-1 describes the features available in the
two Skype for Business plans

Feature Skype for Business Online Plan Skype for Business Online Plan 2
1

Available in the following Office 365 plans F1 Business Essentials, Business Premium, E1, E3 and
E5

Instant Messaging (IM) and Presence Yes Yes

Skype for Business to Skype for Business Audio/Video Calling (1-to-1) Yes Yes

Skype for Business Federation (IM/presence/audio/video) Yes Yes

Click to Communicate in Office Yes Yes

Authenticated Attendee in Skype for Business Meetings Yes Yes

Integration with Microsoft Exchange Yes Yes

Schedule online meetings No Yes (up to 250 attendees)

Initiate ad-hoc and Scheduled Online Meetings No Yes

Initiate Multiparty (3 or more users) Skype for Business Audio/Video No Yes

Sessions

Initiate interactive data sharing (desktop/application/whiteboard) No Yes

Audio Conferencing Add-on No Yes

Phone System Add-on No Yes (only E3 and E5)

Calling Plans Add-on No Yes (only E3 and E5)

Table 16-1: Skype for Business Online Plans in Office 365

Warning: Although Skype for Business Online Plan 2 is included with the Office 365 Business Essentials, Office 365 Business Premium and Microsoft 365 Business
subscriptions, you can’t add cloud voice calling features. The reason is that these subscriptions include Skype for Business basic features with all the conferencing
features, but no advanced calling features. You can buy a separate Skype for Business Plan 2 license for the users who need the Audio Conferencing and Phone System
add-ons. Microsoft recommend that you evaluate an Enterprise E5 (or Enterprise E3) plan to see if it is more cost effective.

Client Calling and Meeting Capabilities

Teams is available on most platforms. The client auto updates on all platforms which makes sure you always run the latest version with the newest capabilities. Since the
OS X and Windows clients are based on Electron, you should see the same features across clients. For instance, all clients across PC, web and mobile support up to four
video streams, where in Skype for Business this was different on each platform. The most notable improvement over Skype for Business is the persistent chat experience
where you always have the latest chat dialogue instantly available on any platform regardless of where you started or continued the chat.

Teams Windows client includes all the latest capabilities and does not need administrative access to Windows to be installed. It can also be deployed using MSI.
Bitness does not have to match OS or Office version which means you can combine the 64-bit Teams client with 32-bit Office. The client is available for Windows 7
and above.
Teams Mac client has feature parity with the Windows client and is where Teams really outshines the Skype for Business Mac client, which is built specifically for
Mac. The client supports Mac 10.10 and newer.
Teams Mobile client includes most of the user-focused capabilities you would expect in a mobile client. Generally, Teams administration features are weaker in
the mobile clients. Teams is available on iOS 10 or newer and Android 4.4 or newer.
Teams browser offers different capabilities based on browser type. The Edge browser (RS2 or later) supports calling and meeting capabilities without any plugin.
It is recommended to use the PC client for optimal experience because of local caching. The Chrome browser supports meetings on Chrome version 59 or newer
with calling support coming soon. Teams does not support the Safari browser and if you try to open Teams in Safari, you will be redirected to download the Teams
desktop client
Teams Surface Hub client was released in September 2018 to support the Teams app on Surface Hub devices. Three modes are available: Skype as preferred
meeting app, Teams as preferred meeting and Teams as exclusive meeting app. Read more about the setup here .
 Teams Phone Application was announced at Enterprise Connect in March 2018. This client features the Teams UI on IP phones and room phones using an
Android app to support voice centric features. The client works on targeted hardware (new and existing devices).
Teams Room System enables Teams meetings capabilities for Room System devices through the Teams UI and features. With the 4.0.8.0 update, Microsoft
introduced support for Teams meetings on Skype Room System devices. This needs to be manually enabled through the settings or pushed out through xml file.
A Teams VDI client is not on the drawing board at present. Teams is not recommended in a VDI environment today, but if you do not have a choice then consider
combining the web application with a physical calling device in the future. In July 2018, Citrix announced that Teams Optimization Pack is in the works, but gave no
concrete dates. This could be a reason to persist with people using VDI on the Skype for Business client.

Warning: Office 365 will enforce mandatory use of TLS 1.2 from October 31, 2018, so Lync Phone Edition (which does not support TLS 1.2) will be unable to connect
to Skype for Business Online or Teams thereafter. Plantronics/Polycom, Jabra, Yealink and AudioCodes have replacement offers to enable you to move to devices that
will support TLS 1.2 and Teams in the future. Read more in this Tech Community article .

Certified for Teams

Microsoft introduced the Certified for Microsoft Teams program at Microsoft Ignite 2018. This program builds on the Certified for Skype for Business program by
confirming the same functionality for Teams. The new program adds the Teams invocation and notification features to some devices. The Certified for Microsoft Teams
equipment is designed for enterprise audio to deliver the best sounding experience. iPhone and BOSE headsets are not designed for either Skype for Business Online
and Teams and the microphones do not perform well in noisy environments. However, in many situations, users do not respect this guidance. This advice is proscriptive
rather than a firm recommendation. The reason is that the certified headsets are optimized and thoroughly tested with Microsoft Teams. Some of the factors designed
into these headsets include:

Automatic selection as default audio device in Teams because they are USB based.
Plug and play.
Basic call control which means that you are muted in Teams when you click mute on your headset.
Audio quality such as no echo or excessive glitches, echo cancellation across devices, wideband audio support, comfort noise support.
The added certified Teams functionality are the invocation and notification features, which mean that you get an extra Teams button on certain devices to notify the
about missed calls. When you press the button, it opens the Teams app in the correct context on your PC or Phone (invocation). Examples of devices with the Teams
button are a new Jabra speaker device and the Plantronics Elara 60.

The biggest advantage of Certified for Microsoft Teams headsets is “comfort noise support.” This is a sound that is generated by the Teams clients when no one is
talking. BOSE or gaming headsets noise cancel this sound because that is how they are designed. Certified for Teams headsets do not filter out this noise because it is
comforting to know that you are still on the line and even though no one is talking you can hear that the call is active, avoiding the need to ask “are you still there?”
because the call suddenly went silent. This was a big problem in the early days of IP telephony and is why Microsoft added comfort noise to Voice over IP solutions.
When investing in Teams headsets, always look for the Certified for Teams logo.

The future of managing Teams devices looks bright. The Microsoft Teams and Skype for Business Admin center will include a device tab to help tenants manage their
certified devices. The new functionality is not yet in preview, but is intended to allow admins to view, assign config, get heartbeat, ping, restart and factory reset devices
such as IP phones, conference phones and Skype for Business room systems. You will also be able to update their firmware and applications and schedule the updates at
an appropriate time is also on the roadmap. Microsoft also confirmed that Surface Hub management is on the roadmap, but this is not yet available.

Note: To find devices certified for Microsoft Teams go to https://office.com/teamsdevices

Meetings in Teams
Meetings in Teams are part of Microsoft’s Intelligent Communication strategy where they want the meeting experience to be as intuitive as possible for chat, notes,
content, audio, and video. Getting comfortable with meetings can change how you plan your day because you can join meetings from wherever you are. The
responsiveness and stability of the meeting experience on the Teams mobile client is far better than what exists in Skype for Business and may help companies to adopt
the idea of meetings on the go. Read more about creating meetings in Chapter 13.

You can divide meeting scenarios in three general categories. Different preparations go into creating each category. These are:

Collaborative and workshop meetings with up to 20 or so participants . Typically, attendees have a lower expectation of quality and the meetings are
informal enough to fix performance issues on the go. Can either be an ad-hoc meeting from private chat, channel or outlook invitation. Audio Conferencing is an
important feature for these types of meetings because attendees can call in to join the meeting when they are in a location with poor internet connectivity.
Planned informative meetings and departmental meetings with up to 100 participants . Less interaction occurs between presenters and attendees during
the meeting and attendees have less acceptance for technical difficulty and performance issues. This type of meeting is either a regulated Teams meeting with a
producer and presenter, and the meeting is recorded via Cloud Recording, or a Live Event where attendees can attend via streaming or watch it later via on-
demand video. Q&A is typically handled via Teams chat or Q&A module if it is a Live Event.
All hands meetings, webinars, training, and meetings where the potential attendee list may surpass 100 . There is no acceptance for technical and
performance issues, with little to no interaction in the meeting. Live Events are perfect for these types of meetings. You can produce a live stream of the meeting,
or the one to many presentation, which can be viewed internally for authenticated users or externally for anonymous users. The stream can be consumed in the
Teams client or as a streaming video in a modern browser and will be available on-demand for those who were not able to view it live.

The maximum number of attendees in a regular Teams meeting is 250, which is the same as Skype for Business, while up to twenty people can participate in a private
chat.. There are fewer controls available for the meeting administrator to manage larger meetings in Teams than in Skype for Business. The only control available right
now is the ability to mute individuals or everyone during the meeting. All attendees join as full presenters which means anyone can take over as presenter and unmute
themselves. In meetings larger than 20 participants it is wise to appoint a producer to manage the meetings such as reading questions and muting those who forget to
mute themselves. In even the most organized company, there’s always someone who forgets to mute themselves and shares background noise with the other
participants.

Everyone in your tenant that need to host and attend a Teams meeting needs a Teams license. In addition, you can invite guest users to meetings, who need to be logged
in to your tenant to access the full chat and meeting experience. For instance, if a guest is logged into their tenant rather than yours, they can still connect to a meeting
in your tenant, but they will not be able to chat with other participants. You can control guest user meeting settings under guest access in the Microsoft Teams and
Skype for Business Admin Center. If you send a Teams meeting invitation to an external user who is not a guest in your tenant, they can join the meeting anonymously. In
this mode, the user has audio, video and desktop sharing functionality, but cannot see or contribute to chats. They can join the meeting via Edge or Chrome or use the
Teams desktop client. This is not federation as we know it from Skype for Business, which will come at a later stage.

Real World: Regardless of the scenario, to succeed with audio and video in Teams in general and especially in meetings, there are some rules users must follow. In a
Teams rollout, we received complaints about users having bad experiences with network or devices in meetings. To level set user expectations, the following
information was sent out and put on the intranet. It was one of the most read articles about the project, and after the information was sent out, the number of support
calls about quality almost disappeared. This shows the importance of information and that Teams is a user-centric solution. Since we did nothing on the backend to
improve the solution to reduce support calls, it was all about user awareness. The below text was sent out:

How to improve audio and video quality in Teams

Sometimes you may experience network issues, which may result in poor audio and video performance in Teams calls and meetings. Here are some tips to improve the
quality.

The capacity of the network is good, but some factors affect quality such as video streaming events and heavy usage on the Wi-Fi network. The IT department is
improving Wi-Fi bottlenecks and upgrading equipment continuously. Regardless, there is some advice you can follow to ensure a good sounding meeting, here are a
couple of tips:

Use the Skype certified headsets provided by IT in all scenarios. This equipment is designed for Skype for Business and Teams to give the best sounding
experience. iPhone and BOSE headsets are not designed for Teams, and the microphones do not perform well in noisy environments. Also, avoid using Bluetooth in
free seating areas, connect the headset using the USB wire.
Use wired network whenever it is available and use the network cable when you are in a meeting room.
If you are in a meeting room with no wired connection, but a conference phone is present, you can dial into the meeting using the number provided in the meeting
invitation. In phone calls, the audio goes over the PSTN network instead of Wi-Fi network.
If you are presenting a PowerPoint in a meeting, it is recommended to share it as PowerPoint presentation in the Teams meeting instead of application sharing. The
performance is better, and only next slide commands are sent to each participant instead of a video stream, which makes it less dependent on network quality.
Be mindful of your resolution when sharing your desktop. Try to share the specific application you are sharing from and make the window smaller instead of
sharing the whole desktop.
If you experience poor performance, you can turn off video in the meeting to save network bandwidth.

When planning larger, more structured Teams meetings, it is important to define specific roles for the meeting. The reason for this is because attendees have a high
expectation of performance and a low tolerance for technical issues. The roles are listed below:

Producer , is responsible for making sure the audio and content are presented as expected and check that the presentation is working. The producer is also
responsible for switching layout and checking if changes occur in quality and stability.
Presenter , focuses on content delivery. It is recommended to use a wired certified for Skype for Business headset for good sound quality if it a webcast and a good
wireless Bluetooth headset if you are presenting with video such as the Sennheiser Presence.
Q&A moderator , is responsible for following the QA module and either answer questions directly or make sure the speaker get asked those questions live in the
meeting. The moderator can be a person assigned the producer or presenter role.

Besides the regular meeting functionality in Teams, as discussed in Chapter 13, you have the features Audio Conferencing, Cloud Recordings and Live Events.

Audio Conferencing for Teams Meetings

Assigning an Audio Conferencing license to a user enables them to create Teams meetings with a dial-in telephone number, which means users can dial into a meeting
using a regular phone, in addition to the the Teams client. The dial-in option increases the usability of Teams meetings and is a popular feature with organizations that
make use of conference rooms where attendees want to use the phone in the room to join meetings. Audio Conferencing also allows external users to dial into meetings
from a mobile phone which is beneficial for joining via audio from locations with poor internet coverage. Audio Conferencing does not require the Phone System license.

You need to assign the Audio Conferencing license to user accounts via the Office 365 admin center or add it as part of your license lifecycle management via PowerShell
or Azure Active Directory Group-Based Licensing, (see Chapter 4). An Audio Conferencing number is automatically assigned to users based on location as specified in
the Office 365 admin center. You can change this by editing a user in the Microsoft Teams & Skype for Business Admin Center.

The Audio Conferencing license can be purchased in 72 countries, and you can get dial-in numbers for 90 countries and 400 cities. Dial-out is the capability where you
can add a user to an existing meeting by a phone number. Dial-out is available to 190+ countries. Up until July 1, 2018, Microsoft offered a complementary dial-out
period where dial-out was free in 44 countries. This period has now ended, and users need to be assigned a Communications Credits license to be able to dial-out.

Note: You can find an updated downloadable spreadsheet with country and region availability for Audio Conferencing here .

Cloud Recording
Cloud Recording is the ability to record a Teams meeting with audio, video, desktop and application sharing. PowerPoint sharing is not supported and may not ever be
supported as the feature is a video-based recording system. The recording is uploaded and processed in Microsoft Stream. Being able to record a meeting using a cloud-
based service is a feature that was highly requested for Skype for Business. By uploading the video to Stream, you can get automatic transcripts created for the meeting
(currently only available for English and Spanish speaking videos). This is a big step towards Intelligent Communications strategy since videos are easier to search for
when transcripts are applied.

As with everything else in Teams, Cloud Recording depends on a separate license. In this case, Microsoft Stream, which the meeting organizer and the person who
starts the recording in a meeting need. The person who starts the recording also needs upload rights to the Office 365 Group associated with the team that hosts the
meeting. From November 2018, Office 365 E3 and E5 plans include Stream Plan 2, which means that any video uploaded to Stream, including recordings captured from
Teams meetings, can use the transcription, deep search, and face recognition features. Guests and federated users are not able to start a Cloud Recording. See Chapter
7 in the companion volume for more information about Stream.

The size of a 1-hour recording is approximately 400 MB. The amount of storage you have available for Microsoft Stream is 500 GB base storage plus 500 MB per
licensed user. The storage is pooled tenant wide, and more storage can be purchased. Only the original video file size counts against your pooled storage. Transcoded
videos, thumbnails, channel poster, subtitle and caption files do not contribute towards your storage.

Note: Microsoft Stream is not included in the Business Premium or Business Essentials plans. The Microsoft Stream Plan 2 Add-on is available for Office 365 K, and
E1. Office 365 Enterprise K1 users are not included and do not contribute to additional storage quota.

Figure 16-2 shows a Stream video created via Cloud Recording of a Teams meeting where you only show the current application, in this case, it is a Word document.
Cloud Recording of Teams meetings are enabled by default for those that are assigned both a Teams license and a Stream license.

Figure 16-2: A cloud recording of a Teams meeting being played in Stream

Background Blurring : The CTRL+Shift+P key combination invokes background blur during meetings, meaning that anything but the user’s image is blurred. The
feature only works on Windows PCs equipped with a post-Haswell chipset, which is needed for the AVX2 (Advanced Vector Extensions) calls used to create the blur
effect. The Teams Mac client is scheduled to support background blur in the future.

Teams Live Events

For all-hands meetings, webinars, training, and meetings where the main topic is a one to many presentations you should consider using the Live Events feature, which
is currently in public preview. Live Events is the next iteration of the Skype for Business Meeting Broadcast functionality and it has a much better end-user experience.
You have two types of Teams live events, quick start and external encoder. Quick start events are produced directly in Teams, and the current features are audio, video,
screensharing, application sharing, and QA. External encoder is when you produce the content directly from external hardware or a software-based encoder.

Teams Live Events Quick Start

To get started with live events produced from Teams you need to go the Meetings tab in the Teams client. When you click schedule a meeting, you get a drop down as
shown in Figure 16-3 , under the option to schedule a new meeting. From here the meeting invitation looks just like a regular meeting, except that the organizer is a
producer of the event. You can only invite individuals, not groups or channels, as part of the event group. You have two roles available for the event group

Producer is the host of the meeting; producers ensure attendees have a great viewing experience by controlling the layout and media sources that are sent to the
event. They get extra abilities such as a queueing of layouts and a live layout view.
Presenters join the meeting in Teams and get the same look and feel as in a regular Teams meeting. They can share audio, video, desktop and application in the
meeting and have access to chat, meeting notes and can moderate the Q&A.

Office 365 guests, federated, and anonymous users can't be invited as producers or presenters in Teams live events; they can only watch the live event as
anonymous users.

Figure 16-3: Scheduling a Teams live event

When you click next you get to configure the live event permissions and live event production type

Live event permissions are where you specify who can watch the stream and on-demand recording of the meeting. You can specify people and groups inside your
Office 365 tenant, accept anyone from your Office 365 tenant or make it public. When making it public, anyone with the link can watch the recording which is ideal
for public webinars and presentations.
Live event production type is where you specify if the recording is available for download by the producer and presenter after the meeting. If the stream is available
on-demand for attendees, they can watch the on-demand recording up to 190 days after the stream has ended. You can also disable and enable Q&A
If external encoder is set up in your tenant, then you can specify this as a source.

After you schedule the meeting, you can go back and edit the meeting invite and settings. The public permission and the choice to disable recording for download is
disabled by default (for the preview). To enable these settings, go into Live events policies in the Microsoft Teams & Skype for Business Admin Center and change the
global policy or create a new policy with these settings enabled and assign them to specific users. You can also set this in PowerShell using the Set-
CsTeamsMeetingBroadcastPolicy cmdlet as follows:

[PS] C:\> Set-CsTeamsMeetingBroadcastPolicy -Identity Global -BroadcastAttendeeVisibility Everyone

-BroadcastRecordingMode UserOverride

The preview of Teams Live Events is only available for Office 365 E3 and E5 tenants. Producers and presenters need a Microsoft Teams, Skype for Business, and
Microsoft Stream license. If you are an authenticated user and want to attend a live event, you need to have a Microsoft Teams license assigned.

Teams Live Events External Encoder

If you have access to studio quality production equipment, you can use the external encoder option. The requirement is that the hardware or software-based encoder
support streaming to a Real-time Messaging Protocol (RTMP) service. Some of the encoders that are tested by Microsoft at this point are Haivision KB, OBS Studio,
Wirecast, and XSplit Broadcaster to name a few. To schedule a live event with external encoder, you schedule the meeting the same way as for quick event. On the
settings page, you select external encoder. Note that external encoder events are only consumed via Microsoft Stream, which is why public viewing will be unavailable
for external encoder meetings. The stream can be viewed directly in Stream or via the meeting invite in Teams.

You need to do the following to get started with external encoder live events

A global administrator of the Office 365 tenant needs to be assigned an E3 or E5 Stream license.
Then you need to navigate to https://web.microsoftstream.com/admin .
Click Live events (preview).
Add users that already have Stream licenses as Current unrestricted users and Save .
Wait a little time and then make sure users can click the external encoder choice when scheduling a live event in Teams.

Note: The functionality of Live Events might change after the preview. The option to use external encoders for events is a new feature and is probably going to change
in some aspect before GA. We will add more content on external encoder when it is generally available, but for now, you can read this great write-up by MVP Luca Vitali
on how he set up external encoder integration with OBS Studio. Here is the link to all tested encoders by Microsoft.

Content Delivery Networks

In larger environments such as an all-hands meeting or internal training where you expect a large internal audience, you may want to consider an Enterprise Content
Delivery Network (eCDN) to minimize the streaming footprint on your internet connection. Hive Streaming and Kollective are examples of eCDNs and are already
integrated with the live event service. A stream may consume up to 1700 kbps per stream and may affect internet performance if many users in the same location watch
an event. You need to install the third-party software on each client to set up a peer to peer streaming network inside your organization. The eCDN’s may reduce
internet bandwidth usage by up to 98%, because of the reduction of clients downloading the stream from the internet and thus saving internet bandwidth. An example of
enabling Hive Streaming for your tenant by amending the broadcast configuration with PowerShell is shown below. You need to enable an eCDN provider, specify which
one and redirect the stream to their service.

[PS] C:\> Set-CsTeamsMeetingBroadcastConfiguration -AllowSdnProviderForBroadcastMeeting $True -SdnProviderName hive -SdnLicenseId {license ID GUID provided by Hive} -SdnApiTemplateUrl “{API
template URL provided by Hive}”

Teams Meetings and Cloud Video Interoperability (CVI)

For users to succeed with Teams meetings, they need to be able to book meetings and to join meetings no matter where they are situated at the time. Teams supports
Audio Conferencing to allow users to dial in to a meeting by phone, including a great mobile meeting experience. The last barrier to overcome is the ability to join a
Teams meeting from VTC (Video Teleconferencing) endpoints. Even though Skype Room Systems V2 with Teams software and Surface Hub with Teams software offers
this functionality, it may still be too expensive for an organization to replace fully functional VTC’s in meeting rooms with Certified for Teams devices. This is what the
Teams CVI business case is as it solves two scenarios:

1. Reuse existing VTC’s in an organization.

2. The ability to invite anyone, internal and external, in to a Teams meeting regardless of technology.

The biggest difference with Teams CVI compared to Skype for Business video interop is that the meeting always runs on the Teams platform. A Teams client cannot dial
a virtual meeting directly or other endpoints directly via the interop service. VTCs must always dial in to join the meeting with the meeting details provided in the CVI
enabled meeting invite.

There are three providers available today and they are all preconfigured as teams video interoperability policies in your Teams tenant. You can find them in PowerShell
via the Skype for Business module (see Chapter 4 to learn how to connect). You can run the following command to get the list:

PS C:\> Get-CsTeamsVideoInteropServicePolicy | Format-Table Identity, ProviderName

Identity ProviderName

-------- ------------

Global DefaultProvider

Tag:PolycomServiceProviderEnabled Polycom

Tag:BlueJeansServiceProviderEnabled BlueJeans

Tag:PexipServiceProviderEnabled Pexip

Tag:ServiceProviderDisabled DefaultProviderFunction

Each provider has their own flavor to CVI:

Polycom is the only provider where the licenses integrate into the Office 365 admin portal. It is a shared service and all VTCs join either via the meeting lobby or
direct into a meeting. This is because it is a shared service and there is no way to differentiate between your own endpoints and external endpoints.
BlueJeans is also a shared service. Licenses are bought directly from BlueJeans.
Pexip is the most flexible of the solutions. A tenant can choose to host the service, use dedicated hosting from a service provider, or access a shared service
delivered by a service provider. Licenses are bought from a Pexip partner. This is the only solution that can let your internal VTCs join a meeting directly while
making external VTCs enter the meeting via the lobby.

The most approachable of these providers are Polycom as licenses are managed in the Office 365 admin portal, which suits those wanting a simple solution to enable
CVI. You need one license per user who needs to have the CVI data included in their meeting invite. Pexip is the most flexible solution and suits the enterprise market
with a higher demand for integration with existing video infrastructure, customization and control over media traffic.

After choosing your preferred provider you must run the New-CsVideoInteropServiceProvider command to configure the provider and the settings. You can set this up
with false info in advance just to see how it looks in your tenant. You can create the provider and grant the policy tenant wide or on a per user basis:

[PS] C:\> New-CsVideoInteropServiceProvider -Name CVI -TenantKey "teams@yourdomain.com"

You can then grant the policy either at a tenant wide level or per user. For example:

PS C:\> Grant-CsTeamsVideoInteropServicePolicy -PolicyName PexipServiceProviderEnabled -Global

PS C:\> Grant-CsTeamsVideoInteropServicePolicy -PolicyName PexipServiceProviderEnabled -Identity "username@domain.com"

The example grants the Pexip service provider policy to a user or at a global level. Granting the policy automatically formats your meeting with info from the correct
predefined provider and may look like that shown in Figure 16-4 . The meeting can be booked via any Teams client or via the Outlook plugin.

Figure 16-4: Scheduling a meeting with the CVI information highlighted at the bottom

Calling in Teams
Before someone can use Calling in Teams, their account must be assigned a Phone System license. From there you can either add calling plans, which are available in
select countries, or assign numbers from your own SIP trunk via Direct Routing, which is available in all Office 365 tenants. To understand calling in Teams you need to
understand communication credits, the different type of number and the functionality available through the Phone System license as explained below.

Communications Credits
Communications Credits are a way to pay for dial-out functionality for users assigned an Audio Conferencing license or a Calling Plan via the Phone System license.
Funds are added for the entire tenant, but only the users assigned the license will have the ability to dial out. The capabilities Communication Credits enable are:

Ability to add toll-free numbers so it is free for those dialing in to Audio Conferencing meetings, auto attendants and call queues. Toll-free calls are billed per
minute and need a positive Communications Credits balance.
Dial-out using Audio Conferencing as mentioned above.
Dial-out from a Teams meeting to your mobile or using the call back feature.
Dial any international phone number when you are assigned a Domestic Calling Plans license.

Communications Credits are available in the same countries that has Audio Conferencing available and you will be able to buy and assign the license as soon as you have
either an Audio Conferencing license or a Phone System license in your tenant. The amount of funds you need to allocate varies greatly on expected usage and types of
licenses assigned. You should check the number of licenses, types of licenses and actual usage to tune your Communication Credits funding. The funding can be auto
recharged to avoid any disruption in service.

Note: Funds will be applied only to Communications Credits at Microsoft published rates when the services are used. Any funds not used within 12 months of the
purchase date will expire and be lost.

Phone System Capabilities

Phone System is the equivalent of the Plus CAL for Skype for Business Server. This enables Teams Calls interface and advanced capabilities such as delegation, team call
and voicemail for users and Call Queues for organizations. It also enables you to bring your own SIP trunk to Office 365 with Direct Routing and is a requirement if you
want to implement Skype for Business Server 2019 hybrid voice. Both these solutions enable users homed in Office 365 to be assigned a phone number and make and
receive calls.

Service Numbers
Service Numbers are numbers that can be requested in a limited supply per tenant to handle a large amount of calls. Microsoft provides these numbers and do not
integrate with an existing telecommunications service. You are, however, able to port numbers from your existing telco over the Microsoft Service Numbers. These
numbers are assigned to services such as Audio Conferencing, Auto Attendant and Call Queues.

There are two types of Service Numbers, toll numbers and toll-free numbers. The most common type of service number is a toll number. All that you need to order toll
numbers is a Phone System license. Toll-free numbers can only be ordered when a Communications Credits license is added to the tenant and set up with a prepaid
amount. When assigning toll-free numbers to an Auto Attendant, the call is free of charge for the customer dialing in to the service and the company pays that cost as a
charge against the Communications Credits prepaid amount. When using toll numbers, the user dialing in to the service pays the regular fee. Service numbers are
available in 65 countries today and the number of available markets will grow over time.

Auto Attendants and Call Queues

Auto Attendant is not the same as the Auto Attendant you may know from Exchange Server. The Auto Attendant in Office 365 is an automated interactive system for
incoming calls that can generate welcome messages, give opening hours, provide interactive menu choices, and route calls to other Auto Attendants, Call Queues and
users directly. It also includes a Directory Search to reach a person by name and searches the entire tenant by default.

Call Queues are simple hunt groups. You would typically have an Auto Attendant in front of a call queue to enable opening hours and interactive routing. Call Queues are
used to distribute a large amount of calls (up 200 calls per instance) and use the first in, first out principle. There can be up to 50 agents per Call Queue and it is
possible to opt out of the queue if you are assigned as an agent. The call is presented to all agents in parallel and you are presented with a call even if you are in an
existing call, unless you are offline or set your presence status to do not disturb. Microsoft expects to deliver the ability to use Teams as a client for Auto Attendant and
Call Queues later in 2018.

Note: There are several clients that work with Call Queues. Two examples are the TouchPoint Attendant by Enghouse Interactive and Attendant Pro by Landis
Computer. These are client based attendant consoles and do not need any server component to work with Auto Attendant and Call Queues.

Auto Attendants and Call Queues are part of the Phone System license and do not need extra licensing. Both services need service numbers to be assigned to your
tenant for setup. Users that are part of Auto Attendant and Call Queues are called agents. They need to be assigned a Phone System license, but do not need to be
assigned a phone number. The agents need to be homed in Office 365 to answer the service. Agents are not assigned directly but must be a member of a distribution
group that you assign to Auto Attendants and Call Queues.

Real World: Auto Attendants have opening hours, but you need to add holidays manually or through PowerShell. There is a script created that
imports holidays for your country and automatically sets holidays for your specified Auto Attendant. Download and evaluate the script here .

Caller ID
Caller ID is the ability to control which number is shown when a user dials out from Teams. By default, you will show your assigned number. You can choose to change
that to a service number assigned to your tenant or to be anonymous. The most common scenario is to show a service number assigned to an Auto Attendant in your
organization. To add this feature to a user you first need to create a Calling ID policy and grant this to a user or a set of users. In all cases, the "Service Number" field
should not start with "+."

[PS] C:\> New-CsCallingLineIdentity -Identity "UKOrgAA" -CallingIdSubstitute "Service"

-ServiceNumber "14258828080" -EnableUserOverride $false -Verbose

[PS] C:\> Grant-CsCallingLineIdentity -Identity "amos.marble@contoso.com" -PolicyName “UKOrgAA”

You can also block incoming caller ID if needed using the same calling ID policy command

[PS] C:\> Set-CsCallingLineIdentity -Identity "Block Incoming" -BlockIncomingPstnCallerID $true

-EnableUserOverride $true

[PS] C:\> Grant-CsCallingLineIdentity -Identity "amos.marble@contoso.com" -PolicyName "Block Incoming"

Voicemail
Voicemail is a default part of the Phone System license and is not the same as the voicemail capability available in Exchange Online and On-Premises. Microsoft hosts
the voicemail in Azure and you enable it for an account with the Set-CsUser command. It is also possible to integrate cloud Voicemail with a hybrid Exchange solution,
but the user needs to be hosted online for Teams. Phone System voicemail supports depositing voicemail messages only in an Exchange mailbox and doesn't support any
third-party email systems. As a fallback mechanism, Phone System voicemail can resend messages using SMTP, which means users with a mailbox on a third-party email
system will receive their voicemail messages with no guaranteed service uptime or other voicemail features, such as changing their greetings and other settings.

Transcript is enabled by default for all users enabled for Phone System voicemail and can be turned off using the Set-CsOnlineVoicemailPolicy command. Twenty-seven
languages available for greetings and transcript is available for 10 of those languages. Read more on which languages are supported here .

Phone System with Calling Plans

Calling Plans is the ability to order user numbers from Microsoft and assign them to users. The numbers are provided by Microsoft and this service does not integrate
with existing telecommunications services. Calling Plans requires that the user is assigned the Phone System license in addition to the Calling Plans license. Calling
Plans is a prepaid service consumed tenant wide by those users assigned with these licenses. User Numbers are available in ten countries with more due to be
supported in the future. Typically, domestic calling minutes are 3,000 in US and 1,200 in EU countries, with 600 tenant-wide international calling minutes. If you need
more minutes, you buy another US Calling Plans subscription and you get 3,000 added calling minutes. If you have two Calling Plans subscriptions, you get 6,000
domestic calling minutes each month. Unused calling minutes do not carry over to the next month. You can download the spreadsheet with available countries here .
Calling Plans comes in two types of licenses:

Domestic Calling Plan where licensed users can call out to numbers in the country/region where they are assigned in Office 365.
Domestic and International Calling Plan where licensed users can call out to numbers in the country/region where their Office 365 license is assigned to the
user based on the user's location, and to international numbers in 196 countries/regions.
Communication Credits can be used together with Calling Plans and affects the ability of users with Domestic Calling Plan as it will allow them to dial out to all
available international countries/regions.

When you buy a Calling Plans license, you can order user numbers for that country. User Numbers are numbers that you assign directly to users and are meant to be
used for calls by those individuals. To order User Numbers, you need to have a Calling Plans subscription assigned to your tenant.

Phone System with Direct Routing

The Phone System license allows a company to connect their own SIP trunk to their tenant. This is called Direct Routing and needs only a SIP trunk from a certified
session border controller (SBC). Local or global PSTN connectivity to your SBC can either be provided by a local PBX or connected directly to a SIP trunk. To set Direct
Routing up you need to implement a certified SBC by AudioCodes or Ribbon in your DMZ, a public IP address, DNS entry and a public certificate. Read more about the
detailed infrastructure requirements in this blogpost .

There are three main scenarios for Direct Routing

Installed in Your Datacenter

If you have an advanced telephony deployment already in use in your environment, you can stretch this to your Teams users. You can deploy multiple SBCs in same
location for redundancy and in different datacenters for global availability with the ability to connect local PSTN providers using local telephone numbers. This ensures
that you will be able to provide PSTN connectivity in countries where calls need to break out locally and you can reuse existing infrastructure.

Hosted by a service provider

Since the integration with Teams using Direct Routing is just a SIP trunk connection, service providers can easily connect to multiple tenants from a centralized SBC
deployment. All they need is the correct basic setup mentioned above and a base DNS domain. The service provider will then be able to register a subdomain per
customer and connect it to the customer tenant. Direct Routing supports wildcard certificates and all the service provider needs to do is register a wildcard certificate
for the base domain. Why does this work? The domain the SBC registers to the Office 365 tenant with has nothing to do with the SIP domains the customers are using in
Teams. Remember this is just a SIP trunk and all it needs is a FQDN registered in Office 365 for connectivity. A service provider can then connect multiple customers to
the same partner operator SIP trunk or provide connectivity to multiple operators to the same Office 365 tenant or multiple Office 365 tenant. Read more about the
specifics here .

Installed in Azure
AudioCodes Virtual Edition SBC is supported by AudioCodes to run in Microsoft Azure. Microsoft has certified this SBC for Direct Routing which implies that they also
support the SBC in Azure. This is the first time we can implement voice features that integrates with a Microsoft service in Azure. If you have a strategy for moving as
many services to Office 365 and Azure, this is a very welcome option. This also applies if you do not have a datacenter but want to consume local PSTN connectivity and
connect it to Office 365 via an Azure appliance. The same general infrastructure requirements for Direct Routing apply when installing in Azure. AudioCodes has created
a whitepaper on how to set up their virtual SBC in Azure

Migrating from Skype for Business to Direct Routing

If you have Skype for Business Server with Enterprise Voice or Skype for Business Cloud Connector Edition (CCE) deployed today, you can place the SBC in front of your
Skype deployment and connect the PBX or PSTN providers SIP trunk to the SBC. You will then be able to route numbers either to the Skype connected SIP trunk or the
Direct Routing connection as part of the move to Microsoft Teams. This approach enables you to re-use your existing PSTN connectivity in Teams. CCE was the way to
connect Skype for Business Online user to a local PSTN connected SIP trunk. CCE does not work for users on Teams and Direct Routing does not work for users on
Skype for Business Online, that is why you must consider the approach described above.

If you have Skype for Business Server, you will get the ability to migrate scheduled meetings and contact lists directly to Teams. This requires that Skype for Business is
integrated with Skype for Business Online via hybrid connectivity. When all users have been moved to Teams, you can decommission the Skype for Business
environment, reducing server footprint and complexity.

You need to look for alternatives to complex third party integrations such as switchboards, call centers or recording solutions. Today, limited possibilities exist to develop
third party call control solutions in Teams. The recommendation is to look for native solutions outside of Teams for the subset of users that has special needs. For
instance, traders need to have their calls recorded for documentation and there are lots of dedicated solutions available that can solve that problem such as Telstra and
Unigy 360. With a simple SIP trunk integration with your Direct Routing SBC, a third party can deliver an advanced specialized solution for the traders. They are still
connected to same telephony solution as the Teams users but get their custom solution for an optimal set of functionalities.

Real World: A typical migration from Skype for Business telephony to Direct Routing can might include the following steps:
Implement Skype for Business hybrid.
Validate PowerShell connectivity from one of the Front-End servers to Office 365.

Needed to be able to move users.

Implement the SBC.

Set up and validate Direct Routing.
Move users to Office 365 using the direct to Teams PowerShell switch.

This will move the users directly from Skype for Business Server to the Microsoft Teams service.

Finalize the migration.

Move DNS pointers to Office 365, such as SIP and federation SRV records.
Turn off the Skype for Business Servers for 1-2 weeks.
If no incidents start the decommission process by clearing out the Skype for Business and publish the topology.

This will clean up AD for references to servers and roles.

Replicate the Configuration Store.
Run setup on all Skype for Business servers and uninstall roles.
Disjoin all servers from AD.
Delete all Skype for Business Servers.

Succeeding with Call Quality

Teams helps you to transcend geographical boundaries by connecting you to people in other locations. To succeed within an organization, you need to make sure calls
get connected properly and call quality is good. What constitutes good call quality? This is highly subjective, but packet loss, jitter and latency are key network metrics
you can measure. By understanding that signaling and media can take different paths, you can make sure that media has an optimal path between the sender and
receiver in 1-to-1 calls and conferences. This is where Teams gets difficult to understand and master, but luckily there are several tools available that we can work with
such as The Call Quality Dashboard and Call Analytics. Before diving in to the tools it is important to understand the core technology. Most of what you need to know is
available as part of MyAdvisor for Cloud Voice which is part of the Fast Track program.

MyAdvisor is the latest iteration of the Skype Operations Framework (SOF). The goal of MyAdvisor is to help you in three stages of your Teams Cloud Voice lifecycle:

Planning and preparation where you work with key stakeholders and the business case for why you want to do Teams and Cloud Voice. In this stage you will
 actively determine network readiness by using tools like Network Planner and the Network Assessment tool to discover bottlenecks and showstoppers.
Actively Deploying is where rollout and adoption is important. If you are rolling out Teams for the first time, this stage should be coordinated as you integrate
other workloads with Teams such as Exchange, OneDrive and SharePoint.
Maintaining and supporting is where you proactively and reactively monitor voice quality in the solution. Primary tools are Call Analytics and Call Quality
Dashboard, which are described further down in this chapter.

Note: The bulk of the material in MyAdvisor is still based on Skype for Business Online. Look through that content and get inspired for Teams as it contains a lot of
good elements for succeeding with Cloud Voice. In time, Microsoft will update MyAdvisor with Teams specific material. You can get started with MyAdvisor here
http://aka.ms/MyAdvisor .

You need to have a basic understanding of the underlying technology to make Cloud Voice work to use the Network Planner, the Network Assessment tool, Call Quality
Dashboard and Call Analytics effectively to plan for the optimal media path. Concepts to understand are

Signaling and media and that they can be sent down different routes, depending on scenario.
Codecs and their impact on the network.
Network factors and how they affect voice quality.

Understand Signaling and Media

Every call has two parts, signaling and media. It is important that you have a basic knowledge of this to understand how media traffic flows in your environment.
Signaling is the part that manages your call such as establishing, maintaining, and terminating it. The concept in Teams is the same as for Skype for Business Online.
Skype for Business was based on the Session Initiated Protocol (SIP) while Teams uses HTTP REST signaling between the Teams clients and the back-end service. Media
communications is still based on UDP and travels as direct as possible in all scenarios.

Figure 16-5: Signaling goes via the Call Control and media goes directly between the clients

In 1-to-1 calls media will try to go directly between the clients, after signaling has established that it is possible for them to connect directly because they are in the
same corporate network or in the same subnet as shown in Figure 16-5 . Voice and video quality have a higher chance of good quality with less packet drops than when
media goes over the internet.

In 1-to-1 calls where media cannot go directly because a firewall is between subnets or one user is external, the media will be relayed through Office 365 as shown in
Figure 16-6 .

Figure 16-6: Signaling goes via the Call Control and media goes via the relay service in Azure

In an ad-hoc multiparty call with more than two participants or in a scheduled meeting, media will go to the conferencing service in Office 365 as shown in Figure 16-7 .
Regardless of who invites to the scheduled meeting, the first participant to join the call is where the conference is hosted. For instance, if you have three participants in
US and 10 participants in EU and a US participant joins first, the conference will be hosted in a US datacenter, and all 10 EU participants must join the call hosted in the
US. This is different than it was in Skype for Business where the meeting was hosted where the scheduler was homed.
Figure 16-7: Signaling goes via the Call Control and media goes to the AVMCU where the first joiner is homed

Codecs and their impact on the network

The biggest difference between Skype and Teams is that the SILK codec is used in almost all scenarios. This is part of the next generation platform that is built for from
the ground as a cloud service for Teams. There are three codecs used for Teams:

SILK is the primary codec in all scenarios and is both wideband and narrowband across clients except for Edge and Chrome but including the PSTN calling
scenario.
G.722 is the secondary codec that is available in all scenarios across clients.
G.711 is the secondary codec in PSTN calls.

Table 16-2 shows the typical bandwidth usage of Teams codecs. Forward Error Correction is when you start having packet loss in the network and Teams compensates
by sending overlapping information per packet over the network so that we lose less information per packet lost.

Audio Scenarios Audio payload Bandwidth Payload, IP Bandwidth payload, IP header, Bandwidth Payload, IP header, UDP, RTP, SRTP,
codec bitrate header UDP, RTP, SRTP Forward Error Correction

UDP, RTP

SILK Primary in 36.0 52.0 64.0 100.0

Wideband all

SILK Primary in 13.0 29.0 41.0 54.0

Narrowband all

G.722 Secondary 64.0 80.0 95.6 159.6

in all

G.711 PSTN 64.0 80.0 92.0 156.0

Table 16-2: Teams and codec bandwidth in kbps for audio

Video is based on X.264, the same as Skype for Business. Resolution may vary based on the screen the participants use in the meeting and the resolution they request.
The more participants there are in the meeting, the smaller the requested video gets. All clients support up to four video streams. Video Based Screen Sharing (VbSS) is
the only way desktop and windows are shared. PowerPoint is still presented through the Office Online Server and is not video based, that is why PowerPoint content is
not recorded when using Cloud Recording. Table 16-3 shows some video scenarios and potential bandwidth when using 1080p resolution. Be aware of the equipment
your users are using and where most of them may run video meetings and make sure you scale available bandwidth to anticipate the estimated load. This is where the
network planner will help.

Participants/Activity
Max resolution Total max download bit rate (Mbps) Total max upload bit rate (Mbps)

2 Participants
1 * 1920x1080 4 4

3 Participants 8
2 * 1920x1080 (Full Bleed) 6.5

2 * 1280x720 5

4 Participants
1 * 1280x720 + 2 * 960x540 5.5 4

5+ Participants
4 * 960x540 6 1.5

Video Based Screen Sharing (Only)
1 * 1920x1080 4 4

N Participant + VBSS [N=0-4] 1 * 1920x1080 + N * 424x240 4 + (N*350 Kbps) ~4.34

Table 16-3: Teams and codec bandwidth for video

Network factors that affect voice quality

Network metrics such as packet loss, jitter and latency affect voice quality. These network metrics affect real-time media and are experienced in different ways.

Packet loss is experienced in two ways. That metallic and variable sound quality you hear is caused by random packets being dropped. Sometimes you may hear that the
person talking may go silent for several seconds, that is caused by burst loss of packets, where contiguous packets are lost. Packet loss is typically caused by
transmission errors and router congestion. If contiguous packet loss exceeds 10 seconds, the call may be disconnected. This is typically a problem over Wi-Fi that is
designed for access and not throughput. Especially if you walk around with an active call and need to switch access point, the handover time may be too long, and the
call gets disconnected.

Jitter is caused by packets arriving in a different order than the order they were put on the network and with longer intervals. It is typically caused by packet taking
different routes due to load balancers or re-direction due to router congestion. You experience jitter when you hear a short silence and then hearing the person talking
faster than normal. That is the Teams client buffering packets and playing them back when enough packets have arrived. If there are more than 20 ms between packets
the audio healer will start dropping packet and the experience will be the same as packet loss.

Latency is the time it takes for a packet to travel from the sender to the receiver. When measuring latency, it is important put the result in context. A latency higher than
100 ms within the same country is not good. A latency of 150 ms across continents is very good. Latency is typically caused by distance, queuing, and buffer overflow on
the network. You experience latency when people you talk to on a call seem to take a long time to answer. A call with a lot of latency can be quite difficult to manage and
those on the call may end up talking at the same time.

Table 16-4 shows the target network metrics in an unmanaged network, which include all the locations the network administrator has no control over such as the
internet, home office or your favorite coffee shop. Table 16-5 shows the target network metrics in a managed network, typically inside corporate offices. Meeting these
network metrics is important to meet user’s expectations of good voice quality and dependent stability of calls. We look at managed networks in two scenarios, where
the users are and the corporate network edge. The goal is to know when it is an internet problem and when it is a local network problem.

Unmanaged Network Voice Optimal Acceptable Poor

Inter arrival packet jitter (average) ≤ 5ms ≤ 10ms > 10ms

Inter arrival packet jitter (maximum) ≤ 40ms ≤ 80ms > 80ms

Packet loss rate (average) ≤ 1.0% ≤ 5.0% > 5.0%

Network latency one-way < 100ms ≤ 100ms > 100ms

Table 16-4: Unmanaged network targets

Managed Network Voice Teams client to Microsoft network edge Corporate edge to Microsoft network edge

Latency (one-way) <50 ms <30 ms

Latency (round-trip) <100 ms <60 ms

Burst packet loss <10% during any 200 ms interval <1%

Packet loss <1% during any 15s interval <0.1% during any 15s interval

Packet inter-arrival jitter <30ms during any 15s interval <15ms during any 15s interval

Packet reorder <0.05% out-of-order packets <0.01% out-of-order packets

Table 16-5: Networking requirements for Teams media in managed networks (source: Microsoft)

Some companies express concern about the firewall configuration for Teams. Microsoft publishes an RSS feed for the Office 365 URLs and IP addresses, including those
used by Teams. This information changes often. To stay abreast of changes, you can configure an RSS connector to fetch this feed and bring it into a suitable team.

Note: To learn more about networking requirements, check out this comprehensive article called networking requirements for Skype for Business Online , which
applies to Teams as well.

Planning for Optimal Media Path

By now, you know the following:

1. In 1-to-1 calls, media goes local between clients if possible

2. In conferences, media goes from clients to Teams services in the Azure and Office 365
3. Latency one way should be below 100 ms and latency can be caused by distance

When planning for the optimal media path you should strive to have local internet breakout in branch offices for two reasons. The first reason is that the direct path to
the conferencing servers should go from a local internet breakout. The second reason is if you have a global environment, users in different regions will suffer if you
have a centralized internet breakout because Microsoft uses GEO DNS for their services. The goal is always to get to the nearest Microsoft network edge as soon as
possible.

Note: Quality of Service tags are now available in the Teams and Skype for Business Online admin portal. These are only utilized in 1-to-1 calls. This means that you
can prioritize internal traffic, but as soon as it goes over the internet, no QoS tagging will be available. Read more about QoS tagging in Teams here .
Network Planner
You are now ready to map out and plan your network using the Network Planner which is part of MyAdvisor. You achieve two things by using the Network Planner:

You create an overview and a suggested bandwidth footprint per location with a geographical representation of your sites.
You get an exportable subnet file which you can import in to the Call Quality Dashboard and Call Analytics console instead of using manually created file.

The Network Planner can be found here , and it is recommended to sign in to the portal with your Office 365 account. You are then able to save your topology and revisit
it at a later stage and expand on it or export it for import into the Call Quality Dashboard or the Call Analytics console. Without the subnet information both dashboards
will be next to useless since you are not able to determine the difference between internal or external traffic. Figure 16-8 shows the geographical view you get of your
sites and which elements you need to think about when working with the Network Planner.

Sites and personas are the primary areas to complete. These areas need a lot of information from your environment, but if you sign into Office 365, you can revisit the
plan and tune the information. Personas are difficult to define, and it is strongly recommended to read the help section about personas which describes how to think
about usage of different modalities per person such as voice, video, desktop sharing and PSTN calling.

Figure 16-8: Network Planner and the geographical view of the sites

Network Assessment Tool

You can use the Skype for Business Network Assessment Tool, which also supports Teams, to measure the metrics you get from your location to the Teams services. This
tool should be run in two places, close to your network edge and from where your users are. By comparing the results, you can see if there are any differences in quality
between your network edge and where your users are, which may indicate bottlenecks in your network. Remember that a single result may not be a good enough
measurement to launch a network investigation. By running a lot of tests over time on different times of the day will show some trends within your network that may
give a good enough reason to do a more thorough investigation of internal network quality. The Network Assessment tool is a command line tool and can be downloaded
from the Microsoft Download Center. The tool runs an actual call for 17 seconds and it does not require any Skype for Business or Teams components to be installed.
The result will give some actual network metrics to analyze, here is an example:

PS C:\Program Files (x86)\Microsoft Skype for Business Network Assessment Tool> .\NetworkAssessmentTool.exe

Skype for Business - Network Assessment Tool

Initializing audio call.

***************

Starting new call

Iteration 1 / 1

Audio call started. Waiting for call to end...

Call should end shortly after configured duration of 17 s.

Call completed.

Packet loss rate: 0,00235571260306243

RTT latency: 30 ms

Packets sent: 849

Packets received: 847

Average jitter: 6,007759 ms

Packet reorder ratio: 0

**************

The output is written to: C:\Users\hanst\AppData\Local\Microsoft Skype for Business Network Assessment Tool\performance_results.tsv. As you see from the trace, zero
packets were lost, two-way latency was 29,5 ms and average jitter was 3,4 ms. This is well within the limits of good quality call over an unmanaged and managed
network when comparing the numbers to the data in Table 16-4 and Table 16-5 . Read the Usage.docx file in the download to configure the tool to run in multiple
iterations.
You can also measure number of hops to the same IP address as shown in the Network Assessment Tools to check that you have an optimal path over the internet to the
Microsoft datacenters. Using tracert in the example below gives us the following result:

C:\>tracert 13.107.8.20

Tracing route to 13.107.8.20 over a maximum of 30 hops

1 8 ms 8 ms 8 ms 192.168.10.1

2 16 ms 14 ms 19 ms 1.51-175-128.customer.lyse.net [51.175.128.1]

3 21 ms 18 ms 18 ms 111.81-166-123.customer.lyse.net [81.166.123.111]

4 13 ms 29 ms 18 ms microsoft.dix.dk [192.38.7.76]

5 24 ms 29 ms 18 ms tor3.ber30.msedge.net [104.44.80.170]

6 * * * Request timed out.

7 29 ms 18 ms 18 ms 13.107.8.20

At hop number five we entered the Microsoft network and we reached our destination after seven hops. This is a very good result and you should expect to see between
seven and twelve hops before reaching the Skype for Business service within the Microsoft network. More than twelve hops are not wrong, but if you see more that
twenty hops you should work with you ISP to get a more optimal path. More hops can introduce latency and jitter and may be suboptimal in terms of media path.

Call Analytics and role-based access control

Call Analytics is nothing new in SfBO and Teams. It has always been available to extract individual call information through PowerShell using the Get-CsUserSession
cmdlet. It was complex and difficult to get started with making it unpractical for most organizations. Now we have the Call Analytics website which can be accessed
from the Teams and Skype admin portal. Call Analytics is focused on single calls and meetings, providing a detailed analysis of the selected call or meeting for issue
resolution by the helpdesk.

You can look at four core areas:

Device shows you specific information around the capture and render device the user was using during this call and you can determine if it was a Skype for
Business optimized device.
System shows you system statistics such as OS and client patch level.
Connectivity tells you how the system was connected for example via wireless.
Network shows you networking statistics such as packet loss, jitter and delay.

Figure 16-9 shows an example of the report where we see a good call where network performance is acceptable. The report shows you real actionable statistics in each
category described above. You can use what you have learned in this chapter to analyze the metrics collected.

Figure 16-9: Call Analytics example of a conference

Analyzing the metrics, you can tell the user why the call failed, if it was because of packet loss on a Wi-Fi connection or if they used an unsupported audio device. You
can also use the data to investigate further trending on the network location or driver version the user was using. The report highlights poor calls for individuals and it
is only good for investigating individual calls. If you want to review further trends in your subnets you should use the Call Quality Dashboard. You can delegate access to
call analytics using the following Teams RBAC roles:

Teams Service Admin, can manage the Teams service without the need of Global Admin rights.
Teams Communications Administrator, can manage calling and meeting features within Teams.
Teams Communications Support Engineer, can troubleshoot communications issues with access to advanced tools.
Teams Communications Support Specialist, can troubleshoot communications issues within Teams by using basic tools.

As shown in Figure 16-10 , users assigned the Support Specialist role only have access to Call Analytics and cannot see the organizational configuration in the admin
center. The Support Engineer role has access to the advanced tab which shows you metrics as you may be used to from monitoring reports in Skype for Business server.
This role also has access to the debug tab where you can see IP addresses and port ranges used. This is a great way to delegate voice quality troubleshooting access to
tier 1 and tier 2 support users.

Figure 16-10: Example of limited access in the admin center found at https://admin.teams.microsoft.com

To determine the RBAC role that meets you need, review table 16-6 with the different roles and types of access in the Microsoft Teams and Skype for Business Admin
Center.

Table 16-6: Admin roles and their access level

Call Quality Dashboard

You should use the Call Quality Dashboard (CQD) when you want to look for trending quality information across your subnets. CQD is part of the Microsoft Teams &
Skype for Business Admin Center and you need to have Teams Service Administrator or Teams Communications Admin rights to access the dashboard.

It is important to note that CQD is all about trends to see how many calls you have in a scenario and the overall quality of those calls. It is there for capturing symptoms
and help you get a feel of the solution and where to focus your improvement efforts. You may find a location with a high amount of poor calls or it may be an unexpected
high amount of VPN traffic. When opening the dashboard for the first time you may feel that it does not give you much feedback. It is only when you upload your subnets
to the portal that it starts to become useful, because that is when you can differentiate on call quality per location. You will also see the difference between outside
traffic which is typically unmanaged networks and inside traffic which is typically your internal networks. When uploading subnets to the portal you should also make
sure you specify the IP range for VPN traffic as well by creating a region for that. Unexpected high VPN traffic should be investigated because it is not optimal since it is
preferred that media goes directly from the external client to the conference bridge.

To import subnets to CQD you need to fill out a CSV file and upload it in the portal using these steps:

1. The CSV file you are uploading must not have the headers shown below, only the actual data.
2. It is recommended to use the exported CSV files from the Network Planner, the format should be as described below:

1. The headers are as follows:

1. Network,NetworkName,NetworkRange,BuildingName,OwnershipType,BuildingType,BuildingOfficeType,City,ZipCode,Country,State,Region,InsideCorp,ExpressRoute

2. The data should be completed as shown below:

1. 192.168.1.0,USA/Seattle/SEATTLE-SEA-1,24,SEATTLE-SEA-1,Contoso,IT Termination,Engineering,Seattle,98001,US,WA,MSUS,1,0

3. To upload the file, go to the cogwheel in the CQD portal and choose tenant data upload.
4. Specify your file and upload the data, unless you want to have specific start and end data, just leave the default options.
5. When imported successfully you will see “Successfully uploaded file.”
6. It may take up to 24 hours before the reports reflect the new subnets.
7. The same csv file can be uploaded the Advanced Call Analytics dashboard as well.

When the CQD portal is updated, go to the Location-Enhanced reports found in the menu bar. Check the “Client : Client Wired ” section first and then “Client : Server
Wired .” This is the indication of how 1-to-1 calls and wired conferencing performs in your network and is a good starting point to and overview of the quality in your
network. “Client : Client Wi-Fi ” should always be checked last because it is the most difficult scenario to improve, and if no effort has been made to optimize for
throughput you may assume that this network is an unmanaged network.

The CQD is all about streams. A stream is a one-way direction of a media modality in a call. You can have multiple streams within a call, so when looking at number of
streams in a location the number is much higher than the number of calls. It is dependent on how many users were in the call and how many streams there where per
user. The streams are classified in three categories which are Good, Poor and Unclassified. You will also see the percentage of poor streams which is indicative of how
many of the total streams that were graded as poor. Unclassified streams are either too short calls to gather data or calls with federated parties that do not log data to
you tenant. Figure 16-11 shows a summary report from the CQD portal, showing the overall number of streams for any given month.
Figure 16-11: All Audio Streams default report in CQD

What classifies as a poor stream? There are five factors that makes a stream poor and the stream is classified as poor if one or all metrics are above the threshold shown
in Table 16-7 . Degradation average is when the network got worse during a call, if degradation average is above 1 you have an under-dimensioned or unstable network,
look for routers or switches that are underperforming or overloaded Wi-Fi. Ratio Concealed Audio is a technique used to compensate for dropped network packets.
Average concealed samples Ratio (%) is the percentage of packets that were concealed in a call. Network metrics that classifies as Poor Stream in CQD is higher than
what is generally considered a poor stream. This way you know that a poor stream really is poor.

Network Metrics Poor Stream

Packet Loss Rate > 10%

Round Trip Time > 500 ms

Jitter > 30 ms

Degradation Average > 1

Ratio Concealed Average > 0,07 (7%)

Table 16-7: Network metrics and poor streams

A great starting point to get started with CQD is to download the SOF CQD template and upload it under Detailed Reports in the dashboard. It will show you an overall
view of the number of streams for your environment and will drill down and show reports such as how users responded to the Rate my Call survey, what client versions
they are using, and devices used in your organization. To customize these reports, you click edit and the natural first customization in the query editor is adding Second
Building Name as a dimension. Then you will be able to sort on location as well as shown in Figure 16-12 . In this example, we add a Second Building Name to the
Devices – Microphone report in the SOF CQD template.

Figure 16-12: Amending the CQD template

Note: There are many factors to use with report customization, the SOF CQD template is a great starting point and can be downloaded here . It still applies to Teams.
If you want to learn more about customization, the SOF CQD training videos found on the Skype for Business YouTube channel are highly recommended still in a Teams
setting

Mail Flow
We have covered a lot of detail about call flow in this chapter. Mail flow is another important aspect of Office 365 because if email cannot flow in and out of user
mailboxes without problems, including being protected against malware, the usefulness of Office 365 is much degraded. Let’s go to Chapter 17 to talk about how email
flows into and out of Exchange Online.
Chapter 17: Mail Flow and Exchange Online Protection
Brian Reid

This chapter focuses on mail flow, or how you manage Office 365 to ensure that messages transit securely and reliably from senders to recipients. Managing mail flow covers much more than
ensuring the successful delivery of messages. It is also about keeping your environment safe from threats such as mass-mailings, malware, targeted phishing attacks, spoofing, and
(accidental) data loss.

The tasks discussed here are carried out through the Security and Compliance Center (SCC), the Exchange Admin Center (EAC) or by running PowerShell cmdlets. Many examples use
PowerShell because it is often easier and quicker to update a setting with PowerShell and the commands are much less prone to change than any of the GUI-based tools. In any case, we
should begin with a discussion about online protection.

Exchange Online Protection

Exchange Online Protection (EOP) is Microsoft’s cloud-based email filtering service. EOP protects all email exchanged between Office 365 and external sources. Protection is in place to
divert spam and malware and to guard against potential data loss. There are three configurations for EOP:

Office 365/Exchange Online (cloud-only) deployment. Although EOP is a separate feature, it is integral part of and tightly integrated into Exchange Online. If you have an Exchange
Online tenant, you automatically use EOP.
Standalone. Organizations that do not use Office 365 can route to EOP to use it as an email cleansing service. This can be an on-premises Exchange environment or another email
solution, hosted or on-premises.
Hybrid deployments . In this scenario, EOP protects traffic between the cloud and the on-premises servers. In a non-centralized mail flow approach, EOP can also protect on-premises
mailboxes like it does in a standalone deployment.

Geographical Routing. Exchange Online Protection runs in Microsoft's worldwide network of datacenters. If an outage occurs, a different datacenter automatically takes over operations
to ensure that email continues to flow. However, Office 365 performs redundancy and load-balancing within a datacenter region. This also answers government regulatory prescriptions
which might need data to remain within a certain geographical area, like for instance the EU data processing guidelines. As such, EOP running in EMEA datacenters processes messages for
tenants in EMEA, messages for US tenants by EOP running in a U.S. datacenter, and so on.

How Exchange Online Protection Processes Email

The filtering system in Exchange Online Protection consists of multiple layers and processes to handle both inbound and outbound messages. To understand how messages are processed,
let's have a look at the diagram shown in Figure 17-1 , which shows how Exchange Online Protection uses inbound mail filtering to protect Office 365 mailboxes:

Figure 17-1: Exchange Online message processing

1. The mail server of the sender looks up the domain MX record information which points to Exchange Online Protection. The MX record resolves to the datacenters in the primary region
to which the Office 365 tenant belongs.
2. The sending mail server tries to deliver the message to the MX endpoint retrieved in the earlier step. Like Exchange on-premises, Exchange Online uses Connectors to control how
messages are sent or received. To accept messages from the Internet, you do not need to configure anything. The default Connector (which is invisible to the administrator)
automatically accepts messages from unauthenticated senders. We discuss connectors in more detail later.
3. Before the message is delivered to the recipient(s), it goes through several layers of filtering:

1. Connection Filtering . The connection filter is the first layer of defence and checks several items such as the sender's reputation. An administrator can also define an IP Allow/Block list
to allow or block connections from specific servers.
2. Anti-Malware filter . This filter inspects the message for malware and viruses. If a message is deemed malicious, it is removed by default.
3. Policy Enforcement . The policy filter checks messages against configured mail flow rules (including the rules created by DLP policies or for supervisory review). If a message matches
a rule, the action(s) configured for that rule or policy are applied to the message.
4. Spam Filtering is where messages contents are checked against a myriad of items to determine whether the message is spam or not. If a message is considered spam, it can be deleted,
sent to the user's Junk Email folder, or quarantined within EOP.

4. If the tenant has Advanced Threat Protection licenses and the right policies are in place, messages will be redirected for scanning before moving on to the next step.
5. If the message was not dropped, deleted, or quarantined, it is delivered to the recipient(s).

Hybrid mail flow is handled slightly different. Messages that were received from the on-premises organization are marked as "internal" and will therefore bypass content filtering (anti-
spam). Other filters, such as the connection filter and policy filter still apply. Internal email does not bypass the ATP Safe Links feature.

Outbound messages are also scanned and evaluated:

1. Outbound messages first go through the Anti-malware filter where they are scanned for known viruses.
2. Next, messages make their way through the Policy Enforcement engine. Here, messages are evaluated against configured policies such as Mail Flow Rules, Email
Encryption, and DLP rules. If a message matches one of the rules, the action from that rule is applied to the message.
3. The Content Filtering process exists to decide if sent messages are spam. A variety of techniques is used, and if a message is likely to be spam, a few things can
happen:

1. The message is routed through EOP's High-Risk Delivery Pool . This is a pool of EOP servers that is solely used to send messages flagged as potentially being spam.
This means that if one of the IP addresses in the High-Risk pool is blacklisted, this will not affect regular mail flow.
2. Depending on the configuration, and the characteristics of the email, it can also be quarantined.

1. If a message is considered safe, it is routed to its destination through the regular outbound pool. Unlike the high-risk pool, these servers are only used to send
messages considered safe. By default, the message uses the built-in Outbound Connector. If the administrator creates a specific Outbound Connector, for example to
route all messages for a specific partner to a different IP address, the parameters of that Connector will be used to determine how to route the message to its
destination.

Queuing. Sometimes it is possible that the (on-premises) mail server to which Exchange Online Protection should deliver messages is (temporarily) unavailable. Queuing will only happen
for "transient errors" such as connection time-outs, refused connections or other 400-type SMTP errors. Any "hard failure", such as invalid recipients, authentication mismatch or failures
shown by an SMTP 500-type error code, will result in an NDR being generated instead. In hybrid mail flow from on-premises, the Hybrid Wizard changes the configuration of mail flow so
that some 500-type errors are downgraded to 400-type, errors.

Exchange Online Protection will automatically try to re-deliver queued messages approximately every 5 minutes until it is able to successfully deliver the message. If the message cannot be
delivered within 48 hours, the message expires and a non-delivery report (NDR) is generated. Reports on messages that have been queuing over 1 hour are available via the Security and
Compliance Center, along with configured alerts when a given threshold is reached for queued items. This alert is sent to the administrator’s alternative email address.
Configuring Mail Flow
As mentioned before, Exchange Online Protection uses connectors to control inbound and outbound mail flow. This is like how an on-premises Exchange server uses receive and send
connectors. The equivalent of a receive connector is called an inbound connector and a send connector is called an outbound connector . Of course, there are some other, subtle, differences
too. For instance, in an on-premises Exchange organization, you can specify which servers can use a specific connector while you cannot in Exchange Online. Connectors do more than just
enable mail flow with external organizations, they are also used to apply security settings (such as TLS and by-passing spam filtering). Most organizations that sign up for Office 365 do not
need custom connectors: by default, Office 365 already has the necessary (hidden) connectors in place to allow for messages to be sent to and received from the Internet.

Although the concept of inbound and outbound connectors still exists, the EAC does not explicitly show the type of the connector anymore. Instead, each connector now displays the direction
in which mail flows by adding "FROM" and "TO" descriptions to each connector. For instance, the inbound connector that is automatically created by the Hybrid Configuration Wizard now
shows "Your organization's email server" in the FROM-field and "Office 365" in the TO-field. However, if you want to view or change the connector's properties through PowerShell, you will
still use the various *-InboundConnector and *-OutboundConnector cmdlets.

Setting up Basic Mail Flow

Before Exchange Online Protection accepts messages for your domains, you must add and confirm your email domains in the tenant. This will ensure that these domains are made known as
accepted domains to Exchange Online. The process for adding domains to Office 365 is outlined in Chapter 4. After the domain(s) are added to the tenant, you must configure the domain's
MX records to point to EOP so that remote organizations know to where to route email.

Note: Some scenarios exist which do not need you to update your MX records immediately. For instance, this would be the case if you have deployed a hybrid configuration and chose the
centralized mail flow approach. Instead, messages will be routed through the on-premises organization before being delivered to recipients in Exchange Online. The same choice is also
available to customers who use EOP to protect on-premises mailboxes.

The information to setup the necessary DNS records for your domain(s) is automatically returned in the wizard while configuring your domain(s), along with other relevant DNS records, such
as Autodiscover or, if you have selected other services, records for Skype for Business. For mail flow purposes, only the MX and SPF records are initially relevant. As soon as you configure
extra protection features, like DKIM (explained later), added DNS records might be required.

Real world: If you do not add your custom domains to Office 365, Exchange Online will only accept messages for the default tenant service domain, which is usually
tenantname.onmicrosoft.com . This is not very useful if you want your tenant to process mail for all your domains, but it allows you to setup a test tenant to verify certain features without
having the need to register a new domain.

MX record
When you configure your domain to use Exchange Online Protection, the MX record for that domain points to a hostname in the form of domain-tld.mail.protection.outlook.com rather than an
IP address. The following example illustrates what the value of the MX record is for a domain that is configured for Exchange Online Protection:

[PS] C:\> Resolve-DnsName office365itpros.com -Type MX

Name Type TTL Section NameExchange Preference

---- ---- --- ------- ------------ ----------

office365itpros.com MX 3572 Answer office365itpros-com.mail.protection.outlook.com 0

When a sending server queries the MX record for the domain office365itpros.com, the A record office365itpros-com.mail.protection.outlook.com is returned. Next, the sending server will try
to translate that hostname into an IP address, for which it needs to perform an extra DNS query. Before responding to the query, Microsoft's servers perform an internal lookup to check the
region the tenant with that registered domain name belongs to. Once the region is known, the service responds with the IP addresses of the EOP systems within the same region of the
tenant.

With MX records it is important to ensure that they are correct. Microsoft often sees issues that result from incorrectly configured MX and other DNS records. For example, if your Office 365
tenant has existed for some time, it might be that your MX record is not pointing to the current correct endpoint. For more information, see Microsoft's best practice article .

Real world: When first adding a new domain to your Office 365 tenant, you are asked to configure various DNS records amongst which are the MX records for your domain. In a greenfield
deployment, it is probably OK to go ahead and configure the MX records to point to Exchange Online Protection (EOP). However, if you already have an on-premises organization which you
need to migrate to Office 365 first, reconfiguring the MX records now could break your current mail flow setup. Skipping the configuration of the MX record will cause the Office 365 Portal
to show a problem status for the domain. Do not let this confuse you and force you to configure the MX records regardless. Switching from your current solution to EOP should be done at
the right time. When exactly that is, depends on your migration approach etc. Typically, you reconfigure your MX records once your pilot for Exchange Online is complete or you leave it until
most of your users have been migrated to Office 365. The time for changing your MX record is when you have ensured your configuration is ready to accept messages for on-premises
recipients from EOP.

SPF record
The SPF record is part of the Sender Policy Framework (SPF) validation system which has been created to protect from the receipt of spoofed messages. The receiving server does this by
evaluating the purported sender domain’s SPF record. The receiving email system will use the information on the SPF record to determine whether the email was received from a messaging
system authorized to send emails from that domain.

Real world: SPF records are not the only way to fight off spoofing/phishing attacks. DKIM, DMARC and other built-in anti-spoofing features all work together to reduce the likelihood of
someone falling prey to the cunning ways of malicious attackers spoofing legitimate email addresses to lure people into wiring money to them or download a payload to take control of a
remote device. To protect your domain against being spoofed, you need to implement DMARC quarantine or reject policies as well as SPF and or DKIM.

When a message is received from an external system, the receiving system examines the IP address of the prior server in the mail flow chain. It then looks for an SPF record for the sender's
domain. Then, one of two things can happen:

1. If an SPF record is found, the IP address is verified against the values specified in the SPF record. Depending on how the record is configured, this will either generate a neutral result
or a "soft" or "hard failure".
2. If no record is found, the SPF lookup is considered a "soft failure."

The suggested SPF record for Exchange Online looks like the example below. The record specifies that only the servers specified in the SPF record for spf.protection.outlook.com can send
messages on behalf of this domain:

v=spf1 include:spf.protection.outlook.com –all

If another server, not listed on the SPF record for spf.protection.outlook.com , tries to deliver messages for this domain, a hard failure should be generated because "-all " parameter is
specified.

The syntax of the SPF record controls how a failure should be treated by the receiving system:

-all : the minus sign shows that any SPF failure should be considered a hard fail. Whether the hard fail should result in the message being rejected depends on how it is configured in the
receiving server's anti-spam solution.
~all : the tilde shows that any SPF failure should be treated as a soft fail, yet the message should be accepted.
?all : the question mark is used to indicate that the owner of the domain is neutral towards the use of SPF records. A failure will not generate a soft or hard fail, instead generate a
neutral result. The message should be accepted regardless.

If Exchange Online Protection is the only system that will send messages for your domain(s), the suggested SPF record will suffice. However, if other systems also send email for your
domain(s), you must manually tweak the suggested SPF record to include other systems too. For instance, if another system, mail.office365itpros.com, also sends messages on behalf of the
domain, the SPF record should be modified to look like the following:

v=spf1 include:spf.protection.outlook.com a:mail.office365itpros.com –all

In the scenario where an on-premises organization uses Exchange Online Protection to protect incoming and outbound email, you should also include the public NAT IP address of the on-
premises Exchange servers in the SPF record:

v=spf1 include:spf.protection.outlook.com a:mail.office365itpros.com –ip4:1.2.3.4 –all

 Note: Some SPF records, like the example shown above, use the include statement to point to another SPF record (for another domain). This SPF record might also contain the include
statement which then points to yet another SPF record that might also contains the include statement, and so on… Every time a hostname or pointer to another record is added to an SPF
record (such as the include statement), a new DNS lookup is triggered to try and resolve the hostname to an IP address. The total number of lookups must not exceed 10, as this will cause
the SPF check to fail.

Creating and updating SPF records is not a trivial task, especially because most email administrators do not need to do it very often. There are various tools on the Internet that can help. For
instance, this web site helps to generate the SPF record to configure in your external DNS and this web site can help you check you have not gone over the maximum of ten DNS lookups.

Real world: Although not all organizations check for a sender's domain SPF records, it is important that you configure the records for your domain to reduce the risk of encountering email
delivery issues elsewhere. Organizations do not always know what other mail systems send messages on behalf of their domain. For instance, it might be the case that the marketing
department periodically sends out messages through a third-party solution. In such case, you should change your domain's SPF record to include the hostnames or IP address of the systems
from where the third-party solution sends messages. If you cannot obtain that information, it is better to change the SPF record to not trigger a soft or hard fail. The best practice is to use a
different domain to protect your corporate SMTP namespace. For instance, marketing.domain.com. Make sure that the namespace uses a different public IP address and SPF record than
your corporate main SMTP namespace. That way, if the domain marketing.domain.com gets blacklisted, it does not affect email deliver for the main corporate namespace.

Managing Connectors
You can use the New-InboundConnector and New-OutboundConnector cmdlets to create connectors in Exchange Management Shell. However, the wizard in the EAC is much easier to work
with. You do not have to specify what type of connector you want to create (inbound/outbound). Instead, you specify the source and target systems and the wizard will automatically create
the right connector. To open the wizard, open the Exchange Admin Center and navigate to mail flow > connectors . From there, click the plus sign.

In this example, we will create a new connector which forces TLS on all incoming messages from a partner organization called "Office365ITPro". After specifying Partner Organization in the
From: field and Office365 in the To: field, the wizard will run through the following steps:

1. Give a name and description for the new connector. It is always a clever idea to have a descriptive name such as "Inbound from Office365ITPro". Setting a description will help other
administrators to understand what this connector is used for. In this step, you can select whether the connector should be enabled after completing the wizard. If you do not wish to use
the connector for now, make sure to deselect the option.
2. The next step allows you to specify how the remote organization is identified. You can do this by specifying either the sender's domain(s) or the IP addresses in the next step. Using the
sender's domain allows for more flexibility than using an IP address. For instance, mail delivery will not be interrupted in case the partner organization decides to change public IP
addresses or if they forgot to include some of the IP addresses they use.
3. Depending on the choice made in the previous step, here you specify either the domain names or IP addresses which identify the remote organization.
4. In this step, you specify added security settings. If you select Reject email messages if they aren't sent over TLS , you configure TLS to be needed. Additionally, you can say that
messages must originate from a specific IP address and/or you can specify the subject name of the certificate that is used by the remote organization.
5. Before creating the connector, you can review the configuration parameters and go back to make changes, if needed.

Note: Requiring TLS encryption for inbound messages often requires the involvement from the sender's organization. If you want to use the certificate's subject name, like in the above
example, the remote organization must provide you with that information. Note that if TLS in enforced and the configuration is wrong, mail flow using that connector will fail.

According to Microsoft, many reported mail flow problems are caused by incorrectly configured tenant connectors. To reduce the amount of problems due to misconfiguration, the wizard
tries to confirm that the connector is functional before creating it: An attempt is made to send a test message to a recipient you specify using the configuration parameters you selected in the
earlier steps. If the wizard can successfully connect and deliver the message to the remote environment, the test is considered successful.

The idea is that if the test fails, you can troubleshoot where the problem lies and adjust the configuration to avoid similar issues when the connector is in use. By double-clicking the message
or clicking the pencil, you can get more diagnostic information such as the SMTP headers which might help you understand why delivery of the message failed. Despite a failure, the wizard
does not stop you from moving ahead and creating the connector anyway. The reason for this is that an administrator might have good reason for configuring a connector ahead of time;
maybe before the target organization has adjusted their configuration to match the connector’s settings.

After the connector is created, the Exchange Admin Center provides you with a summary of the configuration. If you need to look at the advanced properties, you can run the Get-
InboundConnector or Get-OutboundConnector cmdlets:

[PS] C:\> Get-InboundConnector | Format-List Name,*tls*,restrict*

Name : Inbound from Office365itpros

RequireTls : True

TlsSenderCertificateName : *. office365itpros.com

RestrictDomainsToIPAddresses : False

RestrictDomainsToCertificate : True

Conditional Mail Routing

Conditional Mail Routing allows you to use mail flow rules (previously known as transport rules) to alter the default routing behavior for specific messages that match the conditions of a
transport rule. If you want to control the behavior for all outbound messages, it is better to create a custom connector instead.

Consider the following scenario: You are a large organization with two (or more) types of users. For instance, an educational institution with students on one hand, and staff and faculty on the
other. You use Office 365 for both types of users, but want email from staff and faculty to be routed to the internet through an on-premises appliance which offers a variety of features like
journaling, secure messaging etc. Even though these features also exist in Office 365, you previously already invested in the solution and you do not want to lose on those investments. If you
were to create a regular outbound connector, all messages – including those from students – would be routed through the external appliance. Instead, you decide to create a new connector,
only to be used by a transport rule which defines that outbound messages sent by mailboxes matching a specific value (“Staff") in one of the custom attributes should be routed through the
new connector.

This scenario is a good example of how conditional mail routing helped the organization to meet regulatory requirements relatively easily, without making extensive changes to other parts of
the configuration and leveraging prior investments. Although the conditional mail routing adds a layer of complexity to the entire solution by splitting the message routing logic, it can be
extremely valuable if carefully planned and used for the right purposes.

Real world: The most common use of conditional mail routing is to ensure that messages sent to specific users or applications are either encrypted with TLS or sent directly to a specific
system. The latter option requires the outbound connector to be configured with a smart host. As such, the MX records for the recipient's domain are ignored and message are delivered
directly to the specified smart host. This is also done to ensure that messages are sent within a specific region. Often the recipient's organization does not have the infrastructure to
dynamically (and geographically) route incoming messages directly to servers in the same region as the mailbox. In such case, the sender's organization can then use conditional mail
routing to accomplish this task for them.

Conditional Mail Routing, or criteria-based routing, relies on mail flow rules to check the conditions of a message. If a specific condition is met, then the mail flow rules will redirect the
message to the selected connector. Thus, before you can create a transport rule to redirect messages to a specific connector, you must first setup a connector and configure it for conditional
mail routing. To do this, you use the same wizard to create a custom outbound connector. However, before during the new connector configuration, you must enable Only when I have a
transport rule set up that redirects messages to this connector when asked When do you want to use this connector?

In the backend, Office 365 will set the value of the IsTransportRuleScoped property of the connector to True . This ensures that the connector is available to new mail flow rules. Using the
Get-OutboundConnector cmdlet, you can check the value for the property.

[PS] C:\> Get-OutboundConnector | Select Name, IsTransportRuleScoped

Name IsTransportRuleScoped

---- ---------- -----------

Outbound to Office365itpros False

As an alternative to the EAC, you can also use PowerShell to configure the property using the Set-OutboundConnector cmdlet:

[PS] C:\> Set-OutboundConnector "Outbound to Office365itpros"

–IsTransportRuleScoped $True

Name IsTransportRuleScoped
---- ---------------------

Outbound to Office365itpros True

In the next example, we will configure a mail flow rule to redirect all messages sent to "jessica@office365itpros.com" through the custom connector Outbound to Office365ITPros . We will
only configure the bare minimum to make this rule work. Other options for mail flow rules are discussed at length later. Start by opening the new transport rule wizard from the EAC: select
mail flow , navigate to rules and then click on the plus-sign to show the drop-down menu and select Create a new rule...

By default, the list of actions displayed on screen is limited and does not offer the option to redirect messages through a specific connector. To enable access to the more advanced options,
you must first select More options… . After clicking the link, the additional settings will be displayed.

1. Specify a descriptive name for the rule. This will ensure that you can easily find the rule in the list of rules.
2. Select a condition to which the message should apply. In the example, select The recipient is… If you haven't created a contact for the recipient, type it in manually in the check names
field and then hit Enter. For example, type jessica@office365itpros.com.
3. From the actions drop-down menu, select Redirect the message to… and then the following outbound connector .
4. Select Outbound to Office365ITPros from the list of available connectors.

If you now save the connector, it is enabled by default and immediately apply to any outbound message that is sent to jessica@office365itpros.com. Here, we used a relatively simple example
to trigger the rule. Using the full range of conditions available to mail flow rules, you can configure multiple conditions within a single rule which allows you to be extremely granular in what
messages will get rerouted.

Exchange Online Protection and SMTP devices or applications

Even in the digital world we live in, people still print many documents. Often, these documents are scanned again for digital archiving. For years now, scanners can email scanned messages.
Despite being able to send email messages, many of these devices do not support authentication and can only send plain, unauthenticated, SMTP messages. Scanned messages mostly go to
internal recipients anyway. However, the same is not always true for some line-of-business (LOB) applications that for instance automatically send a message to an external recipient when an
order is updated.

In an organization where your email servers are not directly connected to the Internet, this isn't much of a problem as you can scope your firewall to accept messages from specific device or
application IP addresses. However, it's not that simple with Exchange Online Protection. Because the endpoint is publicly available on the Internet, allowing unauthenticated SMTP would
open the door to spammers to use EOP as an "open relay". Today, there are four possible scenarios:

If your device or application supports authentication , you configure it to use an Office 365 mailbox's credentials which will allow it to send messages internally and externally, to a
maximum of three concurrent connections. If you have more than three devices or applications connecting to the mailbox at the same time, then you will need more mailboxes.
The SMTP device or application sends the message directly to Office 365 using "plain" SMTP. The recipient should be a mailbox within your organization. Any external recipient will be
rejected.
A custom or 3 rd party SMTP Relay solution is used to forward messages to Office 365 'on behalf of' the SMTP device or application. For instance, the Windows SMTP component can be
used to create a custom SMTP relay solution . The SMTP service will accept unauthenticated messages from devices and applications within your network and in turn forward the
messages to Office 365 using the credentials of an Office 365 mailbox.

A lesser known option is to use a custom connector to identify and authenticate all on-premises devices and applications that need to send email messages, also to recipients outside
the organization. This option will most likely also require the network team to get involved to ensure that all outbound connections from these devices and application to Office 365
happen with a specific set of IP addresses that can be matched in the connector.

Note: When using an Exchange Online SMTP Relay, it is also a good idea to update the SPF record to include the on-premises IP address that are used to send messages to Office 365.
Otherwise the SPF lookup, which is performed by Exchange Online Protection, will fail and might cause the message to be delivered in the user's junk mail folder.

No Open Relays
To be able to receive email for organizations serviced by Office 365, MX records exist in DNS to tell other servers how to connect to Office 365 when people want to send email to those
organizations. EOP handles the inbound connections. You can even use old clients like Telnet to connect to EOP and send email. This sometimes raises the fear that Office 365 might act as an
open relay, a server that will accept email from anywhere and send it (or relay the messages) to anywhere else. For instance, to send a message to a user in the Office365itpros.com domain,
you can use Telnet to:

C:\> telnet office365itpros-com.mail.protection.outlook.com 25

220 AM5EUR02FT010.mail.protection.outlook.com Microsoft ESMTP MAIL Service ready at ...

helo

250 AM5EUR02FT010.mail.protection.outlook.com [80.246.206.65]

mail from: <anyone@anydomain.com>

250 2.1.0 Sender OK

rcpt to: <kim.akers@Office365itpros.com>

250 2.1.5 Recipient OK

data

354 Start mail input; end with <CRLF>.<CRLF>

From: "Anyone" <anyone@anydomain.com>

To: "Kim Akers" <kim.akers@Office365itpros.com>

Subject: Test email

Test…

Note that there is a blank line between the end of the headers and the start of the message body.

In this case, EOP accepts a message from a bogus address. However, the sender must provide a real recipient email address that exists in the domain. If not, EOP will fail to confirm the
address against the tenant directory and refuse to accept the message. When everything is complete, EOP accepts the message as normal. Is this a problem? Office 365 is not acting as an
open relay because the message can only go to a valid recipient. There is nothing strange here as the IP address for the workstation sending the message is not listed for a connector and the
domain exists inside Office 365. EOP treats this as a message from an external domain and treats the message on its merits. EOP applies its suite of anti-malware tests to the message to
check that it is safe. Depending on the outcome, the message might reach the recipient’s mailbox. This is not an open relay. It is just accepting messages, which is what EOP does all day long
for tenants.

In a blog post published in 2016 , Microsoft outlines the requirements needed to relay through Exchange Online Protection for hybrid customers. It is a requirement that either the email is
sent from a registered domain or a certificate containing one of your registered domains is used on the connector. Without this information Exchange Online cannot determine the legitimacy
of an email that attempts to relay through the service and it will get rejected with error “550 5.7.64 Relay Access Denied ATTR36”

Advanced Mail Flow Scenarios

Sharing namespaces
Many organizations use one or more unique SMTP domain names (for example, office365itpros.com). Those organizations sometimes need to share a common SMTP domain name across
multiple environments to present a single domain name to the outside world.

Sharing a SMTP namespace with another Office 365 tenant is not possible because you cannot register a domain name in more than one tenant. As such, it is technically impossible to share a
common email domain without some elaborate scheme. One solution is to create a routing domain for other Office 365 tenants (that cannot register the domain name), and continuously
synchronize recipient information across all connected systems. This allows messages to be forwarded from the primary tenant to their destination – albeit based on a different routing
address. It becomes a lot more complicated once you need to send messages from a single domain name across multiple Office 365 tenants. Because this is not possible within the default
capabilities of Office 365, it is hard to configure, and not recommended. We are not going to discuss this scenario in more detail. However, we will consider how you can share a namespace
across an Office 365 tenant and one, or more, on-premises environments.

Sharing a domain name across an on-premises organization and Office 365 is a lot easier than sharing a namespace across multiple Office 365 tenants. There are several ways you can do
this:

Use a third-party, or custom, broker-service that "catches" all incoming emails and route those messages to the backend system hosting the recipient mailboxes. For the broker
service to know to what host to direct a message, you must synchronize the recipient information from all connected systems to a specific location where the broker service can access it;
sometimes this is a directory of the broker service itself. An example of such a service capable of handling routing for various backend systems is Mimecast. The downside of this
approach is that it introduces another component in the mail flow process, therefore increasing the complexity of your solution.
Reconfigure domains in Exchange from Authoritative to Internal Relay . By default, a domain is registered as an Authoritative domain in Exchange Online when your tenant has
at least one Exchange Online mailbox licence in place. This means that Exchange Online will accept and handle mail flow for the domain, if a recipient exists in the tenant directory. If a
 recipient's email address does not exist, Exchange Online generates a NDR. When you reconfigure a domain as an Internal Relay domain, Exchange Online first tries to deliver the
message locally. If no matching recipient is found, the message can be forwarded to another mail system through a connector. If no connector is found to match the address, Exchange
performs an MX lookup to decide how to route the message. Therefore, you should first configure a connector before reconfiguring a domain as an Internal Relay domain. In this
scenario, you can have the MX record point to Exchange Online, after which messages for the on-premises recipients are forwarded to the mail server specified in the Connector. This
approach works well if you have a single Office 365 tenant and an on-premises environment.
Setup routing domains and configure each environment to forward messages for specific recipients using the configured forwarding address (based on the specific routing domain).
This is the approach used in a Hybrid deployment and discussed later in this chapter.

With multiple environments, it can quickly become tricky to manage this solution: the first environment (Office 365) must forward messages to the second environment, after which the
second environments forwards messages to the third, and so on. The last environment to receive messages should then reject unknown recipients. If you do not reject unknown recipients and
forward the emails back to any of the previous environments, you will introduce mail loops. Mail loops occur when messages go to recipients that does not exist in any environment. Because
none of the environments can find the recipient information in their local directory, they keep forwarding the message to the next hop until the message expires or the hop count is exceeded.
For the hop count to be exceeded the received headers must not stripped off the message at any stage. Potentially, a message loop like this can take down a messaging infrastructure and
therefore having one environment that is authoritative for a namespace should exist at the end of the collection of different email environments.

Remote Domains and Forwarding

Any email domain that is not registered in your Office 365 tenant is considered a remote domain and can deliver email to any remote domain. However, an organization might want to limit
what messages get sent to remote organizations, or maybe they want to control the message format. This is where the Remote Domains options come in handy.

Unlike in an on-premises Exchange 2013 environment, where you can only use PowerShell to configure remote domains, the EAC allows you to add and update remote domains via the GUI.
Add a new remote domain by clicking the plus sign under mail flow and remote domains .

A remote domain allows you to control:

Out of Office replies . Allow or deny replies to be sent to the domain. You can also control whether the internal or external message is sent.
Automatic replies and forwarding . You can control if a user-generated rule can be used to automatically reply or forward messages to the remote domain.
Message reporting . This allows you to control whether message status notifications should be sent to the remote domain. For instance, you can prevent non-delivery reports from
being sent to a remote domain or you can configure that meeting forward notifications are sent; by default, meeting forward notifications are only sent within the organization.
Format of the message . Allows you to define if a message should be sent in rich-text and what MIME character set should be used; if at all.

Note: The settings you specify for a remote domain overrides any user-specific settings, for instance configured through the Set-MailboxAutoReplyConfiguration cmdlet. If you want to
modify default settings for all other unlisted domains, you should modify the default remote domain.

The default behavior in Exchange Online is to allow forwarding of emails to external domains. This is not the case in Exchange Server. Care must be taken when moving from Exchange
Server to Exchange Online not to allow a forwarding scenario that you previously did not need to be concerned with.

Over the last year there has been an uptick in hacking scenarios where users with weak passwords have found that their account has been logged into, email forwarding enabled, and
therefore a copy of all their emails have been extracted from the organization. Disabling remote domain forwarding is a brute force way to protect your domain, though options like adding
mail flow rules to control forwarding to anything other than a pre-authorized whitelist might be preferable.

Transport Layer Security (TLS)

Requirements for TLS are almost always specified by the receiving server. Exceptions to this rule can exists, for instance when an on-premises server is configured to require TLS on a
specific Send Connector. By default, EOP will always try to negotiate TLS with the receiving server. This is called opportunistic TLS because, when TLS cannot be used, the sending server
(EOP in this case) will revert to using plain text. However, if the recipient's message systems support TLS, EOP uses TLS for the connection.

Messages destined for the on-premises Exchange environment in a hybrid deployment are, by default, always secured with TLS. The same is true for messages sent to domains protected by
Exchange Online Protection. If you often communicate with a business partner or if you send highly confidential information, you might want to enforce specific security settings, such as
needing TLS encryption. There are several ways in which TLS encryption can be enforced. For instance, you can require the certificate which is used to encrypt the traffic to have a specific
subject name. Alternatively, you can use the remote system's IP address to enforce TLS. Or you can do both. If you want to configure specific TLS settings for a remote organization, you must
create a custom connector.

Figure 17-2 shows the TLS configuration section during the creation of a new Outbound Connector. In the example below, TLS is needed for all messages routed through this connector, but
without needing a specific domain name or certificate to be used. If you need the domain to be authenticated as well, you should acquire the certificate to be from a trusted CA and specify
the hostnames on that certificate. This information is typically obtained through the partner organization.

Figure 17-2: Configuring TLS on a new Outbound Connector

Note: Although it is possible to enforce specific settings, such as requiring TLS, based on the IP address of the remote host, it is in your best interest to stay away from doing so as much as
you can. As a best practice going forward, you should rely on the use of certificates to enforce TLS whenever possible.

From the end of October 2018, Office 365 will not support connections over TLS versions 1.0 and 1.1. They will only support version 1.2 and later. Microsoft originally planned to block these
protocols from connecting in March 2018 and then delayed the implementation of the decision to allow customers extra time to switchover in the knowledge that Microsoft plans to remove
support for these protocols in the future. When that happens, Microsoft could turn off TLS 1.0 or 1.1 with the expectation that you have already upgraded your clients, servers and proxies
etc. to support TLS 1.2 (at least). It is therefore recommended that all clients and servers in your control where TLS 1.2 is not enabled by default (Windows Server 2008 R2 and Windows 7
clients typically) should be updated to support TLS 1.2 to avoid outages in when Office 365 ceases support for the earlier protocols. Note that Microsoft does not require you to disable TLS
1.0 and TLS 1.1 immediately. However, Microsoft support will not deal with any connection issues experienced with these protocols and you will be asked to retry the connection using TLS
1.2 in order to gain support. If devices support TLS 1.2, Office 365 will use that version even if you also allow earlier versions of TLS.

Figure 17-3: Viewing TLS statistics from the Mail Flow Insights report

The Reports Dashboard in the Security and Compliance Center is the place to find your TLS reports for SMTP. Select Reports, and then pick the TLS Report in the dashboard. The report
shows seven days of TLS metrics for the tenant and shows the percentage of connections made over the different TLS protocols to and from your mail flow connectors. For example, when I
checked my tenant I could see that TLS 1.0 protects 6.75% of my inbound mail flow. The rest of the connections is shared between TLS 1.2 (81.75%) and no encryption (11.5%). All my
outbound mail flow though is TLS protected (100%). Microsoft have said that they plan to retire this report and replace it with Mail Flow Insights.

The Mail Flow Insights report began being visible to tenants in August 2018. An example of an insight that is reports is shown in Figure 17-3 . This report shows that the on-premises
infrastructure should be updated to support TLS 1.2 before Microsoft retires support for TLS 1.1 (and earlier connections) at the end of October 2018.

Mail Flow Insights

Mail Flow Insights (accessed through the Mail Flow section of the Security and Compliance Center) are a series of reports and recommendations, some with associated alerts, that first
appeared in Office 365 in mid-2018 (an early version was called the Mail Flow dashboard). The TLS reports are included in the set along with very useful reports on message queues in the
service and a forwarding report.

Management of message queues has long been a role of the on-premises Exchange Administrator, but when you delegate server administration to Microsoft in Office 365, you lose visibility
into the server functions. Mail Flow Insights returns this data to the administrator by rolling up the queues across thousands of servers to let you know when you might have an issue to deal
with. Figure 17-4 shows the Mail Flow Insights dashboard with some of the controls displaying alerts for the administrator to investigate. As with most other Office 365 dashboards, you can
customize the arrangement of the widgets to suit your tenant’s needs.

Figure 17-4: Mail Flow Insights in the Security and Compliance Center

Mail Flow
The top widget (Inbound and outbound mail flow) displays two donut charts showing the measure of outbound and inbound mail flow, with a green indicator and the text showing the
percentage of messages protected by TLS. Although the numbers look like hyperlinks, they cannot be drilled down into, but the View Details link takes you to a pop-out that shows a
breakdown of the numbers for TLS and a link to the Connector Report .

The Connector report takes you to a dedicated screen showing mail flow by connector or TLS status. Clicking the All Mail Flow drop-down allows you to choose to data for all mail traffic or
to view inbound or outbound mail flow, both for connector-controlled message delivery and for delivery where no connector was utilized. The same report allows you to view TLS Usage for
selected connectors, meaning that you can drill down into the data as necessary. For this, you need to click Mail Flow to see TLS Usage, and once that choice is selected, select the connector
of interest.

Message Queues
The second default widget displayed on the dashboard is the Queues widget. The Queues widget shows you the number of messages stuck in an Office 365 mail flow connector for more than
one hour. If the number shown is non-zero, you can click it to go through a series of flyout reports showing information such as the number of queued messages and the connector name
(linked through to Exchange Control Panel where you can fix the connector in error). The number of queued messages can also be clicked through to show a message trace report for the
items stuck in the queue and further information on what might be the problem (in other words, why messages are queued) and where to go to fix it. As an example, a queue that contains the
wrong smarthost value might show the information in Figure 17-5 when drilled down from the queue report. The detail in the popup in the screenshot shows that the target host on the
internet refused to allow an SMTP (TCP 25) connection, otherwise known as a socket error.

Figure 17-5: Message trace information for a queued email

Forwarded Messages
The third widget on the dashboard is Auto-Forwarded Messages . This report is designed to help you combat a current trend in data exfiltration. By default, an Office 365 tenant allows
email to be forwarded outside of Exchange Online. This means though that an attacker can log into the account of a user with a guessable or common password with OWA and set up some
mail forwarding rules. Often this happens during a Business Email Compromise (BEC) attack, when the attacker wishes to understand the mail traffic patterns of a victim before they send
phishing email to try and lure someone into making a payment to them. It is recommended that you limit this behaviour so that email is only forwarded outside the organization when justified
by business reasons. The widget reports the number of forwarded items in the past week and the Alerts widget below reports within the hour when a new forwarding rule is put in place. I
used this alert notification to protect the mailbox of the CEO of one of my clients as he insisted on a password he could easily remember and refused to allow the enablement of MFA on his
account. Within the hour of his account being hacked, the IT team where aware and had remediated the issue. An example of the sort of alert they saw can be seen in Figure 17-4 above.

The widget shows the number of emails forwarded outside the tenant. If you click on the number, you see the method used to forward them (inbox rules, mail flow rules or SMTP forwarding)
and the top five forwarding users and domains. Reviewing this information and blocking or removing the rules that are forwarding email inappropriately along with implementing harder to
guess passwords and MFA for high-profile accounts that might be targeted by attackers should be a regular task of the Exchange Online or Security Administrator.

It is also worth noting that you can create alert policies (see Chapter 21) to signal an alert when a user forwards email from their account. Also, Microsoft Secure Score (Chapter 4) gives
tenants that do not allow mail forwarding a higher score.

Other Widgets
The final widgets show the mail flow reports that you can find on the Reports dashboard of the Security and Compliance Center and a Recommended for you widget. For example, if you
have a complex mail flow rule that is delaying mail flow, a note to that effect might be surfaced here. A misconfiguration that is generating a message loop might also be something that you
would see here.

Transport Limits
The Radicati Group Email Statistics Report for 2018-2022 estimates that over 3.8 billion people used email in 2018, with the number expected to grow to over 4.2 billion email users by the
end of 2022. In total, these accounts send and receive roughly 281 billion messages per day. With so many people using email, it is no wonder that email is by far the most popular medium to
send unsolicited messages (spam). To protect its service from those who would like to abuse Office 365 by using it to send spam, and to ensure that every tenant is treated equally, Microsoft
imposes several transport-related limits on its service.

Without limits a single user could, in theory, send hundreds of thousands of messages per day. As a result, the transport subsystem would consume enormous resources to process those
messages, and so reduce its capacity to process legitimate email. One way to solve this problem is to deploy more hardware to increase the capacity of messages that can be processed.
However, that is a solution that cannot be sustained over time. First, you cannot just throw hardware at a problem every time it occurs as it would be too costly. Secondly, it would only bring
relief for a very brief time as spammers would see the newly available resources as an opportunity to send even more messages. The transport limits are imposed at various levels and are the
same for all tenants, regardless of the plan you buy from Microsoft. Table 17-1 lists the various limits imposed in Exchange Online for E3-E5 plans. See this page for the full list of limits
imposed by Office 365 for other plans.

Feature Limit Additional Information

Recipient rate limit 10,000 recipients per

day

Recipient limit 500 recipients The number of recipients you can add to a single message.

Message rate limit 30 messages per The number of messages the transport subsystem will process per minute; sometimes also referred to as the throttling threshold;
minute over SMTP client submission

Message size limit (Outlook) 150 MB The maximum size of a message when sent through Outlook

Message size limit (OWA) 112 MB The maximum size of a message when sent through Outlook Web App

Message size limit (Outlook for 150 MB The maximum size of a message when sent through Outlook for Mac
Mac)

File attachments limit 250 attachments The maximum amount of attachments that can be added to a single message

Subject length limit 255 characters

File attachment size limit (Outlook) 150 MB The maximum size of a single attachment when sent through Outlook

File attachment size limit (OWA) 35 MB The maximum size of a single attachment when sent through Outlook

File attachment size limit (Outlook 150 MB The maximum size of a single attachment when sent through Outlook for Mac
for Mac)

Table 17-1: Transport limits imposed by Office 365

In general, Microsoft does not grant exceptions to any of these limits. Although you can raise a support request to increase the limits, the chances of succeeding are slim. Instead, many of
these requests can be worked around. A good example is the 10,000 recipients per day limit. It is usual for larger organizations to broadcast corporate communication messages to all its
users. If you have more than 10,000 users, you will not be able to send the message to everyone at once. At least that is the theory. Instead of adding individual recipients to a message, you
should use distribution groups. Each distribution group can hold up to 100,000 recipients and is counted only as a single recipient.

Real world : The actual message size can vary from one client to another and is also different for messages routed outside of Office 365. In the latter scenario, messages could grow roughly
33% in size because of extra encoding, effectively lowering the maximum size from 150MB to approximately 112 MB.

The above limits also used to be much lower when Office 365 first launched. Over time, Microsoft gradually increased the maximum supported message sizes, but they did not change the
default maximum message size. As such, even a new tenant imposes a 35 MB message size limit for every user by default. An excellent article outlines how to determine what a user's
current configured limit is and how you can modify that limit to a value within the boundaries of the supported maximum sizes described in Table 17-1

Another example is an organization in which the marketing departments needs to send many messages to its customers. At a rate of 30 messages per minute, 'only' 43,200 messages can be
sent per day. If that number does not fit your needs, you should consider deploying a hybrid connection and keeping an Exchange server on-premises as that server is not subject to the same
limits, provided it does not send outbound messages through Exchange Online Protection. Alternatively, third party applications such as MailChimp are built for this purpose, as are SMTP
sending applications such as SendGrid. Many of the above limits relate to the user or the user's client. Microsoft also imposes some backend limits on administrative operations to prevent the
clogging up the transport subsystem. Table 17-2 lists some of the various backend limits.

Feature Limit Additional Information

Journal Rules 10 The maximum number of journal rules that can be created in an Office 365 tenant

Mail Flow Rules 300 The maximum number of mail flow rules that can be created in an Office 365 tenant

Scanning limits for 1MB The transport rule conditions enable you to examine the content of message attachments, but only the first 1 MB of the text extracted from an attachment is
content of inspected. This 1 MB limit refers to the text extracted from the attachment, not to the file size of the attachment. For example, a 2 MB file may contain less than 1
attachments MB of text, so all of the text would be inspected

Table 17-2: Backend transport limits

Starting in June 2018, a new limit comes into play with the SMTP Authenticated Submission limit. This is typically used for third party email clients and devices such as multi-function
printers. After June 1 st 2018 the mailbox the client or device authenticates as will allow three concurrent connections as well as storing the items it sends in the Send Items folder.

Note: Limits and service descriptions are constantly changing. Microsoft does a pretty good job of documenting these limits for each of the different Office 365 plans. Whether you are
considering moving to Office 365 or are managing an Office 365 deployment, it is worth revisiting these limits from time to time to make sure nothing has changed.

Mail Flow Rules

A modern messaging system must be able to handle the various requirements an organization might have. It is no longer enough to just can send messages from point A to point B. As
organizations evolve, its messaging system must be able to offer the right feature set to support the changing requirements of that organization.

One could compare the changing behavior of interacting with a messaging system to how people interact with the post office. Where before it was good enough to just deliver a written letter
to its recipient, today the post office offers many additional services. For instance, when you move to a new address, you can request the post office to automatically forward messages that
were sent to your old address, to your new address. This not only gives you the time to inform all users about the address change, but more importantly, it prevents mail from being returned
to sender or worse: delivered at the wrong address.

The general idea behind mail flow rules (also known as transport rules) is very like the above example. It provides Exchange (and Exchange Online) with a way to dynamically handle
incoming or outgoing messages based on one or more criteria. As we will see during this topic, you can use mail flow rules for much more than forwarding messages to a different email
address. Mail flow rules also support the Data Loss Prevention feature (Chapter 22), and by the Supervisory Rules feature (Chapter 19).

Because mail flow rules are created to support Data Loss Prevention and Supervisory Rules policies, it is possible that a rule is inserted into the pipeline in a position (as set by the priority
assigned to the rule) where it interferes with other rules. For this reason, it is important that administrators check the set of mail flow rules that are configured for a tenant on a regular basis
to ensure that all the rules operate in the intended manner and have the correct priority.

The way mail flow rules work in Office 365 is very like how they operate in on-premises Exchange, with the main exception that mail flow rules in Office 365 have more conditions and actions
to choose from. In Office 365, an organization can create up to 300 different mail flow rules which should warrant enough flexibility for even the most demanding organizations.

Mail Flow Rule Conditions

Conditions (also known as predicates) are used to define when a transport rule should be triggered. For instance, a condition can be used to check for a specific value in the sender's email
address or it can look for the existence of an attachment. The easiest way to build a new transport rule is to use the Exchange admin center (EAC), as it allows you to select a condition from a
drop-down list, as illustrated in Figure 17-6 :

Figure 17-6: Selecting a condition for a new transport rule in the EAC

You can also use PowerShell to create a new transport rule. Microsoft publishes the list of conditions and exceptions supported by mail flow rules that also state the PowerShell name for each
condition. Unlike most other PowerShell cmdlets, each condition is its own parameter to the New-TransportRule or Set-TransportRule cmdlet. For instance, to create transport rule that will
trigger when a message is received from a specific email address, you would use the following PowerShell command:

[PS] C:\> New-TransportRule –Name RandomRule –From:email@domain.com

–Quarantine $True

Name State Mode Priority Comments

---- ----- ---- -------- --------

RandomRule Enabled Enforce 0

A rule can contain multiple conditions, or a condition can contain multiple values. If multiple values are specified, the message must match one of the values assigned to the condition ("OR").
In case multiple conditions are specified, the message must match all individual conditions before the action in the transport rule is triggered ("AND"). In the following example, we create a
transport rule which contains multiple conditions: the rule will only be applied if the message was received from the specified email address and sent to the specified recipient within the
organization:

[PS] C:\> New-TransportRule –Name RandomRule2 –From:email@domain.com

–SentTo tredmond@office365itpros.com -Quarantine $True

Name State Mode Priority Comments

---- ----- ---- -------- --------

RandomRule2 Enabled Enforce 2

If a single rule needs to fire for different values of the same conditions, then add multiple values to a single condition. In PowerShell you do this by separating each value by a comma. The
example below illustrates how to create a rule that will fire for emails from the specified sender to either of the specified recipients.

[PS] C:\> New-TransportRule –Name RandomRule3 –From:email@domain.com

–SentTo "tredmond@office365itprs.com","michael@office365itpros.com" –Quarantine $True

Name State Mode Priority Comments

---- --- -- ---- -------- --------

RandomRule3 Enabled Enforce 3

Note: It is also possible to create a "catch-all" rule. If you do not specify a condition (or you select [Apply to all messages] ), the transport rule will be applied to each message that flows
through the organization.
Mail Flow Rule Exceptions
Exceptions are, just like conditions, used to scope the applicability of the transport rule. Exceptions are used in conjunction with 'regular' conditions used to exclude matches against (one of)
the primary condition(s). Each condition has a matching exception. In PowerShell, this means that you can simply append ExceptIf to the name of the condition. For instance, if a condition is
to match all email addresses from a specific domain, an exception can be used to exclude a single, or multiple, email addresses from that domain:

[PS] C:\> New-TransportRule -Name "testrule" -SenderDomainIs "office365itpros.com"

-ExceptIfFrom "tredmond@office365itpros.com" -Quarantine $True

Name State Mode Priority Comments

---- ----- ---- -------- --------

testrule Enabled Enforce 1

Note: Any predicate can be used if messages are unencrypted when they are processed. Encrypted messages, such as S/MIME encrypted messages or rights-management protected
message cannot be processed by mail flow rules based on conditions that inspect the contents of the message. The only exception to this rule is in the case the organization's Exchange
Online Azure Information Protection subscription is used to encrypt the message and thus Exchange Online has the key to decrypt the contents. In any other case, mail flow rules with
conditions based on a message's envelope header will still process correctly.

Mail Flow Rule Actions

Actions ultimately carry out a task on the message. Each action affects the message in a unique way; either by changing some of the message's properties or altering the routing behavior of
the message. Amongst other actions, you can for instance re-route, reject, mark as spam, or silently redirect a message. Microsoft publishes the list of actions , including the PowerShell name
for each action. For instance, in the earlier examples, the action defined to move a message to the quarantine if the transport rule applied:

[PS] C:\> New-TransportRule -Name "testrule" -SenderDomainIs "office365itpros.com"

-ExceptIfFrom "tredmond@office365itpros.com" -Quarantine $True

Name State Mode Priority Comments

---- ---- - ---- -------- --------

testrule Enabled Enforce 1

A transport rule can include multiple actions. Each rule is assigned a priority order and the rules are processed in ascending order of priority This means that if you want multiple actions to
be executed on a message but defined through a separate rule, you must plan the rules so that the desired actions are executed in the right order, considering the possibility that a rule might
cause all further processing to stop after it complete because it includes the Stop processing more rules option. Rules also stop processing if a rule's action is to drop (delete) a message as
the next rule won't have a message to process! Something similar happens if the action is set to moderate the message (sent for approval): later rules will not be processed until the message
is approved after which the other rules are evaluated against the message.

Note: Over the years since the introduction of Exchange Online Protection, the options for spam and malware filtering have expanded. This means there are less reasons to use mail flow
rules for spam/malware filtering. For example, the malware filter includes attachment protection settings, thus avoiding the need to look for attachments by file extension name in a mail
flow rule.

Creating a New Mail Flow Rule

As mentioned earlier, the easiest way to create a new transport rule is to use the wizard in the EAC as it allows you to select conditions, exceptions and actions from a drop-down list, rather
than having to specify the corresponding value from one of the long lists of options. The EAC also allows you to select a pre-canned transport rule from a list of templates. These templates
cover most common scenarios, such as applying rights protection to a message or allowing a specific message to bypass spam filtering. Depending on what template you selected, the
conditions, exceptions and actions are already pre-selected. You only need to specify unique value(s) for any of the specified properties. For instance, if you select the Apply rights
protection to messages… template, the Apply rights protection to the message with… action has already been pre-selected, and you then select which RMS template you want to apply
to the message.

The EAC can also be used to build your own rule from scratch. By default, if you select Create a new rule , you will only be shown a subset of conditions and actions. However, by clicking
the More options link on the page, you are presented with the full list of conditions, exceptions and actions that are available in the service.

Testing New Mail Flow Rules

Figure 17-7: Selecting message properties to include in the incident report

Before you put a new transport rule into production, it is wise to verify that the rule does what you expect it to. Just like DLP policies, you can verify whether a transport rule works by
enabling it in test mode. You will have the following options to choose from when creating a new transport rule:

Enforce (default).
Test with policy tips.
Test without policy tips.
These settings are affect how Data Loss Prevention (DLP) rules work, but we can make use of one DLP feature to implement a test mode for any mail flow rule. The Enforce option will
trigger the rule if all the conditions of the rule are met. However, Test without policy tips will stop the rule action from triggering as the rule is in Audit mode. But if the rule does not fire,
how do you know if it has worked? The answer to this is to add the Generate incident report and send it to… action, as shown in Figure 17-7 .

The Generate incident report and send it to action sends an email to the selected recipient with information about the message that caused the rule to trigger. In Audit mode (with Test
without policy tips enabled), Exchange Online generates and sends the incident report email, but the other actions specified in the rule, such as setting a disclaimer as shown in Figure 17-7 ,
are not performed. The incident report action allows you to select the properties of the message you want to include in the incident report. If you are dealing with sensitive information, or
local legislation prevents you to access the contents of a message without notifying the user, you can opt to not include the original mail, and make sure that the incident report only reveals
non-crucial information about the message, An example of an incident report email is illustrated in Figure 17-8 , where we see that the disclaimer rule “Disclaimer: Berlin Office” was
triggered, but as the rule was in audit mode, the actual disclaimer was not applied and therefore not seen by the recipient.

Figure 17-8: An example of an incident report

Incident Reports are a straightforward way to check the effectiveness of a transport rule and, at the same time, gather information about what kind of messages will trigger the rule. This will
allow you to decide whether the transport rule meets your expectations. Once you decide that the transport rule is ready for production, you can edit the transport rule and choose to enforce
it. Don't forget to remove the action to send an incident report, or you will continue to receive a report each time the rule is triggered!

Monitoring Mail Flow Rule Usage

The effectiveness of a transport rule can be measured by the amount of times it was triggered. Of course, there might be cases where you hope a transport rule is not triggered at all, for
instance when a transport rule is created to create an ethical wall to prevent two departments from communicating with one another. This is something often seen in financial institutions
where regulations define that market researchers are not allowed to communicate with brokers to avoid a conflict of interest or (in-)voluntary market manipulation.

The Office 365 reporting center has a section dedicated to mail flow rules. The reports can be accessed from the Office 365 main dashboard, by clicking Reports > Security and
Compliance > Rules and navigating to the Rules section . Today, there are two reports available:

Top rule matches for mail this report is a rollup of the second report and gives a summary of the rules that were triggered the most within the specified time frame.
Rule matches for mail allows you to review the activity per transport rule, and scope down the results of the various elements of the rule such as the severity level or the direction of
the message that triggered the rule.

Both reports can be useful to check the usage of a transport rule. But sometimes you need more than that. For instance, you might be looking for a specific event or a specific violation.
Neither the reports, nor the EAC allow you to do that. Luckily, there's PowerShell to save the day!

The Get-MailDetailTransportRuleReport cmdlet can be used to search for specific transport-rule related events. For instance, it can be used to search for 'hits' within a given timeframe or to
look for actions that have been performed because a transport rule was executed. The example shown below reports all transport rule hits in the past 5 days and list the date and time,
message subject, action taken, and rule that was executed:

[PS] C:\> Get-MailDetailTransportRuleReport –StartDate (Get-Date).AddDays(-5)

–EndDate (Get-Date) | Format-List Date, Subject, Action, *Rule*

If you need to search for hits of a specific transport rule, you can use the following command:

[PS] C:\> Get-MailDetailTransportRuleReport –TransportRule "Check for sensitive data"

The cmdlet has many more parameters that allow you to further refine the results. For instance, you can look for messages from a specific sender:

[PS] C:\> Get-MailDetailTransportRuleReport –StartDate (Get-Date).AddDays(-5)

–EndDate (Get-Date) –Sender Joe@office365itpros.com

The reports in the Office 365 dashboard and the PowerShell cmdlet all use Microsoft's Reporting Web Service. This service, in turn, gets its data from Office 365's reporting warehouse. The
only downside to this approach is that the information in the warehouse isn't always up to date. From experience, it can take up to a day before these cmdlets return complete data. If you are
looking for an immediate way to measure mail flow rules, the first method (adding an incident report) is the preferred way.

Note: When you create a new transport rule, you can assign a severity to the transport rule. The severity level will not have an impact on how the transport rule works, but it is a great way
to differentiate sets of mail flow rules from one another in reports. The Rule matches for mail report can filter based on the severity of a transport rule. That way, you can create specific
reports that only show information about mail flow rules with the same severity rating. If you do not want a transport rule to show up in a report, simply clear the checkbox next to Audit this
rule with severity level .

Common Use Cases for Mail Flow Rules

Because of the many conditions that are available, mail flow rules can be used in a myriad of scenarios. It would be impossible to list all the use cases here, but we will look at some of the
more common scenarios and tasks. The examples from the below scenarios should provide you with enough insights into the capabilities of mail flow rules so that you can come up with
possible solutions to any unique requirements you have.

Organization-wide Disclaimers
Sometimes legal or regulatory requirements dictate an organization to add a disclaimer to outgoing messages. Mail flow rules offer an adequate solution to the problem. One of the actions
available to a transport rule is the ability to prepend or append a disclaimer to a message. The content of the message can be plain text, but it can also include HTML code to make it more
dynamic.

Additionally, you can include certain variables, based on a user's AD attributes, so that user-specific values are included in the disclaimer. For instance, by adding %%DisplayName%% to the
disclaimer text, the display name of the user who sent the message will be injected into the text.

Note: One important limitation of mail flow rules is that the total size of the disclaimer cannot exceed 5,000 characters. This includes the text itself and any HTML tags or Cascading Style
Sheet (CSS) code you might add!

You can configure a transport rule to add a disclaimer either through the EAC or PowerShell. The latter option is a little daunting as the syntax of the PowerShell command necessary to add
the disclaimer can be complex, especially if you use HTML code:

[PS] C:\> New-TransportRule –Name "CompanyDisclaimer" –Enabled $True

–SentToScope "NotInOrganization" –ApplyHtmlDisclaimerLocation "Append"

-ApplyHtmlDisclaimerText "<table border="0"><tr><td>Column1</td> <td>Column2</td></tr><tr><td colspan='2'><div style='font-size:10pt; font-family:verdana'>This is the disclaimer text and cannot exceed 5,000
characters. The HTML code does not make any sense, it is just to illustrate how quickly it can become messy.</div></td></tr></table>"

Figure 17-9 shows that configuring a transport rule with the same HTML code through the EAC is considerably easier compared to the above PowerShell code:
Figure 17-9: Configuring a disclaimer through the EAC

Automatically Adding a Signature to All Outbound Messages

Even though you might be tempted to use mail flow rules to automatically add a signature to outgoing messages, the reality is that the user experience might be below what you expect.
When you opt to append a disclaimer text (or signature) to a message, the text will be added at the end of the message. Of course, this is exactly where you would expect it to be for a new
message. However, if you are replying to an email thread that has been going on for a while, you will notice that the disclaimer text (or signature) is added at the very bottom of the entire
message, not in-line with the latest reply. This might be perfectly acceptable for a disclaimer, but it looks strange for signatures as you can see from Figure 17-10 :

Figure 17-10: Signatures added by a transport rule are added to the bottom of the message stream

The signature for the first message (1) is inserted at the bottom of the message, right where one would expect it. However, the signature to the second reply (2) was also added at the bottom.
Now, imagine an email thread with 20 replies. As you can imagine from the example above, accumulating signatures or disclaimers at the bottom of the message does not look very
professional. Although you cannot make a transport rule add text at the end of the latest message, you can stop the addition of a disclaimer text or signature if one already exists by adding an
exception to the rule, as illustrated in the following PowerShell example:

[PS] C:\> New-TransportRule –Name "CompanyDisclaimer" –Enabled $True

–SentToScope "NotInOrganization" –ApplyHtmlDisclaimerLocation "Append"

-ApplyHtmlDisclaimerText "<table border="0"> <tr><td>Column1</td> <td>Column2</td></tr><tr><td colspan="2"><div style="font-size:10pt; font-family:'verdana'">This is the disclaimer text and cannot exceed 5,000
characters. The HTLM code does not make any sense, it is just to illustrate how quickly it can become messy.</div></td></tr></table>"

–ExceptIfSubjectOrBodyContainsWords "This is the disclaimer text and cannot exceed"

The first time the message is processed, the rule will fire and add the disclaimer because it's very unlikely that the phrase mentioned in the exception is found in the body or subject.
However, any subsequent time the message is processed, for example after a response is received to the original message and a further reply is sent, the exception will prevent the rule from
adding the disclaimer because the body now contains the exact words that were added by the same rule the first time the message was processed.

Solving a Reply-all Mail Storm

Who hasn't come across the scenario that starts with someone (accidentally) sending a message to a large group of recipients, maybe members of a distribution group, and then someone hits
"reply-all" to respond to the message, after which another person does the same thing? It would not be the first time that an organization's messaging environment has been brought to its
knees because of this. In Exchange Online, having enough server capacity available to process messages is the least of your worries; that is something for Microsoft to worry about. However,
a mail storm can also greatly disrupt the productivity of your employees and it is better to break the proverbial 'mail storm' before it can cause any real damage.

MailTips can be a subtle way to warn people that they are about to send a message to a large audience. However, it does not prevent users from clicking the send button. Mail flow rules can
bring immediate relief, after the fact. The solution might not be airtight, but the idea is as simple as it is effective: as soon as you detect a mail storm, you can create a new transport rule to
'drop' (delete) all messages that match a specific condition. Often the condition of the rule will be based on the message subject or specific words in the subject. After you have enabled the
transport rule, you must either communicate that people should not "reply-to-all" to the message or you can run a script to programmatically remove the message from each user's mailbox.
The following example creates a transport rule that will drop messages of which the subject contains specific words:

[PS] C:\> New-TransportRule –Name "BreakMailStorm" –SubjectContainsWords "message subject"

–DeleteMessage $True

Real world : Using mail flow rules to suppress a mail storm is not an airtight solution. You cannot use the exact message subject because email clients such as Outlook prepend "RE:" to
each reply. Of course, you can create a rule which includes the additional text, but this does not cover international replies. For instance, replies from a German client are likely to prepend
"AW:" instead of "RE:". Even though chances are slim, by not using the exact message subject you open the door for the transport rule to also delete legitimate messages which have nothing
to do with the mail that caused the storm. The best remedy is, of course, to prevent mail storms from happening by educating your end users. But it's better to have a solution for the
problem, rather than having to hope it will never happen.

The use of email moderation is also good for stopping a reply all storm. Enable moderation on the mailbox that users are doing the Reply-All to and have the request for moderation go to a
mailbox that is not really used. For this scenario, you do not plan to approve any of the message that the moderator has been requested to approve. This will quiet down the reply all storm as
well. Indeed, it is often a good idea that large distribution groups such as “Entire Company” and the like are restricted to having just a few authorized users send to them or being moderated
by a few authorized users. Anyone can email the group, but not everyone’s email will be approved for delivery to the recipients.

[PS] C:\> Set-DistributionGroup "Entire Company" -ModerationEnabled $true -ModeratedBy "HR1@office365itpros.com,HR2@office365itpros.com" -BypassModerationFromSendersOrMembers ceo@office365itpros.com

The above PowerShell cmdlet enabled moderation on the named distribution group, has two mailboxes (HR1 and HR2) as the moderators, unless the email to the group is send by the CEO, as
they bypass the moderation.

Bypass Spam Filtering

As we will discuss later in the Message Hygiene topic, Exchange Online Protection automatically collects safe- and blocked sender settings from the junk mail settings manipulated by users
with Outlook and Outlook Web App so that this information can be used to examine incoming messages and help to mark them as safe or spam. Sometimes, however, an organization might
want to centrally control what senders are automatically approved or marked as spam. For instance, this would be the case if an organization introduces a new corporate communications
tools which sends messages from an external system. This is often the case for HR-related tools which typically sends out important messages that you don't want to end up in a user's junk
email folder.

To achieve this in Exchange Online, we can use a mail flow rule to add a message header set to a specific value. In this case, the header is the "SCL" header (Spam Confidence Level) which
will be set to a value of “-1”. The value of the header determines if a message should be sent to the user's junk email folder at the end of the processing pipeline. In Exchange Online. Table
17-3 describes the values used for the SCL header:

SCL Meaning
Value

-1 The message is deemed safe because the sender is safe-listed, the sender's server is on an IP safe list or the recipient is on the safe-recipient list. The message is not processed by
the content filtering engine.

0, 1 After processing by the content filtering engine, the message was deemed to be clean.

5, 6 After processing by the content filtering engine, there is reasonable confidence to mark the message as spam.

9 After processing by the content filtering engine, there is no doubt (high confidence) that the message is spam.

Table 17-3: SCL header values in Exchange Online

In the following PowerShell example, a new transport rule is added to create a safe list that will automatically mark messages from the office365itpros.com and tailspintoys.com domains as
safe:

[PS] C:\> New-TransportRule –Name DomainSafeList –SenderDomainIs "office365itpros.com","tailspintoys.com" –SetSCL "-1"

To update an existing transport rule, for instance to add the "office365itpros.com" domain to the list, use the following PowerShell command:

[PS] C:\> $domains = (Get-TransportRule "DomainSafeList").SenderDomainIs

[PS] C:\> $domains.add("office365itpros.com")

[PS] C:\> Set-TransportRule "DomainSafeList" –SenderDomainIs $domains

Similarly, if you want to remove the office365itpros.com domain from the list, use the following PowerShell command:

[PS] C:\> $domains = (Get-TransportRule "DomainSafeList").SenderDomainIs

[PS] C:\> $domains.Remove("office365itpros.com")

[PS] C:\> Set-TransportRule "DomainSafeList" –SenderDomainIs $domains

Real world : Care though should be taken with setting blanket safe list rules like those shown above. If the domains or senders in question have their email address spoofed, so that it is not
coming from the user or organization who owns the mailbox but from someone else, then the email will still be marked as safe as the rule just reviews the domain or user and not the
reliability of the sender. The anti-phishing rules (part of ATP until September 2018 and then generally available to all tenants) are recommended as a replacement for blanket safe lists.

Exporting and Importing Mail Flow Rules

One of the widely-adopted ways to test new application features or configuration settings before implementing them into production is to use a lab environment that mirrors the production
environment. This is no different for an environment in which Office 365 is used. One way is to do this is to sign up for a second tenant. You can then enable that second tenant for Targeted
Release (previously known as First Release) to ensure that it picks up new features before regular tenants.

Although mail flow rules can be enabled in test mode as explained before, it can sometimes come in handy to experiment with a few settings before creating the rule in production. Instead of
manually recreating the rule in the production tenant, you can export the rule(s) from the test tenant and then import them into the production tenant. To export mail flow rules from a
tenant, run the following code while connected to Exchange Online PowerShell in the test tenant:

[PS] Set-Content C:\ExportTransportRules.xml –Value ((Export-TransportRuleCollection).FileData

–Encoding Byte)

Next, connect to the production tenant and run the following code:

[PS] C:\> Import-TransportRuleCollection -FileData (Get-Content -Path C:\ExportTransportRules.xml -Encoding byte -ReadCount 0)

Confirm

Importing a rule collection will overwrite all existing rules in that collection. Do you want to continue?

[Y] Yes [A] Yes to All [N] No [L] No to All [?] Help (default is "Y"): Y

As you can see, running the Import-TransportRuleCollection cmdlet overwrites existing mail flow rules. This means that if you have made changes to mail flow rules in the production tenant
that you have not made in the test tenant, you should re-do those changes afterwards.

Real world: Because of the limitation of the Import-TransportRuleCollection cmdlet in that it overwrites existing mail flow rules, the combination of the export and import cmdlets are not
used very often. If you can guarantee that the test tenant and the production tenant are kept on par in terms of configuration though, they are excellent at testing and duplicating settings.

However, the Export-TransportRuleCollection cmdlet is the perfect tool to backup mail flow rules to an xml file and keep it for later use. For instance, if someone made an undocumented
change to a transport rule, the XML file can be used to track the change and even revert to the previous version of the transport rule(s), if needed. Note that you cannot use the *-
TransportRuleCollection cmdlets to backup rules on-premises and import them into Exchange Online.

Message Hygiene
EOP has a set of message hygiene features to sanitize the inbound and outbound mail stream and remove threats from messages. Some of the features are well known and visible to end
users, others like DKIM and DMARC are hidden in the background. This section gives an overview of the different message hygiene features used by Exchange Online and how they can be
managed. The default settings of the features are configured so that they meet the requirements of 90% of the tenants in Office 365.

Over the last few years Microsoft have been added functionality to the message hygiene features, but not always been adding this functionality so that it is visible in the Exchange Control
Panel. Most of the options described in this section are available through the Security and Compliance Center with those options that have been around for a while longer in Exchange Online
still visible in EAC. Any difference is in the user interface, which follows the new Admin look and feel rather than the layout used in EAC. Over time, it is Microsoft’s direction to move any
Office 365 functionality related to security to the Security and Compliance Center. Given the importance of email to business communications, message hygiene obviously falls into the
security category. It is also likely that future enhancements to existing message hygiene capabilities and new features will appear in the Security and Compliance Center rather than in the
EAC.
Anti-Spam and Anti-Malware Protection
As defined in Wikipedia, email spam is unsolicited, undesired, or illegal email messages. It should come to no surprise that the various anti-spam features in Exchange Online Protection work
together to:

1. Reduce the amount of spam that is delivered to a user's inbox; and

2. Reduce the number of false-positives; messages that are marked as spam, but in fact are not.

When an organization signs up for Office 365, spam filtering for inbound messages is enabled by default; and it cannot be disabled. However, an administrator has a multitude of options to
tweak the settings for the different anti-spam features.

If outbound messages are routed through Exchange Online Protection, which is the default for each Office 365 tenant, they are also checked for spam. Unlike the policy for inbound
messages, the outbound filtering policy cannot be changed apart from changing some of the alerting options. To protect Exchange Online Protection's reputation, Microsoft routes outbound
messages with a high spam confidence level through a separate pool of servers. This ensures that the 'normal' outbound pool is not affected if one of Exchange Online Protection's servers
from the high-risk outbound pool is blacklisted. In case a customer (or one of the users) continues to send spam through Office 365, the outbound anti-spam protection will eventually block
the user from sending messages.

To stay on top of situations where an internal recipient is sending spam messages, an administrator can change the default outbound policy to generate notifications each time when a
message is deemed suspicious or if a user is blocked from sending message. In the Security and Compliance Center (SCC), go to Threat management > Policy > Anti-spam. From there
you expand the Outbound spam filter policy. You will find this under any custom policies that have been created, as this policy is always enabled. Figure 17-11 shows the available options to
configure an outbound spam filter policy. There isn’t much to configure, except for whom to notify in case of specific events and a mailbox to send suspicious outbound emails to.

Figure 17-11: Editing the outbound spam filtering policy settings

Anti-Spam Message Headers

As described in Table 17-3 , Exchange Online Protection updates the SCL header to show whether EOP considers a message as spam. The SCL header consists of a single digit which does not
give much information to understand why a message is considered spam. To provide administrators with more information, Exchange Online Protection also inserts two extra headers into
each message. These headers are detailed at https://technet.microsoft.com/en-us/library/dn205071(v=exchg.150).aspx and listed below:

X-Forefront-Antispam-Report which is used to provide information on the anti-spam processing of the message; and
X-Microsoft-Antispam which provides information specific about bulk mail- and phishing results.

The X-Forefront-Antispam-Report header consists of many different values, each revealing more information about the message, where it was sent from and what the result of individual anti-
spam tests are. The following shows what the header typically looks like (where this header comes from an email from Twitter):

X-Forefront-Antispam-Report: CIP:199.59.150.72;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(8156002)(31570200002)(2980300002)(438002)(286005)(199004)(189003)(64016003)(53416004)(2616005)(19627405001)(110436001)(956004)
(476003)(126002)(733005)(18926415007)(118246002)(85226003)(53386004)(606006)(336012)(6486002)(58536013)(106002)(106466001)(2160300002)(36756003)(246002)(16586007)(36736006)(486006)(33656002)(356003)(8676002)
(7696005)(26005)(1096003)(620700001)(59450400001)(966005)(551544002)(33964004)(7636002)(84326002)(6306002)(25786009)(53946003)(236005)(4290100001)(146002)(7596002)(16003)(6916009)(270700001)
(579004);DIR:INB;SFP:;SCL:1;SRVR:VI1PR0301MB2318;H:spruce-goose-ac.twitter.com;FPR:;SPF:Pass;LANG:en;PTR:spruce-goose-ac.twitter.com;A:0;MX:1;

At first sight, it may look hard to make some sense of the information in the header. If you take a closer look, you will see that the header in fact consists of several different fields of which
the following give more insight into the anti-spam decision making process:

CIP: has the IP address of the server that delivered the message to EOP. This IP address should be specified when using the IP Allow or Block list, and the IP address should have been
listed on the senders, SPF record.
IPV: this field has two possible values and is used to decide whether the connecting IP was found on an IP Allow list or not.

IPV:CAL : the message passed anti-spam filtering because the connecting IP address was found on an IP Allow list.
IPV:NLI : the IP address was not found on any IP reputation list.

CTRY: specifies the country from where the message was received. To determine the country, EOP uses the connecting IP address which may, or may not, be the same country as from
where the message was originally sent.
SCL: gives the Spam Confidence Level of the message and denotes the likelihood of the message being spam. The higher the number, the more likely the message is spam.
SFV: this field specifies why a message was marked as spam or not as spam, and has several values:

SFV:SFE : the filtering was skipped and the message was let through because it was sent from an address on an individual’s safe sender list.
SFV:BLK : the filtering was skipped and the message was blocked because it was sent from an address on an individual’s blocked sender list.
SFV:SPM: the message was marked as spam by the content filter.
SFV:SKS: the message was marked by spam before the content filter processed the message. This includes the case when a transport rule is used to mark a message as spam.
SFV:SKA : the message skipped filtering and was delivered to the inbox because it matched an allow list in the spam filter policy, such as the Sender allow list.
SFV:SKB : the message was marked as spam because it matched a block list in the spam filter policy, such as the Sender block list.
SFV:SKN : the message was marked as not spam before the content filter processed the message. This includes the case when a transport rule is used to mark a message as safe.
SFV:SKI : the content filter skipped processing the message, for instance because it was received by the on-premises environment and thus marked as internal .
SFV:SKQ : the message was released from quarantine and was sent to the intended recipients
SFV:NSPM : the message was marked as not spam and was sent to the intended recipients.

SPF : specifies information about the SPF lookup result for the message.
SRV:BULK : The message was marked as bulk email message.
H : The HELO or EHLO string of the connecting mail server.
PRT : The PTR record of the sending IP address (also known as the reverse DNS address).

The Microsoft Antispam Header looks like (from the same message used above):

UriScan:;BCL:1;PCL:0;RULEID:(7020095)(5600026)(4605076)(4608076)(1401150)(8001031)(1402068)(71702078);SRVR:VI1PR0301MB2318;

And this header shows the following components:

BCL : The Bulk Complaint Level of the message where the scale is 0-9, and if the value is 8 or 9 then the email comes from a sender that generates a high number of complaints.
PCL : is the Phishing Confidence Level of the message and indicates the likelihood of the message being a phishing message (spoofed message). This value is 0-3 for emails unlikely to be
phishing, and 4-8 or -9990 where the content is likely to be a phishing email.

Recently Microsoft have also increased the information displayed on the Authentication-Results header. This header records the results of checks against SPF, DKIM and DMARC. In the
above headers from an email that was received from Twitter, the following was the Authentication Results header:

spf=pass (sender IP is 199.59.150.72) smtp.mailfrom=bounce.twitter.com; office365itpros.com; dkim=pass (signature was verified) header.d=twitter.com;office365itpros.com; dmarc=pass action=none
header.from=twitter.com;compauth=pass reason=100

While SPF, DKIM, and DMARC are all useful by themselves, they don’t communicate enough authentication status in the event a message has no explicit authentication records. Therefore,
Microsoft has developed an algorithm that combines multiple signals into a single value called Composite Authentication, or compauth for short. Customers in Office 365 have compauth
values stamped into the Authentication-Results header in the message headers. A compauth value of 000 means the message failed DMARC policy with an action of reject or quarantine and a
compauth value of 001 the sender was not publishing and email authentication policy with SPF or DMARC, or if they do publish a policy it is very weak – that is it allows soft fail or neutral or
DMARC (p=none). Full details on these new headers and the reasons for authentication results can be found online in the Anti-spoofing protection in Office 365 article.

The Decision to Use Multiple Filtering Solutions

Companies often consider that they can gain added protection by deploying other mail filtering solutions in conjunction with Exchange Online Protection. Although from a technical
perspective this approach is practical, it could have adverse effects on some of the anti-spam features including IP throttling, IP block lists, bulk-mail filtering, or anti-spoofing. This is also
why Microsoft recommends against using an added filtering solution with EOP.

For instance, Exchange Online Protection relies on the use of IP throttling to prevent spam messages from being delivered: when a message is received from a new IP address, EOP throttles
the incoming connection by issuing an SMTP 450 error. The error indicates a transient error, which means the sending server should retry later. Legitimate servers usually retry the
connection while most spammers do not. The reason for not retrying is very simple: the added effort to interpret the error message and then try again later, is often too much trouble for
spammers. Even though the feature does not stop a spammer from sending messages, it greatly reduces the amount of spam delivered to Office 365. However, when a second filtering
solution is used in front of Exchange Online Protection, all messages delivered for your organization originate from a single set of IP addresses and renders the IP throttling feature
completely useless.

Therefore, when Microsoft detect that your MX record does not point to Exchange Online Protection, they automatically turn off some of the anti-spam features they could use. An example of
one they turn off is the connection filtering controls mentioned in the above paragraph, but they also cannot do IP reputation blocks, IP throttling, sender authentication checks such as SPF
and DMARC, spam filter rules, and more. It is imperative therefore that if you route email to a third-party filtering solution before Exchange Online Protection, and to ensure that that
solution you choose is doing at the very least all the same settings as Exchange Online Protection will do. For example, if you have a security device as your first hop (MX to security device)
which does not do anti-spam, then Microsoft will also not be doing all the anti-spam they could do as well.

It's important to understand that when a third-party filtering solution is used the message content hygiene features in Exchange Online are never bypassed. This means that all messages
delivered to your tenant by the third-party solution will then be rescanned and checked by EOPs various filtering mechanisms. If you add a third-party solution in front of EOP, Microsoft
recommends that you have the third-party solution mark each message with a specific X-header to show whether a message is spam or not, and then use Exchange Mail flow rules to look for
those headers and take appropriate action.

Real world: It is unsupported to use a third-party filtering solution in a hybrid deployment for messages sent between the on-premises organization and Exchange Online. This is because
Exchange's internal header firewall strips important message headers, like the X-MS-Exchange-Organization-AuthAs header from the message. When the headers are no longer present, all
sorts of problems such as messages not being recognized as originating from within the same organization can occur. The header firewall does not strip headers if the sending server is
authenticated using Exchange Server authentication. However, because the third-party filtering solution cannot perform that type of authentication, Exchange does not recognize the traffic
and strips away the headers. For this reason, messages sent or received between the on-premises organization and the hybrid counterpart in Office 365 must flow directly between Exchange
Online and the on-premises Exchange servers. If additional filtering is desired, or you have the requirement to terminate all inbound connections in the perimeter network, you must use an
Edge Transport server. The Edge Transport server is the only additional filtering solution which can be authenticated like an Exchange server in the same manner as the internal transport
servers. You can, of course, continue to use a third-party filtering solution to process all messages that enter and leave your organization to and from the Internet.

User-managed Safe and Blocked Sender Lists

Exchange and Exchange Online allow users to control certain junk email settings themselves. Within Outlook and Outlook Web App a user can do the following:

mark a sender as safe

mark a sender as blocked
add a recipient to a safe-recipient list

For instance, to block a sender from within Outlook, a user must right-click the message and then navigate to Junk . The process is slightly different in Outlook Web App. When a user right-
clicks a message and selects mark as Junk , the message is automatically moved to the junk email folder and the sender is also added to the blocked senders list. Once a sender is blocked, it
is added to the user's blocked senders list in the mailbox. The Junk Mail Assistant (a process which runs in Exchange) will update the Active Directory attributes listed below within 15
minutes of the change to the mailbox. All three user-maintained lists are stored in an attribute of the user's account in (Azure) Active Directory. More specifically, the information is stored in
the following attributes:

msExchSafeSenderHash
msExchSafeRecipientHash
msExchBlockedSenderHash

This information from these attributes is then accessed by Exchange Online Protection; messages from safe senders are automatically marked as 'not spam' and messages from blocked
senders will automatically be forwarded to the user's junk email folder or quarantine; whatever the administrator has configured. On-premises mailboxes protected by EOP also gain from this
feature if Directory Synchronization is enabled. In a hybrid deployment, these attributes are also written-back into the on-premises organization. This ensures that lists are maintained, and
the end user experience is no different if the mailbox is ever moved back to the on-premises organization. It is also important in the scenario of a hybrid deployment where the MX records
point to the on-premises organization instead of EOP. The write-back capability then ensures that on-premises Edge Transport servers also correctly process messages for cloud-based
mailboxes. Note that when an on-premises mailbox is protected by Exchange Online Protection (or a cloud-based mailbox by the on-premises Edge Transport servers), there could be a delay
of at least thirty minutes between a user adding an address to one of the lists and the information being available to Exchange Online Protection or the Edge Transport servers because of the
directory synchronization interval and the worker process in Exchange Online that reads the safe or blocks lists and writes them to the Exchange Online directory.

As the name of the attributes already implies, the address information is hashed before it is stored in the attribute. Hence, there is no sense trying to 'read' the attribute through using a
utility like ADSIEdit. Furthermore, Azure Active Directory does not even allow you to access the attributes directly. To access the information, a user can go to the Junk E-mail Options in
Outlook, where the user can:

view/edit the safe senders , safe recipients and blocked senders list.
adjust in-client junk email filtering through the general options tab.
block messages from specific countries, regions. For instance, if a user selects to block message from Canada (CA), all messages from a domain ending with .ca will be marked as spam.
block messages written in an unfamiliar language .

It is also possible via the general options tab to adjust the settings that the client will perform on junk email. This includes settings such as whether Outlook itself will do its own processing
of junk email (which should be disabled when the mailbox is in Exchange Online, otherwise you will find Outlook making decisions based on its older and less reliable junk email filter). You
can also configure Outlook to remove junk email rather than place it into the Junk Email folder.

Note: Only the safe senders and safe recipients as well as blocked sender lists are used by EOP. All other features are client-specific and will only affect messages after they have been
delivered to the user's mailbox. However, it should be noted that EOP also provides country- and language-based protection which can be configured separately in the connection filter policy.

To access the various user-maintained lists in Outlook Web App, click Settings (cog wheel) and then Options . This will bring up the Exchange Control Panel. From there, select Accounts >
Block or Allow (Figure 17-12 ).
Figure 17-12: Managing block and allow lists within Outlook Web App

Note: Outlook has separate tabs for Safe Senders and Safe Recipients, but Exchange Online only shows a single list called Safe Senders and Recipients. If you make changes in Outlook Web
Access options, the Safe Recipients list from Outlook will be merged with the Safe Senders list and that list will be copied into both Safe Senders and Safe Recipients in Outlook. Also, any
Safe Senders and Safe Recipients that is an internal domain will be removed automatically from these lists shortly after being added.

Junk Mail and Phish Reporting

Despite the anti-spam measures included in Office 365, sometimes spam or phish slips through cracks and into inboxes. Users can report these messages (also known as false negatives) from
their user quarantine by sending them to Microsoft for analysis. If Microsoft's Spam Analysis Team confirms that the message meets the spam classification criteria, they update the EOP
filtering systems to block similar messages in the future. Similarly, users can also report false positives. These are messages that EOP detected as spam but are not.

Tenant administrators do not need to do anything to allow users to report messages to Microsoft. Different options exist, depending on the client. For instance, the report message add-in is
available for Outlook 2013 and later that submits spam or phish to Microsoft in just a few clicks. This add-in replaces the older Junk email reporting add-in. Alternatively, the report message
add-in client plugin is available for Outlook 2013 and later as well as OWA that submits spam or phish to Microsoft in just a few clicks. This add-in replaces the older Junk email reporting
add-in that only worked in Outlook client for the PC. Alternatively, messages can be forwarded to junk@office365.microsoft.com, phish@office365.microsoft.com or
not_junk@office365.microsoft.com , as outlined in this article .

Message Forwarding and the Sender Rewrite Scheme (SRS)

Authentication of your email traffic with DMARC, DKIM, and SPF improves the reputation of your domain and brand by reducing spoofing and increasing the likelihood of detection of spoof
attacks against the domain.

However, some problems exist with these techniques. The primary issue is that SPF lists the IP addresses used for outbound email. If your messages are delivered to another service that
forwards them on again (using mail flow rules or inbox rules for example), then the SPF information published for your domain makes the forwarding service seem to be a generator of spoof
email.

The Sender Rewrite Scheme (SRS), enabled in Office 365 in July 2018, changes the P1 (or envelope from address) of auto-forwarded (or redirected) messages so that the receiving next hop
sees that the sender is the forwarding service and not the original sender, and so avoids an SPF failure. SRS does this by changing the P1 header from the original sender’s address and
domain to an address that encapsulates the original sender’s address but is really the forwarding service. Note though that this means that bounce emails to these newly rewritten addresses
will no longer return to the original sender, but to the forwarding service, which in this case is Office 365. Bounces will now all return to a mailbox called bounces@default-accepted-domain .
The P2, or the From address that the user sees in the email client, will not be changed. For example:

Before SRS is implemented, the P1 address is: brian@office365itprobook.com

After SRS is implemented and the initial receiving tenant forwards messages from the original sender, the P1 address might be something like this:

sales+SRS=f2ss=IX=office365itprobook.com=brian@itprobook.com

This is best broken out as follows:

<Forwarding Mailbox Username>+SRS=<Hash>=<Timestamp>=<Original Sender Domain>=<Original Sender Username>@<Forwarding Mailbox Domain>

Now the forwarded email comes from the forwarding tenant and not the original system, and so should not fail SPF. More details about the implementation of SRS are available on the
Exchange Blog .

Sending Messages to the Junk Email Folder in an On-premises Environment

By default, spam messages are delivered to a user's junk e-mail folder in Office 365. However, if EOP is used to protect on-premises mailboxes, added work is needed to ensure that the on-
premises Exchange servers also deliver messages marked as spam by EOP to the junk e-mail folder. Four new mail flow rules, each looking for a specific message header value, need to be
configured in the on-premises Exchange environment. When the transport system meets the header in a message, the transport rule sets the message's SCL header to the configured value.
To create the rules, execute these commands in the on-premises Exchange environment. These cmdlets will create the four rules and place them above all other rules that might already exist:

[PS] C:\> New-TransportRule "Move EOP Detected Spam (SFV:SPM) to Junk Email Folder"

–HeaderContainsMessageHeader "X-Forefront-Antispam-Report"

–HeaderContainsWords "SFV:SPM" –SetSCL 6 -Priority 0

[PS] C:\> New-TransportRule "Move EOP Detected Spam (SFV:SKS) to Junk Email Folder"

–HeaderContainsMessageHeader "X-Forefront-Antispam-Report"

–HeaderContainsWords "SFV:SKS" –SetSCL 6 -Priority 1

[PS] C:\> New-TransportRule "Move Personal Block List Spam (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SKB" -SetSCL 6 -Priority 2

[PS] C:\> New-TransportRule "Move Bulk Email (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SRV:BULK" -SetSCL 6 -Priority 3

If any of the transport rules are triggered, the SCL header of the message is set to a value of 6. This should trigger the on-premises Exchange server to move the message into the user's junk
e-mail folder. However, whether 6 is appropriate depends on the value of the global SCL threshold for the entire on-premises organization. By default, the value is set to "4". You can retrieve
the value from your on-premises organization by using the Get-OrganizationConfig PowerShell cmdlet:

[PS] C:\> Get-OrganizationConfig | Format-List SCLJunkThreshold

SCLJunkThreshold : 4
The value used in the transport rule should always be higher than the threshold. For instance, if the SCLJunkThreshold parameter is set to 6, the SetSCL value in the above mail flow rules
should be increased to 9.

Message Quarantine
By default, when a message is detected as spam, Exchange routes it to a user's junk email folder. High-confidence spam, phishing messages and bulk email can automatically be routed into
the separate quarantines that exist for each malicious email type. The quarantine is a vault where marked messages are held instead of delivering them to a user's mailbox.

To change the circumstances under which messages are quarantined, you can modify the anti-spam policy and change the specific spam and bulk actions. To access these settings, login to
the Security and Compliance Center and navigate to Threat management > Policy . The Anti-spam settings page has two widgets: Standard and Custom . By default, the tenant is
protected by a series of rules and policies as defined by the Standard Anti-spam settings . To make things simpler for tenants, the Security and Compliance Center groups the policies
together into the anti-spam settings. However, in the end, making changes to the policies through either the EAC or the Security and Compliance Center will have the same effect.

To change the quarantine settings, click on the Custom widget on the Anti-spam settings page, and then click on the switch next to Custom settings to add or change existing policies. Each
tenant has three policies that are enabled by default and cannot be disabled: a spam filter policy, a connection filter policy, and an outbound spam filter policy. To change the quarantine
settings for everyone, it is best to change the default policy. If you want to change the behavior for just a specific set of users, it's easier to create a new policy and scope it so that it only
applies to them. The same options exist when editing a policy or creating a new one.

Messages can be kept in the quarantine for up to 30 days after which they are permanently removed (the previous default value was 15, and so this might be the value set in your tenant). The
default setting is to not notify users when messages are quarantined because the default handling in EOP is to send spam emails to the Junk Email folder. Therefore, you should consider
configuring end-user spam notifications so that users get a digest email with a list of all quarantined messages if you change the default handling of email so that spam or high confidence
spam email is sent to the quarantine. To enable the spam notifications, click Configure end-user spam notifications from the action pane when looking at the Default spam filter policy
settings on the Anti-spam settings page in the Security and Compliance Center. The default duration for notifications is every 3 days, though this can be dropped to a single day as the
lowest setting.

Note: Modifying the default behavior might have an adverse effect on other features such as Zero-Hour Auto Purge (ZAP), explained later in this chapter. For example, ZAP will only work
when messages are set to be delivered to the Junk e-mail folder.

A fourth policy exists called the Spoof Intelligence Policy. Until September 2018, this policy required tenants to subscribe to Advanced Threat Protection, but Microsoft then decided to make
the feature available to all tenants. We cover this policy in more detail later.

Releasing Messages from Quarantine

There are several ways to release a message from the quarantine. The fastest, and easiest way, for a user to release a message from the quarantine is to click the Release to Inbox link next
to the message in the spam notification digest email as shown in Figure 17-13 :

Figure 17-13: The spam notification email

Alternatively, users can also login to the quarantine control panel and manage spam messages from there. To access the quarantine, you need to visit
https://protection.office.com/#/quarantine (which replaces https://admin.protection.outlook.com/quarantine as that will stop working in October 2018) and login with your Office 365
credentials. After logging in, the users will see a list of quarantined messages. From there, the users can:

Release the message.

Release the message(s) and report as false positive when editing a message.
Search through messages.

Users are also able to Preview the email . This allows them to make an informed decision on whether or not to release a message. The preview of an email does not execute any HTML code,
instead, the code is shown on screen as illustrated in the following example:

7/18/2015 9:51 AM

spammer@idontknowthisdomain.com

You can earn up to USD 250,000 per year!

To: brian@office365itpros.com

Cc:

Attachments:

<html><head></head><body><title></title>< base target="_blank"></base>< table style="background-color: #ffffff;" border="0" width="100%">< tbody>< tr>< td></td>< td style="width: 100%;">< div style="padding: 15px;
max-width: 600px; margin: 0px auto; display: block; text-align: left;" align="left">< table bgcolor="#FFFFFF">< tbody>< tr>< td><a href="http://somemarketinglink.dontknowthedomain.com/link.php?
M=1143933&N=101&L=26914&F=H">

The benefit from not executing the HTML code is that it removes the ability of the sender to track if the message was opened or not. It is a common technique to track which messages are
opened by including an 'invisible' image (a “web beacon”) in the email. If the HTML is rendered, the image is downloaded from a remote server and this permits the senders to track who has
read/opened an email and who hasn't. To access messages held in quarantine, the user must have a valid Office 365 account and should have an Exchange Online or Exchange Online
Protection license.

Administrators can also release messages from quarantine. Open the Security and Compliance Center and navigate to Threat management > Review > Quarantine or go direct using
https://protection.office.com/?hash=/quarantine which could be a useful URL to bookmark. Administrators see a rollup view of all messages that are currently quarantined (Figure 17-14 ).
The interface is very similar to the end-user interface and allows an administrator to perform the same actions with the difference that an administrator has more options to release a
message. For example, the administrator can view messages quarantined for different reasons such as the phish quarantine, or the spam quarantine. Messages can also be released to
different/additional recipients.
Figure 17-14: Accessing quarantined messages through the Security and Compliance Center

A user or an administrator can decide to release a message from quarantine which means that Exchange releases the message from quarantine and delivers it to the user.. If the
administrator wants to view quarantined messages for his own mailbox, select Only my messages option from the drop-down menu. As for the administrator the default view shows all
messages by default. The quarantine also filters the emails based on why they are in the quarantine, and these categories are transport rule, bulk, phish, malware and spam (shown above).
For files, the only category is malware.

In larger environments, it is very likely for the message quarantine to hold several hundreds, if not thousands, of messages. Having to work with many messages through the Security and
Compliance Center might prove challenging to an administrator, though changes in 2018 have improved this task. PowerShell also allows an administrator to perform the same actions as in
the GUI. For instance, using the Get-QuarantineMessage cmdlet, an administrator can look for specific messages in the quarantine as illustrated in the following example:

[PS] C:\> Get-QuarantineMessage -SenderAddress "jeff.guillet@office365itpros.com" |

Select Received Time, Type, Subject, Expires

ReceivedTime Type Subject Expires

------------ ---- ------- -------

4/1/2015 11:35:12 PM Transport rule RE: This may shock you 4/8/2015 2:00:00 AM

4/1/2015 11:08:40 PM Transport rule RE: This may shock you 4/8/2015 2:00:00 AM

To release a message from the quarantine, the Release-QuarantineMessage cmdlet can be used:

[PS] C:\> Get-QuarantineMessage -SenderAddress "jeff.guillet@office365itpros.com" |

Release-QuarantineMessage -ReleaseToAll

An earlier version of the quarantine tool was located in the Exchange Control Panel. This version of the tool is being depreciated from October 2018.

NDR Backscatter Filtering

Often spammers send messages to random recipients on the Internet and use someone else's email address to spoof the "From:" header while doing so. When one of these messages is sent to
an invalid recipient, the receiving email server can generate a non-delivery report (NDR) which is sent back to the spoofed email address instead of the spammer. These NDR messages are
referred to as backscatter.

NDR messages are very useful. They can tell someone who sent a message that the message was not successfully delivered, for instance because the mailbox of the recipient is full or maybe
because the email address is invalid. However, false NDR messages are a nuisance for end users, as anyone who has ever received one for a message they know they did not send will
understand.

Exchange Online Protection uses a feature called Boomerang which is designed to reduce the noise generated by backscatter. Boomerang is enabled for all Office 365 tenants, and there is no
way to turn it off.

Boomerang applies a unique signature to each outgoing message. The signature is stored in an SMTP header called x-microsoft-antispam-prvs , as illustrated in the following example:

x-microsoft-antispam-prvs: <AM3PR04MB4028F8BB1D18D36001E6876D9E60@AM3PR04MB402.eurprd04.prod.outlook.com>

If an NDR message is generated for a legitimate message, the NDR message will include the signature header. When the message is then processed by Exchange Online Protection, the
header is recognized and the NDR message is valid. However, if an NDR message is received that does not have the header, the NDR message is backscatter.

If you are using Exchange Online Protection to protect on-premises mailboxes, such as in an EOP standalone configuration or a hybrid deployment, but where mail-flow is outbound and not
via Exchange Online Protection, you should enable the NDR Backscatter feature through the default anti-spam filter policy and under Spam properties. In mail flow routing configurations
where your on-premises mailboxes are protected by Exchange Online Protection inbound as well as outbound then you do not need to enable this option as the x-microsoft-antispam-prvs
header is added to all outbound messages.

Directory-based Edge Blocking (DBEB)

By default, Exchange Online Protection will reject messages for invalid recipients when you have an Exchange Online license. If you have just the Exchange Online Protection license, as you
are using EOP to filter for on-premises, the default behavior is to accept invalid recipients. The accepting or rejecting of invalid recipients is down to the Accepted Domain . When you
register a new domain in Office 365 it is added to Exchange Online as an Authoritative domain, but if you only have an EOP license, it is added as an Internal Relay domain.

Directory based Edge Blocking happens at the connection filter (refer to Figure 17-1 ) when the domain is configured as Authoritative. Invalid recipients are recipients (email addresses)
which do not exist in the Exchange Online directory. This feature is known as directory-based edge blocking (DBEB). DBEB offers several benefits, especially when a domain is under a
dictionary attack by a spammer sending messages to non-existing users. DBEB is enabled for each accepted domain.

In a hybrid deployment, recipients can also exist on-premises. The same is true in a standalone scenario where Exchange Online Protection is used to protect an on-premises organization.
Directory synchronization is then used to synchronize on-premises recipients to Office 365 / Exchange Online. However, directory synchronization does not synchronize all recipient types. For
instance, mail-enabled public folders or dynamic distribution groups are not synchronized. As such, without extra configuration, messages will be rejected for these recipient types. Two
options exist to avoid the dropping of messages for missing recipients:

1. Create a mail contact for the missing recipients in Office 365. This will ensure that a valid recipient exists in the tenant and DBEB will accept the message after which it is forwarded to
the on-premises environment.
2. Reconfigure a domain to be an Internal Relay domain instead of being Authoritative .

In the latter scenario, DBEB will be disabled automatically. Message for recipients that are not found in the Office 365 directory will be forwarded to the on-premises organization, or if you
do not have a dedicated connector, to where the MX record points. For this to work, you might also configure an outbound connector, scoped to the domain name. For instance, if you have
configured the domain office365itpros.com as an internal relay domain, you should also create an outbound connector as illustrated in the following PowerShell example:

[PS] C:\> New-OutboundConnector –Name "To On-premises"

–RecipientDomains "office365itpros.com" –ConnectorType "OnPremises"

–SmartHosts "onprem.domain.com" –UseMXRecords $False

When running the New-OutboundConnector cmdlet, it is important to include the OnPremises connector type. In this example, the connector is scoped to include only the office365itpros.com
domain. However, you might have multiple relay domains. If you want to forward all messages for all accepted domains to a specific smart host without having to specify each domain
individually, you can use the –AllAcceptedDomains switch instead. This will ensure that a single connector is used for all known accepted domains.

[PS] C:\> New-OutboundConnector –Name "To On-premises" –AllAcceptedDomains $True

–ConnectorType "OnPremises" –SmartHosts "onprem.domain.com" –UseMXRecords $False

The benefit of using the AllAcceptedDomains parameter is that new accepted domains are automatically included in the scope of the outbound connector.

Bulk Email Filtering

One could describe bulk email as potentially unwanted messages, often in the form of newsletters that are sent to a vast number of recipients at the same time (hence the reference to bulk).
The difference between bulk email and junk email is relatively small. However, many consider bulk email to be spam. That said, bulk email can be legitimate too! Users may deliberately have
signed up for a newsletter while other users might find the same newsletter spam. Unlike spam messages, determining whether a message is bulk email is much harder.

When bulk email filtering is enabled, Exchange Online Protection uses both internal and external sources to determine whether a bulk message is likely to be spam or not. Next to the
sender's reputation, Exchange Online Protection looks at the message itself. By looking at patterns, links and using machine learning, EOP tries to separate good from bad senders.

In a similar way to how junk email is handled, a message header is used to identify bulk email at the end of the filtering process. The BCL information is encapsulated in the X-Microsoft-
Antispam header in a field called "BCL". The field can have the values described in Table 17-4 :

BCL Value Meaning

0 The message is not from a bulk email sender

1,2,3 The message is from a known bulk sender but is unlikely to generate many complaints.

4,5,6,7 The message is from a known bulk sender and can possibly generate many complaints.

8,9 The message is from a known bulk sender and likely to generate a high number of complaints.

Table 17-4: BCL header values

The settings for bulk email processing are managed from within the anti-spam policy . From there, an administrator can control a variety of options on how to deal with bulk email as shown
in Figure 17-15 :

Figure 17-15: Managing bulk email options from the Security and Compliance Center

Advanced Filtering Options

Next to anti-spam filtering options, the content filtering policy allows you to tune many of the settings that change the likelihood of messages being marked as spam.

Whether you should change the default settings depends on whether the default experience is good enough or if you are still receiving too much spam. In the Spam properties of the spam
filter policy , you can control messages to be marked as spam based on a variety of characteristics, including:

Specific code in the message. For instance, the use of specific HTML tags or JavaScript or VBScript code.
If links in a message has IP addresses instead of a host name; this frequently happens in scam messages. However, it might also block legitimate messages that include links to IP
addresses.
Messages from domains ending on .biz or .info.
Images loaded from remote websites (instead of attached to the email).
SPF failure . This allows you to control if an SPF failure results in a hard fail (message is rejected) or not.
NDR Backscatter . Allows you to enable the the older backscatter protection option. Please refer to the previous section on the Boomerang technology to see if you need to enable this.

Note: You should consider each of these options with care. Changing a setting might increase the likelihood of a message being marked as spam which tends to have a greater impact on
productivity of your end users than the occasional spam message. Some options can run in test mode. This means that instead of enforcing the message to be recognized as spam, you can
add a specific X-header to the message or copy the message to a different recipient so that you can evaluate manually if not too many false positives are generated. While a setting is in test
mode, the original email will also be delivered to the user's inbox.

Domain Keys Identified Mail (DKIM)

DKIM allows an organization to add a cryptographic signature to outgoing emails rather than encrypting the contents of the message. This signature (hash), which is based on the message's
headers and body, is added in a DKIM-specific header called DKIM-Signature . The purpose of DKIM-signing a message is to allow a remote organization to verify whether the message
originated from a platform authorized to send messages from a given domain.

Without going into too much detail, this is how DKIM works: the receiving server will query the external DNS records of the domain specified in the DKIM header to retrieve the public key of
the certificate that was used to generate the signature. That public key is then used to decrypt the hash value and to recalculate the hash of the message. If both match, the message is
proven to be authentic and not tampered with during transit.

DKIM supports multiple keys per domain. This is useful in situations where an organization operates multiple email platforms, wants to delegate control of DKIM signatures to individual
departments within the organization, uses a third-party service to send messages on behalf of one of the organization’s domains, or when it wants to update one of the keys without affecting
current mail flow. To support multiple keys, a “selector” is prepended to the domain name and recorded as part of the DKIM-Signature header. For instance: “selector._domain.com”. With the
selectors in place, each system (or third-party solution) can use a different selector and still generate valid DKIM signatures.

By default, Office 365 verifies incoming DKIM signatures and signs outbound messages – even when a customer has not specifically configured DKIM records. Office 365 uses the “selector1”
and “selector2” selectors.

DKIM Verification
After an inbound message is processed, the result of the DKIM verification process is written to the Authentication-Results header of the message. Depending on the outcome of the test, the
header will show either dkim=pass , dkim=none or dkim=fail, followed by a more human readable explanation of the result in parentheses. For instance, the following Authentication-Results
header reveals that the sender did not sign the outgoing message:

authentication-results: office365itpros.com; dkim=none (message not signed) header.d=none;

However, if the message was signed and the DKIM signature is successfully verified, the header will look similar to this:

authentication-results: office365itpros.com; dkim=pass (signature was verified) header.d=office365itpros.com;

Note that the above two headers are not showing other information that you will find there such as SPF status and the composite authentication result (compauth) and the reason code.

Configuring DKIM Signing

Normally, several steps are necessary to enable signing of outbound messages. However, as described in the section below, you do not necessarily have to run through these steps if you only
have Office 365 because Microsoft enabled DKIM signing by default and uses a clever workaround to ensure that signatures are valid. To know more about this, read the "Default DKIM
signing" section.

If you decide to manually configure DKIM for your domain and not tied to your default domain, you must publish two DKIM key selector DNS records for the domain name for which you want
to enable DKIM signing. Each record points to a specific target and is based on a combination of your domain and tenant name. Imagine you have a domain name called “office365itpros.com”
and a tenant named “mycompany.onmicrosoft.com”. The value of the CNAME records would then be the following:

selector1._domainkey.office365itpros.com CNAME

selector1-office365itpros-com._domainkey.tenantname.onmicrosoft.com

and

selector2._domainkey.office365itpros.com CNAME

selector2-office365itpros-com._domainkey.tenantname.onmicrosoft.com

The target of the CNAME record always starts with either selector1 or selector2 followed by a hyphen and the domain GUID. The domain GUID is the first part of the MX record for your
domain from the Office 365 domains information page. The last part of the CNAME target is your tenant domain name. This is the default domain that is automatically created when you sign
up for an Office 365 tenant and is in the form of <name>.onmicrosoft.com . If you are unsure what your tenant domain name is, you can always retrieve it using the Get-AzureADDomain
cmdlet. After publishing the DNS records, the easiest way to enable DKIM signing is through the Security and Compliance Center under Threat management > Policy > DKIM by clicking
enable for the domain name you wish to enable DKIM for.

Alternatively, you can enable DKIM signing with PowerShell as shown below:

[PS] C:\> New-DkimSigningConfig –DomainName office365itpros.com –Enabled $true

Domain Enabled

------ -------

office365itpros.com True

If the CNAME records are unavailable or if the records were created incorrectly, the above command will fail with the following error:

WARNING: Config is created but cannot be enabled since CNAME records are not published. Please enable this policy using Set-DkimSigningConfig once CNAME records are published.

If you have recently updated your DNS records and set the CNAME record, then you might need to wait until the DNS hosting company publishes the records to the internet, and not just the
fact that you have updated them in their admin portal. After that, you can attempt to enable the configuration again using the following command:

[PS] C:\> Set-DkimSigningConfig –Identity office365itpros.com –Enabled $true

If you continue to see the error message, it might be worth comparing the values of the CNAME records you created with the values that Exchange Online Protection expects. To retrieve the
correct selector CNAME values, run the following command:

[PS] C:\> Get-DkimSigningConfig | fl *CNAME*

Selector1CNAME : selector1-office365itpros-com._domainkey.mycompany.onmicrosoft.com

Selector2CNAME : selector2-office365itpros-com._domainkey.mycompany.onmicrosoft.com

It can take up to one hour after you configure DKIM before the changes have successfully replicated to all Exchange Online Protection servers. There is nothing you must do to use DKIM
signing as all outbound messages are automatically signed. The following output is taken from a test message, sent to an external recipient from an Office 365 tenant which has DKIM signing
enabled. Note the dkim=pass reference:

Authentication-Results: spf=pass (sender IP is 1.2.3.4) smtp.mailfrom=office365itpros.com; dkim=pass (signature was verified) header.d=office365itpros.com; dmarc=bestguesspass
action=none header.from=office365itpros.com;compauth=pass reason=109

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=office365itpros.com;

s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version;

bh=WeGO72W2hUk4jdiG4YKREVBP5fA4PaMBqox2yZKnVYI=;

b=mSTzijcUsZiAJizFSwmWOEpaNYBjUFXDoFQNaAgFwpbDhCcun5P5W+MYPxRRx5//KAnWT8hsv559VEU/E6PCENAQbRUnJ/Cg

biQRcoh6+X5+vtisdXenC7gncoTOXWDAK1sqh46mzW3LslvGVH4xdzFV6i7spZRoMif2cd/qtmU=

Along with the signature (shown prepended by b=), the DKIM header also includes additional information, represented as tags which are used during the DKIM verification process. For
instance, it indicates which selector was used (s=selector1) and a list of headers used during the signing process (h=). Trying to decipher the headers can be extremely tedious and is usually
unnecessary. To learn more about the finer details of the DKIM header and the various tags it can have, read rfc6367 .

Default DKIM Signing

DKIM is a very important tool to fight spoofed messages. As such, Microsoft has made the decision to automatically sign all outbound email traffic, even when the customer has not setup
DKIM or configured any specific DKIM DNS records. You might wonder how Microsoft is able to do that, especially because they do not control the DNS zone for your domains? In fact, they
do control one DNS zone which is directly linked to your Office 365 tenant: the default tenant domain <tenantname>.onmicrosoft.com . This allows them to create a DKIM signature based on
the tenant domain, and still correctly represent your organization. The following example shows a regular DKIM signature as you would expect when you have configured DKIM and the
appropriate DNS records:

From: sender@office365itpros.com

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

d=office365itpros.com; s=selector1;

h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version;

bh=<body hash>;

b=<signed field>;

Now, take a look at the following example of a DKIM signature when you have not explicitly setup DKIM. Notice the selector (s= ) and domain (d= ) fields:

From: sender@office365itpros.com

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

d=o365exchangebook.onmicrosoft.com ; s=selector1-o365itpros-com ;

h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version;
bh=<body hash>;

b=<signed field>;

Based on the above information, you can reconstruct the actual DNS records being referenced to the following value: selector1-o365itpros-
com._domainkey.o365itpros.onmicrosoft.com .

Using Resolve-DNSName (or nslookup for those of you who are old school) you can then verify that the record exists:

PS C:> Resolve-DnsName selector1-microsoft-com._domainkey.microsoft.onmicrosoft.com -Type txt

v=DKIM1; k=rsa;
p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkHq3ztGIm1R8alD+7oZiaG5mTUttFdFOlpKHRBZCPFG4sugV1EfF5F6JpwbJDzZmyIlqYfTgUkmYOvbHsoYvW7rddLKVTh+vE1SZ5P9coIHrw759hXbpPDSQ9JNP8aN+Bfrg6YMEWnOGA+PL+ZpyvswcB0jz9M6yMvowOxCHv5QIDAQAB;
n=1024,1435867504,1

The key returned by the record is used by the target system to verify the DKIM Signature and is no different from the record otherwise used when manually setting up DKIM. After all, when
setting up explicit DKIM signing in Office 365 yourself, you do not create a TXT record with the key; you create a CNAME record which points to the exact same TXT record in Microsoft's
DNS zone for your tenant, as illustrated in the example.

Automatic DKIM Key Rotation

Microsoft uses two different keys for DKIM signing. As mentioned earlier, there are several reasons for doing so, but the most important one is that it allows Microsoft to regularly rotate
(update) keys without affecting the DKIM verification process.

Here's how it works: at any given time, messages are signed using one of the two selector keys. For instance, selector1 . When Microsoft wants to update the key value behind selector1, it
would not just update the key value. If it would do that, messages that are in transit and thus messages that have not yet been verified for their DKIM signature would be invalidated. This is
because the recipient's system queries the key value as it processes the message. As such, if the key value is updated between the moment a message is signed (with the old value) and
checked (with the new value), the verification process fails and the DKIM Signature is considered fake because the key used to sign the message is not the same as they that is used to verify
the signature.

To overcome this problem, Microsoft uses a two-step process to update the key values. In the above example, Microsoft would first switch to using the key value behind the second selector
(selector2). Any new message leaving Office 365 from then on is signed with this new key. When those messages are then processed by remote systems, they would query the key value for
selector2 instead of selector1. Messages that were still signed by the first selector key, can also still be verified because the key value has not (yet) been updated.

After a week, Microsoft updates the key value of the first selector key. This leaves ample room for messages to be delivered and verified, therefore removing the need for that key to stay the
same. One week is a reasonable interval, especially considering the common expiration time of 48 hours for an email message; messages not delivered within 48 hours typically bounce back
with an NDR stating as much. Once the selector key is updated, Microsoft repeats the process for the second key.

By using a two-step process, Microsoft avoids invalidating messages still in transit or potential delays in DNS replication; after all, it could take several hours for the new key values to
replicate across the internet and every system picking it up correctly.

Domain-based Message Authentication, Reporting and Conformance (DMARC)

DMARC is a method for authenticating messages which builds upon the existing SPF and DKIM features. It proves particularly useful to prevent phishing attacks. This is when an attacker
spoofs a legitimate domain to trick the recipients in visiting a malicious website and possibly revealing personal information such as passwords. Spoofing a message means that the spammer
places a value in the RFC5322:From field that is not their own domain, and uses a value from another domain to pretend that the email is from that organization instead. The RFC5322:From
header is the one that is shown in clients such as Outlook. Because of this, spoofed messages are hard to differentiate from legitimate emails for end-users.

DMARC specifies what action should be taken if the SPF and DKIM validation for your domain fail and thus remove any ambiguity. Without DMARC telling the receiving system what the
sending systems intent is with regard to quarantine or reject, the receiver may still decide to not reject messages if either SPF or DKIM validation failed.

Similar to SPF and DKIM, DMARC requires a DNS TXT record which can be queried by the receiving party. A typical DMARC header looks like the following:

v=DMARC1; p=reject; pct=50; rua=mailto:postmaster@domain.com; ruf=mailto:dmarcfailures@domain.com; fo=1

v specifies the DMARC protocol version.

p specifies what action should be taken if SPF and DKIM validation fail. Possible actions are:

Reject.
Quarantine.
None (monitor mode).

pct gives the percentage of messages subjected to the DMARC policy. The default is 100
rua or reporting URI for aggregate reports contains the email address(es) to which aggregate reports should be sent.
ruf or reporting URI for forensic reports contains the email address(es) to which forensic reports should be sent.
fo or forensic options . In this case a value of 1 means sent forensic reports (ruf value) when either DKIM or SPF fails. The default value is 0 if fo is not mentioned, and this means send
forensic reports when both DKIM and SPF fails.
sp for sub-domain policy . This is not shown in the above example. When this option is not listed, this means all subdomains inherit the reject policy found in p=. If you want a different
policy for subdomains than the parent policy use sp=. For example if you know you have no subdomains then add sp=reject;p=none. This will report on DMARC for the domain, but
automatically reject all subdomains as you do not send email from them. If you have specific subdomains that you do you, then add a DMARC record for that domain with its own policy
to override the parent policy.

Aggregate reports are reports that the receiver sends daily to the email address specified in the rua field, which contain information such as how many emails have been received and if these
messages passed SPF and DKIM tests. Forensic reports are better known as failure reports and are sent to the recipients specified in the ruf field, each time DMARC fails. These reports are
extremely useful to figure out why messages fail DMARC processing. Note that most of the large receivers (for example Office 365, Gmail, and Yahoo) do not send forensic reports regardless
of you requesting them.

How DMARC Works

The following steps outline how DMARC processes a message:

1. A user receives a message. For example, from emails@domain.com .

2. First, the receiving server will verify if "domain.com" has a DMARC policy (DNS record).
3. If yes, does the message pass SPF/DKIM processing?
4. If yes for SPF, does the RFC5322:From field and the envelope sender (RFC5321:MailFrom or RFC5321:EHLO domain) header match? Or, if yes for DKIM checks, then does the
RFC5322:From match the DKIM signing domain (d= header). This process is also referred to as alignment checking.
5. If the alignment check fails, the message fails DMARC processing and the action specified in the DMARC policy is executed.

If we apply this logic to the DMARC example policy from above, this means that a spoofed message (which would fail SPF or DKIM) would be rejected and a failure report would be sent to
dmarcfailures@domain.com :

v=DMARC1; p= reject ; pct=50; rua=mailto:postmaster@domain.com; ruf=mailto: dmarcfailures@domain.com ; fo=1

If you decide to create a DMARC record for your domain, it is best to set it up in monitor mode first. This allows you to sift through the failure reports to better understand why any messages
from your domain fail DMARC processing. DMARC processing does not always fail because a message is spoofed! Sometimes messages are sent from a legitimate server that was forgotten to
be included in the SPF record or does not do DKIM. The more messages your environments process (or the more messages that are spoofed), the more failure reports you will receive. For
large organizations, going through the failure reports can be very time-consuming. Luckily third-party services exist which you can specify in the DMARC policy. These services will then
process the failure reports for you and provide you with a digest of the failures and the reasons.

Microsoft has published some statistics in 2015 relating to the effectiveness of DMARC and how email messages are treated when DMARC processing fails. For instance, 24% of the messages
that fail DMARC processing are still delivered to the user's Inbox for a variety of reasons. In a third of the cases, this is because the customer has configured a transport rule or has added the
sender to a safe list therefore overruling the default DMARC processing. All this goes to illustrate that, even with the right mechanisms in place, the necessary actions aren't always what
they should be, or what you expect them to be.

Real world: If your domain is a target for a lot of spoofed messages, which is often the case for larger and well-known organizations, enabling DMARC might decrease the amount of
phishing and spoofed email messages your organization receives. However, if you are a smaller organization the risk from spoofed emails is likely just as high, as lots and lots of
organizations are getting business compromise attack emails and the end users are responding and revealing passwords or transferring money etc. Enabling DMARC to reject is strongly
recommended.
When Microsoft process an inbound email from a domain with a DMARC p=reject policy on the domain, because many of their customers using Exchange Online have complex routing before
EOP (other vendors or on-premises routing for example), Microsoft cannot guarantee a DMARC failure really is a failure because the previous hop might be on the receivers side of message
routing and not being spoofed by someone other than the sender. Microsoft will therefore add “action=oreject” to the authentication header for an email that fails DMARC that has a reject
policy, but Microsoft has not rejected it. This “oreject” code (or “oquarantine” as another lesser seen example) can be used in transport rules to block some phishing emails. Most emails in
this category are rejected before reaching the end users mailbox but as mentioned above, emails can be rescued from the spam filter by settings such as whitelisting domains or using safe
senders.

DMARC Processing on Inbound Messages

Office 365 also performs DMARC processing on inbound messages if a DMARC policy exists for the sender's domain. Just like SPF and DKIM processing, the result of the DMARC test is then
written to the Authentication-Results header:

authentication-results: spf=pass (sender IP is 1.2.3.4) smtp.mailfrom= phishing.com ; dkim=none (message not signed) header.d=office365itpros.com; dmarc=fail action=quarantine
header.from= office365itpros.com ; compauth=fail reason=000

In this example, the DMARC test failed because the alignment between the RFC5322.From address and the envelope sender did not match (phising.com does not equal office365itpros.com).
The composite authentication result of SPF, DKIM and DMARC together (the compauth value) shows a failure and the reason code means the message failed explicit DMARC authentication.
Compauth headers are only shown for ATP customers.

If the SPF domain in the mail from header does not match the From header on any email entering Office 365, Microsoft will mark that email as junk. This is because the mailfrom header
could match a valid SPF record, but be unrelated to the domain that appears in the client application (the From header). This scenario would result in spoofed emails being marked as ok.
Therefore Microsoft require that the mailfrom and from headers need to match so that the email will be not considered a spoofed email.

Anti-spoofing Protection and Reporting

Spoofing is when someone sends messages using your email domain through an email system other than your primary (internal) messaging system. Often a malicious attacker tries to
impersonate a legitimate person's email address to convince the recipient of the message to perform actions helpful to the attacker. The concept of spoofing is not new and over the last few
years another form called "insider spoofing" has emerged and quickly gained popularity with attackers. Insider spoofing, sometimes also referred to a spear phishing attack, is no different
from regular spoofing, except for the fact that the message appears to be coming from an internal recipient, which makes it much harder for the recipient to figure out if a message is invalid
or not. Typically, insider-spoofing impersonates highly-ranked employees (such as a C-level executive) and targets other employees to convince them to perform an action such as sending a
wire transfer.

Fighting spam and phishing is an ongoing task. Even more recently, a newer form of insider spoofing where user accounts are compromised (maybe due to a phishing attack) and the spoof
emails are actually sent from those accounts. The malicious actor logs in as the user and sends email with the spoofed information within it from a compromised user’s real mailbox.
Protection against this form of attack include using multi-factor authentication everywhere together with proper password policies and protections. These topics are covered elsewhere in this
book.

Although features such as SPF records, DKIM signing and DMARC help to detect and repel spoofing attacks, only a minority of organizations have successfully adopted some or all these
features, so gaps are left for attackers to exploit.

To detect spoofed messages, Microsoft processes each incoming message and inspects the various TO and FROM headers in the message and compares the values to one another. If a
message is sent to someone inside your organization and the FROM field matches an internal domain, several checks are performed. If the message was sent from inside the organization, or
the message was received through a host or service which may send messages on your behalf (for example, because the host is present in the SPF record), the message is deemed to be
legitimate. However, if the message does not pass any of these tests, or if it was received by a host with a bad reputation, the message could be considered spoofed.

When the anti-spoofing feature intervenes to mark a message as spam, the value SFTY:9.5 is added to the X-Microsoft-Antispam header, which also has information about the results of other
scanning engines:

X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(71701004)(71702002);SRVR:BY2PR12MB0565;SFTY:9.5

Spoof Intelligence
There can be legitimate reasons that justify the spoofing of your email domain(s). For instance, if you have hired an external marketing company to send out electronic polls to your customers
or employees, or if you have a business application that sends notification emails to your internal users. Differentiating between legitimate and malicious attempts to spoof your domain is not
an easy task; especially because features like SPF, DKIM, or DMARC are often not used or badly implemented. It is also true that none of these features protect your tenant when accounts
are compromised.

If you have purchased E5 or standalone Advanced Threat Protection licenses, you have access to the Spoof Intelligence feature. This is designed to control what senders can send messages
on behalf of your organization. The feature is available within the Anti-Spam policy in the Security and Compliance Center. Using the Spoof Intelligence dashboard, you can allow or revoke a
specific sender's permission to spoof your domain. When you open the Spoof Intelligence policy, you can see a list of senders known to have sent messages on behalf of your company in the
past 30 days. Added details are given to help you determine if a sender should be allowed to spoof messages. For instance, next to the information about how many spoofed messages were
received so far, or which users were spoofed, the policy tries to show who the sender is. The specific sender information is obtained by EOP by looking at the PTR DNS record of the sending
server's IP address. If no PTR record is found, the IP address is displayed in the report.

Note: Often legitimate senders have a PTR record that enables Office 365 to look up and display the hostname or domain name information for the sender. If no PTR record is found, extra
caution is advised as it is very likely that the sender should not be allowed to spoof your domain. This being said, even malicious senders can have A PTR record too!

Although you cannot stop a (malicious) sender from trying to spoof your domain, the Spoof Intelligence feature controls how Exchange Online Protection handles incoming email. If a sender
is not explicitly allowed to spoof one or more users within your organization (domain), messages from that sender will be marked as spoofing attempts and users will see be notified of the
fact.

If you feel adventurous, you can also control the Spoof Intelligence feature through PowerShell using the Get-PhishFilterPolicy and Set-PhishFilterPolicy cmdlets. Retrieving the current list of
senders spoofing messages for your domain is relatively easy by running the following command:

[PS] C:\> Get-PhishFilterPolicy -SpoofAllowBlockList

Sender Spoofed user # of messages # of user complaints Allowed to spoof

------ ------------ ------------- -------------------- ----------------

persgroep.be tony@office365itprobook.com 1 0 No

easyhost.be More than one 22 0 Yes

telenet-ops.be More than one 4 0 Yes

telenet.be michael@office365itprobook.com 14 0 Yes

mailprotect.be More than one 8 0 Yes

Unfortunately, making changes to the list of senders is not as straightforward. First, you need to export the information to a CSV file. Then, after editing the CSV file using a text editor like
Notepad, you must import the contents of the file again and then specify the Spoof Intelligence policy to import the updated settings as follows:

[PS] C:\> Set-PhishFilterPolicy -SpoofAllowBlockList (Get-Content -Raw <filename>)

After importing the changes, you can use the Get-PhishFilterPolicy command again to verify that the changes were applied correctly.

In addition to the new anti-spoofing features designed to stop spoofed messages from being delivered to the user inboxes, administrators can now also request a new report, called the Spoof
Mail Report. The report is accessible through the Office 365 Portal or PowerShell. To get the report in PowerShell, use the following command:

[PS] C:\> Get-SpoofMailReport | Select Date, Direction, Action, SpoofedSender, TrueSender, SenderIP

Date : 14/04/2016 0:00:00

Direction : Inbound

Action : GoodMail

SpoofedSender : Boss.Man@office365itpros.com

TrueSender : hubspot.com
SenderIp : 50.31.57.0/24

Date : 11/04/2016 0:00:00

Direction : Inbound

Action : CaughtAsSpam

SpoofedSender : CEO-of-the-company@office365itpros.com

TrueSender : someonlinemarketingservice.com

SenderIp : 1.2.3.4/24

The report displays information about spoofed messages, like the information that is available in the Spoof Intelligence policy. It can show you what spoofed messages were received, whom
they were impersonating, and who sent the message. In the above example, the top message was apparently sent by an authorized service, as that message was classified as GoodMail .

In the example below, the report shows a spoofed message which was marked as spam. This could be for a variety of reasons, most likely because the sender was not explicitly allowed to
send mails on behalf of the user:

The report serves multiple purposes. First, it helps you understand how many messages inside your organization are spoofed. Perhaps more importantly, it tells you which account is being
spoofed most, and can reveal spear phishing attempts. Secondly, the report enables you to generate a list of hosts sending messages on your behalf. You can then use this list to verify if
authorized senders are correctly configured and whether you have represented them on your own SPF records, lowering the likelihood of them being marked as spam.

Safety Tips
To increase visibility for end users to messages holding potential threats such as spam, malware, spoofing, or phishing, Microsoft includes visual cues in messages to warn users when they
meet potentially dangerous messages. Safety Tips , which work in much the same manner as Mail Tips, are tags that are inserted into messages by Exchange Online Protection as email is
processed before delivery to end users. Whenever a message is considered suspicious, Exchange inserts a safety tip. Similarly, if a message is received from a trusted sender, the notification
will make this clear. In a first phase, OWA supports four different safety tips (red, green, yellow, and grey), but Outlook only displays the red safety tip when a message has some dangerous
content. Table 17-5 lists the types of safety tip supported by EOP.

Color Safety Tip Condition

Red Suspicious . These messages are likely to EOP detects the presence of a known phishing message or the message characteristics are such that EOP considers it to be likely to
condition phishing scams and should not be be a scam. See this blog post for an explanation of why messages are assigned red safety tips.
opened.

Yellow Unknown . The message is spam. EOP has scanned the message and it has failed the standard anti-spam tests.

Green Trusted . The message is from a trusted Microsoft has a list of domains owned by trusted sources (such as itself). Messages from these domains are considered trusted.
source.

Grey Safe . An informational tab that indicates the The message is from a domain considered to be safe by the tenant (for example, the IP address for the server is in the IP allowed list
message has been marked safe by the tenant ), or on the user’s safe senders list, or it was put in the user’s Junk Email folder and subsequently moved back to the Inbox to indicate
or user. its safe status.

Table 17-5: Types of safety tips

Safety tips are now deployed across Office 365 and are enabled by default for all tenants. Safety tips can be enabled or disabled for a tenant by updating the Anti-Spam settings policies used
by EOP. The Set-HostedContentFilterPolicy <name> -InlineSafetyTipsEnabled command is used to update the setting for safety tips via PowerShell. Removing Safety Tips is not
recommended.

It’s possible to create a mail flow rule that examines message characteristics and then injects an HTML disclaimer to achieve a similar experience for the end user but placing a banner on all
emails saying they are “external” or words to that effect can cause the user to ignore the message as they see it all the time. Better options are to examine the message properties and only
write a banner if you know that the email is suspect. Note though that this functionality is now built into the product and safety tips are displayed when required as well as moving emails to
quarantine or junk folder as required. Adding headers to messages can end up being contrary to the intent to warn users about suspect emails.

Reporting Phishing messages to Microsoft

Many of the features Microsoft develops around spam and malware fighting are based on data that Microsoft harvests from Office 365. Using machine learning techniques, Microsoft distills
useful information from the data to improve the efficiency of its protection features. However, due to the way that spammers continually introduce new techniques to bypass checks, it’s
impossible to give a 100% guarantee that every threat will be detected. Sometimes messages slip past the all-seeing eye of EOP.

When users believe they have received a Phishing email, they can use the Junk drop-down option (Figure 17-16 ) in OWA to report the message to Microsoft, or the Report Message Add-In as
mentioned earlier. In turn, Microsoft will then analyze the message to help improve the efficiency of its protection systems. If a user marks a message as a Phishing message, it is
automatically moved to the Deleted Items folder.

Figure 17-16: Marking a Phishing message with OWA

Zero-hour Auto Purge (ZAP)

Despite all the efforts to fight off spam, it is entirely possible that a spam message manages to escape the scrutiny of EOP's multiple scanning engines and find its way into a user's Inbox.
There are many reasons that could lead to a message not being classified as spam or malware. Sometimes this is because the sender (still) has a good reputation, or perhaps because the
malware in the email was so new the malware signature database in EOP was not yet updated and, as such, could not successfully find the malware when it was delivered to your
organization.

Zero-hour Auto Purge (ZAP), attempts to deal with the above scenario by continuously monitoring EOP for updates to malware and spam signatures and, if a new signature is detected that
matches one of a message that was already delivered to user's mailboxes, it will retro-actively move those messages into the mailbox' Junk e-mail folder, provided that the user has not yet
read the message. The opposite is also true for false-positives: if a message is considered safe after all, ZAP can move it back into the user's Inbox.

ZAP is enabled by default and can only be turned off through Exchange Online PowerShell. The feature will only work if the following conditions are met:

1. The user's Junk e-mail filtering is enabled. By default, this is also true, but users can turn it off themselves through the options in OWA.
2. The spam filter policy is configured to move messages to the Junk e-mail folder. This is also turned on by default, but an administrator can override the behavior and configure messages
to be placed in quarantine instead.

To verify if the feature is enabled or disabled, you can use the following command:

[PS] C:\> Get-MalwareFilterPolicy | Select ZapEnabled

ZapEnabled

----------

True
Even though the actions are transparent to the end user, ZAP leaves some information for the administrator in the message's routing information. Look for the mention of Zero-Hour Auto
Purge (ZAP) in the details of the message trace. If the information is there, it means ZAP intervened. The steps to perform a message trace, are explained later. Mail flow reports also will
show the number of emails that have been moved due to ZAP.

Invalid and Non-Registered Domain Emails

In November 2017 Microsoft stopped the relay of emails through Exchange Online Protection where the emails did not follow the RFC 5322 standard. For example, if an application was
generating emails where the From addresses (not the RFC 5321 Mail From address) was application the email used to be accepted, but now they are routed to Junk Email automatically.
Emails must be valid in the addressing syntax for the sender. In the previous example, the application would work if the From address was changed to application@contoso.com . It is the lack
of the domain name (in this example) that would cause the email to go to the Junk Email folder. Microsoft have a support article that outlines the different issues they have seen and how to
resolve them. For example, an application might send email where the From address is:

From: "Office 365" <sender@contoso.com> (Sent by a process)

This is now invalid because the format of the From address is not compliant with RFC 5322. The following would be an example of what would be allowed, and the above article covers many
more examples.

From: "Office 365" <sender@contoso.com>

Later in 2018 this policy will be strengthened so that you will not be able to use domains that you have not already verified in Office 365 as well. All mail flow will need to come from a domain
that you have proven that you already own (that is, confirmed in Office 365).

Testing Anti-Spam
To test that Exchange Online Protection properly detects and acts upon spam messages, you can send a GTUBE message to one of your recipients. The GTUBE message works in a similar
way to the widely-adopted EICAR antivirus test file does. However, instead of adding a malicious attachment, the message body contains a specific string which should always trigger the anti-
spam engine to mark the message as spam.

To test the anti-spam engine, send a message to a recipient in your domain and include the string below in the body of the message. Please note that there should not be anything else in the
body and that the string must be on a single line, without any spaces, line breaks or added characters. Other message properties such as the message subject can be chosen at will.

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

You do not need to create a rule to catch this message. Exchange Online Protection will automatically detect it as spam.

Real world: Even though this message should be caught by most spam-filtering solutions, results might be inconsistent. During our testing, for some tenants EOP blocked the message
while for others it did not.

Delisting Your IP Address from the Office 365 Block List

The Office 365 Block List is used by EOP to automatically block messages received from hosts on the list. Although an IP address does not get blacklisted for no reason, sometimes it happens
outside of your control and it can be very annoying to deal with. Common causes for an IP address to be listed is because it was repeatedly found as the source of spam, malware, spoofed or
phishing messages. Although in correctly-configured environments this should not be the case, you will find yourself on the list if you have an open relay or if you do not pay enough attention
to outbound messages.

When your IP address is listed in the Office 365 Block List, messages you sent to recipients in Office 365 will fail and a Delivery Status Notification (DSN) message is returned to the sender.
In this DSN message, the following information will be present:

5.7.1 Service unavailable; Client host [1.2.3.4] blocked using Blocklist 1; To request removal from this list please forward this message to delist@messaging.microsoft.com

Real world: Consider the following scenario: the marketing department for a large organization decides to start sending (unsolicited) bulk email to customers, or even worse, unknown
email addresses it obtained from a 3 rd -party marketing service. Because the high volume, but also because the generic form of the email, or its content, EOP decides to mark the sender –
your legitimate email platform – as a host sending spam. Although this will most likely not happen after the first batch of emails, you are almost guaranteed ending up on the block list if you
continue sending those unsolicited messages.

Just like with most other 3 rd -party block list providers, you can use a web form to request being removed from the Office 365 Block List. The form is located on https://sender.office.com and
takes you through three simple steps:

1. You must enter your email address and the IP address to be delisted.
2. A verification email is sent to the email address.
3. If the verification was successful, you can now continue to delist the IP address.

Once the delist request has been received, the IP address is typically removed within 30 minutes.

Note: It is good network design to ensure that the default external IP address that your users connect to the internet over is not the same IP address that you use to send email out via. If
both web surfing and SMTP share an external IP address you will find yourself on block lists quicker as any infection picked up from a suspicious server or site will route via the default route
to the internet, which is typically the same route web browsing traffic takes.

Unblocking Blocked Users

To protect its reputation, Office 365 will block users that continuously send email that it classified as spam. When this happens, the sender receives Non-Deliver Reports for outgoing
messages that provide specific information on what they need to do to be unblocked.

When a user is blocked, the account will show up in the action center of the protection- section in the Exchange admin center. From there, an administrator can review blocked accounts
and unblock them. You can also configure the outbound malware settings in the malware policy to notify administrators when internal users are blocked.

Note that an administrator can only unblock an account so many times. If the limit for an account has been reached, even the administrator will receive an error and a support ticket must be
opened for investigation.

Anti-Malware Protection
Exchange Online Protection also scans inbound messages for malware. The term malware covers a variety of malicious items such as viruses and spyware. Exchange Online Protection uses a
multi-layered approach to detect malware by using multiple, different anti-malware engines to scan messages for malicious code in the message body or attachments.

Unlike anti-spam configuration options, you cannot alter how malware is processed. Due to the potentially disastrous nature of a malware message, Exchange Online Protection only allows
you to delete the entire message. However, you can control what type of attachments are automatically denied or how users are notified when an email was considered to hold malware or a
blocked attachment type. The former option is also referred to as Common Attachment Type Filtering. By default, EOP maintains a list of common attachment file types that are often
associated with malware. Amongst some other file types, the default attachment filter will block out all .exe, .vbs and .reg files. By editing the default anti-malware policy or creating a new
one, you can add or remove file types to control whether emails containing such attachments are automatically quarantined.

Custom Malware Policy

The default malware policy is enough for most customers. However, if you need to have a different policy for a subset of your users or you need to update settings as described earlier, you
can create a new malware policy. To create a new policy, click the plus sign under Threat management > Policy > Anti-malware in the SCC.

You can scope the policy to a subset of users, for instance based on specific recipients or the email domain of those recipients. Like how mail flow rules are scoped, the policy uses conditions
and exceptions. The following conditions are available:

The recipient is (select a single- or multiple recipients).

The recipient domain is (all recipients within the selected domain(s)).
The recipient is a member of (select group).

 Note: Custom policies will always take precedence over the default policy and are processed in ascending order. The lower the policy priority, the sooner it will be processed.

Office 365 Advanced Threat Protection

The purpose of Office 365 Advanced Threat Protection (ATP) is to provide an answer to day zero exploits, or new methods built by attackers to bypass and fool protection systems such as
EOP. In the current threat landscape of email, having a zero-day malware and link protection service that operates on both internal and externally sourced emails should be considered by all
Office 365 administrators as a required purchase.

In this chapter we will refer to it as ATP though its real name is Office 365 ATP as Azure Advanced Threat Protection and Windows Defender Advanced Threat Protection are also separate
products in the Microsoft threat protection range.

ATP is included in the E5 plan or available as an add-on through the Microsoft Online Subscription Program, costing approximately $2 per mailbox per month (in the U.S.). Like most of the
other services in Office 365, you can enable the added protection for your entire organization, or just a select group of users. Note that until you buy at least a single license, ATP is
unavailable through the Security and Compliance Center.

Safe Attachments
The Safe Attachments feature delivers extra protection against zero-day malware. Exchange Online Protection uses multiple anti-virus scanning engines to process incoming (and outgoing)
messages. However, all these engines use "signature" files to detect known viruses and malware. You can compare it to a database that has the essential characteristics of a virus to enable
the anti-virus engine to recognize an instance of the virus when it occurs in email. The problem with this approach is that when a new virus is created, time is needed for security researches
to decipher the virus and construct its signature and then update the database. During this period, messages holding the virus are likely to penetrate past standard scanning and arrive into
user mailboxes. When ATP is enabled, messages with attachments are rerouted to a virtual sandbox environment where the content is subjected to a "behavioral analysis" based on machine
learning. During this process, the attachment is executed, scanned, and observed to determine whether it is malicious or not. If no suspicious activity is detected, the message is released for
delivery to the user's mailbox.

Each of the ATP features works as an 'add-on' to EOP. This means that messages are only subject to additional scanning or processing if none of the other EOP features have found anything
suspicious about the message. In case of the Safe Attachments feature, a message is thus only rerouted when EOP's anti-malware engine successfully processed the message and did not
detect a threat.

The rerouting of the message itself is fully transparent to the end user. When ATP was first released, the virtual sandbox that the attachment is detonated in needed to be created for each
new email, and this was effectively causing 15+ minute processing of emails. Currently the average time of email delay for ATP Safe Attachments is 45 seconds.

Before the improvements in reducing delays, Microsoft made changes to ATP so that the message body can be delivered instantly with a feature called Dynamic Delivery. This is where the
attachment is substituted with a placeholder informing the recipient that it is currently being analyzed. Once the attachment has been analyzed, and found safe, the placeholder is replaced
by the actual attachment. This was a useful development whilst attachments took more than minutes to be processed. Now that the average processing time is less than a minute it might not
be necessary to forward the message body and reattach the safe attachment later, and just accept the sub-1-minute delays. Figure 17-17 shows an email sent from one Office 365 tenant to
another that contained an attachment and that there is next to no noticeable delay in the delivery of the email even though the attachment was checked for being zero-day malware. This
message took 53 seconds in the ATP detonation chamber. If a message cannot be processed within 30 minutes, ATP performs the action defined in the Safe Attachments policy that is created
by the administrator.

Figure 17-17: SMTP Headers for an email with an attachment processed by ATP in the Outlook Message Analyzer

ATP leaves little information within the message headers. Using a tool, such as Microsoft’s Message Analyzer , you can check the message headers for the following information:

1. The Received headers section, illustrated in Figure 17-17 , does not show a delay between EOP servers – viewing this email before the end of 2017 would have shown a delay. The delay
or lack of it by itself, however, is no proof that ATP processed the message – for example internal routing errors could result in delays that are nothing to do with ATP.
2. In the Other headers section, a new header "X-MS-Exchange-Organization-SafeAttachmentProcessing" is present. The header indicates that the Safe Attachment feature processed the
message. Note that this header has no value.

Both elements together can provide you with a first impression on whether ATP processed the message. However, the best way to discover how ATP processed a specific message is by
running a message trace. The process used to run a message trace is described in a later section.

Figure 17-18 shows a series of ATP events as they appear in new style message trace results. The fourth line from the bottom is one of the ATP information rows and is shown in the popup
that appears when hovering the mouse over the row:

Figure 17-18: Message Trace results

The first DEFER-event (hidden by the popup in Figure 17-18 ) shows that the message is rerouted to the ATP scanning environment. The event called Advanced Threat Protection, gives
information about the outcome of the scanning process. In this case, the attachment was scanned and redirected back to EOP for delivery to the recipient within six seconds.
In addition to protection of attachments in inbound emails from external sources, ATP Safe Attachments file protection is also available for SharePoint Online and OneDrive for Business.
Applications like Microsoft Teams that use SharePoint Online for document management also gain protection from the ATP service when the document is stored in SharePoint Online.
Therefore, a document stored in a third-party add-on to Teams would not benefit from the ATP protection unless that document was first emailed into an ATP protected mailbox. The option to
email ATP for mailboxes is driven by a policy that applies to whomever the policy is configured for. The option for ATP protection for OneDrive, SharePoint Online and Teams is a global
setting.

Configuring a Safe Attachments Policy

Even when you assign an ATP license to an Office 365 user, the ATP Safe Attachments feature will not process any messages until you have created a policy. To create a policy, login to the
Security and Compliance Center and navigate to Threat management > Policy > ATP Safe attachments. From there, you can click the plus-sign to create a new Safe Attachments policy.
A policy consists of several elements:

A response which defines what action ATP should take when a potentially dangerous attachment is detected. Options are: Monitor , Block , Replace, or Dynamic Delivery . Monitoring
can be handy when you are first enabling the feature and allows you to assess how (well) it is performing. Although unsafe attachments are still delivered to the recipient, all the
reporting features are available. In blocking mode, both the attachment and the email are blocked and will not be delivered to the recipient while the replace mode will remove the
unsafe attachment and continue to deliver the original message. Dynamic Delivery, as earlier mentioned, will ensure that the message body is delivered in a timely fashion while the
attachment is being analyzed. Once the attachment has been processed, it is reattached to the original message. With current delivery times now being very quick, Dynamic Delivery
does not add the considerable benefit that it used to prior to the end of 2017.
Whether to redirect malicious attachments to other recipients. This might be interesting if you do not want to lose the original attachment, even if it is considered unsafe. Caution is
advised if you do decide to open the delivered attachment.
What action to take when message processing times out or a message could not be processed due to some underlying error. Note that, here, you only can select to apply the same
response as configured earlier or to just deliver the message with attachment. For instance, there is no option to quarantine or replace the attachment when the policy is configured to
block unsafe messages.

In addition to these options, you must also define the scope of the policy to say which users the policy applies. The scope is called a Safe Attachment Rule . Like mail flow rules, this rule
decides when ATP must kick into action by defining conditions and exceptions that the message has to match to. When a rule is triggered, the policy that is linked to the rule is used to decide
what actions to take. Unlike mail flow rules though, there are only a few conditions and exceptions available when creating a new Safe Attachments Rule :

The Recipient is condition allows you to select one or more individuals from your organization. This can be helpful if you need to single out a handful of users. For larger groups of
people, it is better to use one of the other conditions.
The Recipient domain is condition allows you to enable a policy for all recipients that use a specific domain. You can select both custom and default domains here. The latter makes it
ideal to test the feature if your tenant does not have a custom domain.
The Recipient is a member of condition enables you to select one or more groups to which the user must belong. If the user does not belong to one of the selected groups, the safe
attachments feature will not be executed.

Avoiding confusion. When creating a policy through the GUI, both the policy and the rule (scope) get the same name. This might lead you to believe that what you are seeing in the GUI is
the policy. However, you are in fact shown the rule which belongs to the policy. This is because the rule is what triggers the action defined in the policy. At first, this can be a little confusing,
but once you have created a few policies using PowerShell, it will all become clear.

As mentioned earlier, PowerShell can also be used to create, edit, and view Safe Attachment policies or related elements. To do so, make sure that your PowerShell session is connected to
Exchange Online. The following example creates a new policy called "TestPolicy". This policy is only applied if the message is sent to a user whose email address is
blane@office365itpros.com. Note how the policy and the scope (rule) are created as separate objects:

[PS] C:\> New-SafeAttachmentPolicy –Name "TestPolicy" –Enable $True –Action "Replace"

–ActionOnError $True –Redirect $false

[PS] C:\> New-SafeAttachmentRule "to-BruceLane" –SentTo "blane@office365itpros.com"

–SafeAttachmentPolicy "TestPolicy"

As indicated earlier, the GUI does not show a policy called "TestPolicy". Instead the GUI will show a "policy" called to-BruceLane exists. In fact, this is not a policy, but the rule that was
created. Just like mail flow rules, you can create multiple rules and prioritize them. This allows you to use different settings for different (groups of) users in your environment. Once you have
created multiple policies, use the up and down arrow to determine the priority of the rule. Using PowerShell, you can do the same by using the Set-SafeAttachmentRule cmdlet. In the
following example, the priority of the rule "to-BruceLane" is changed from 4 to 0:

[PS] C:\> Set-SafeAttachmentRule "to-BruceLane" –Priority 0

Note: Just like most other features in EOP, it can take up to 30 minutes before a new policy, rule or setting is fully active. This is because the new configuration needs time to replicate to all
EOP servers.

To enable Safe Attachments protection for SharePoint, OneDrive, and Microsoft Teams you just need to check the box at the top of the Safe Attachments page. From then on, any file shared
within these environments is scanned for malware at the time of sharing.

Safe Links
Phishing messages often include malicious links that redirect users away from apparently legitimate sites to sites under the control of the attacker. When these messages are processed by
Exchange Online Protection, a high probability exists that these links are already flagged as malicious and the message will be caught by the anti-malware or anti-spam engine. Unfortunately,
this is not always the case. Sometimes, malicious links aren't known yet or perhaps the link, at the time the message was processed, was not yet activated. The weakness in the current
approach is that it gives protection for messages at delivery. If an attacker updates a malicious webpage after it has been delivered, there's nothing the traditional anti-malware protection
can do. Usually, this is when an organization relies on the computer-based security software to kick in, but with more users responsible for their own devices under BYOD policies,
organizations don't always have control over what endpoints are used.

The Safe Links feature analyzes links at the time they are clicked. To do so, it rewrites any hyperlinks in messages at the time they are received into Exchange Online Protection. ATP will by
default only rewrite links if the sender of the message is from an anonymous source, like external senders, and if the link is in the FQDN format. A link with only a server name (no dots) is not
rewritten. Since early 2018 it has been possible to enable Safe Links for internal messages as well. This choice is off by default but can be turned on in the Safe Links policy. This protects
against a scenario where a mailbox login details are compromised, and a malicious actor uses the owned mailbox to send internal bad links.

Safe Links also applies to links written or clicked in Office 365 ProPlus and Office for iOS and Android. This option is also off by default and so if you have a ATP Safe Links policy that
predates the middle of 2017 or a Safe Attachments policy created before early 2018 it is worth revisiting these policies and turning on the new options.

Once the email is delivered to the recipient, the actual link will look like the following once you hover over the it:

https://emea01.safelinks.protection.outlook.com/?
url=http%3A%2F%2Fwww.nbconsult.co&data=02%7C01%7Cbrian.reid%40c7solutions.com%7Cbf3ccb9a107f4bf2f41008d5a3ab4aaa%7C26595c349a914bf297206914c98f0771%7C0%7C0%7C636

Note: The base URL (*.safelinks.protection.outlook.com) will always be the same, the information that follows varies on the message and the link(s) within that message.

Soon Microsoft will change the display of the link in Outlook so that users don’t see the complicated nature of the rewritten link. Opening the message in a non-Microsoft client or an older
Microsoft client though will still show the rewritten link.

When a user clicks a link in a message, the Safe Links feature checks the reputation of the remote host (website) against a set of spam lists. This usually happens within a matter of seconds
and doesn't produce any significant delays for the user. When a website is not on one of the spam lists, the user is redirected to the requested website. However, when a match is found, the
user will be prevented access to the remote website and the warning as is shown in Figure 17-19 (the new style warning pages started to rollout in October 2018). The rollout is staggered
across the service, so for a while you might see the older warning pages instead.

Depending on how the administrator configures the Safe Links policy, the user can be presented with the choice to continue to the website, despite the warning. By default, however, the
option is not enabled, and users are prevented from clicking through to malicious websites. If the administrator does allow blocked links to be clicked through to (not recommended), the click
through is registered and will show up in the reports available to the administrator.
Figure 17-19: ATP Safe Links warning page

Real world: The choice to fully block users from navigating to websites that ATP blockscould be very invasive. This is especially the case when the blocked link is accidently blocked. To
block or not block the final link is a security question, and in most cases as the blocked site is malicious it is best to ensure that the user cannot click through to the target site. Note though
that the ATP feature does not stop a user manually copying and-pasting the source link into another browser or tab to open the web site from there. While the Safe Links feature does
increase security, it is by no means a replacement for end user training and added protection on the endpoint itself.

URL Detonation
The Safe Links feature uses a list of known malicious targets which is great if a URL has already been reported as malicious. However, whenever a new malicious page is created, there is
some time between the page being live and it first being reported as malicious or being blocked. The URL Detonation feature attempts to intervene during that time by actively scanning the
target URL before allowing a user to access it. When the target location of a link is considered malicious, and the user attempts to access the link, a warning pop-up window, like when a
malicious attachment is found, is shown and the user is prevented from opening the target location. Unlike the Safe Links feature, there is nothing to configure for the URL Detonation
feature apart from turning the option on.

Configuring a Safe Links Policy

To create a policy, open the SCC and navigate to Threat management > Policy > ATP Safe Links . From there, you can choose to modify the global policy which applies to everyone in the
organization or you can create or update a policy that is applicable for only a subset of your users.

In addition to a list of hard-coded URLs to block, the global policy also controls whether to enable Safe Links for Office applications on the PC or mobile devices. When enabled, hyperlinks
within certain Office document types are also evaluated before a user is taken to them. If the link points to downloadable content, the global policy can define whether such downloaded
content should also be protected using the Safe Attachments feature.

When creating a new policy scoped to a subset of your users, you have the following options:

Select whether URLs should be rewritten or not (on/off).

Whether or not to track user clicks.
Whether or not to allow users to click through to the original URL (described above).
Whether a link that is direct to executable content should be processed in Safe Attachments instead. In this case, the first click to the site triggers the download of the content and the
use gets a yellow backed warning page. If they revisit the link a few minutes later the content can be accessed if Safe Attachments assesses it as safe.

You can also add a "safe list" of URLs which will never be rewritten. This can be particularly handy if you regularly receive emails that point to a known service or resource that your company
rely up, but which appear to break in Safe Links.

Once you have selected the desired options, you must scope the policy. Similar to how a Safe Attachment policy is created, you can scope the policy per domain, based on group membership
or on a user-by-user basis.

Anti-Phishing Policy
Phishing attacks are a popular attack vector. The EOP anti-phishing policy was a part of the standard Office 365 ATP product until September 2018, and then became a standard feature for
all tenants. The feature detects when someone tries to impersonate nominated users (like the CEO) or partners on emails inbound to your organization. A common use case is when an
attacker tries to fool people into executing a transaction (like transferring money into their account) by sending them an email that seems to come from the email address of a company
executive. Alternatively, the attacker tries to trick the recipient by sending the message from an address that appears very similar to a real address. For example, they might use
ceo@office365itpros.com instead of ceo@offce365itpros.com . While using a lookalike domain might not fool everyone, misleading one person can be enough to cause severe (financial)
damage. Like many other Office 365 features, EOP anti-phishing relies on Machine Learning (AI) capabilities to detect phishing attacks. Microsoft does not disclose exactly how the
mechanism works, as this would make it easier for attackers to circumvent the policy.

When an anti-phishing policy is in place for a tenant, EOP evaluates inbound messages by checking either the P2 FROM header and Display Name against a list of protected users or against a
list of protected domains configured as policy settings. To reduce the chance of false positives, Microsoft uses “mailbox intelligence,” an internal graph holding signals generated from user
emailing patterns (who you email, and how often). The signals help to better judge if a message is malicious. A simple example is that people with whom you communicate often are not
flagged as quickly/easily as an unknown/new sender.

Creating an Anti-Phishing Policy

Like other EOP policies, you manage the anti-phishing policy through the Security & Compliance Center. Navigate to Threat management > Policy > then select ATP anti-phishing . From
there either view (and edit) the Default policy by clicking “Default Policy” or click “Create” to make a new policy. The Default policy will always apply and cannot be deleted. The terminology
in the wizard is a little ambiguous. “Add users to protect” and “Add domains to protect” do not define to whom a policy applies. Instead, these are the email addresses and domains for which
anti-phishing will compare the P2 header information of incoming messages. For example, you configure ceo@office365itpros.com as a user to protect, and office365itpros.com as a domain to
protect, and apply the policy to all users in your organization. Then, whenever a message is received from, let’s say ceo@offce365itpros.com (note the lookalike address), that message will be
flagged as a phishing attempt if the person receives email from that address for the first time. When configuring a policy, you have the following options:

Name , Description and Applied To . As you can create multiple policies for your organization, it is a good idea to use a clear name, or at least describe what the policy is used for. The
Applied To value says what users the policy will be run against. This can be for all your users or a subset of your users, like all recipients in a specific (sub-)domain or members of a
distribution group. You can also exclude specific users from the policy.
“Add users to protect ” and “Add domains to protect ” define what email addresses or domains are scrutinized while scanning for incoming phishing attacks. Typically, you would
want to protect high-value assets such as your C-level executives or people with high privileges within your organization. To protect all the domains owned by the tenant, set the
Automatically include the domains I own slider to On.
In the action section, you can choose the action taken by EOP when it considers a message to be a phishing attempt. You can select a different action based on whether a message was
trying to spoof a user or use a domain lookalike. Actions include the ability to bcc a recipient, redirect the message, deliver it to the Junk Mail folder, quarantine the message, delete the
messge or do nothing. You also have the option to enable phishing protection tips . Like various other tips (mail tips, etc.), this will display a warning in various clients whenever a
suspicious message is received.
Enable/disable the use of Mailbox Intelligence to allow the policy to use the Graph of user email signals. This helps ensure that people you have communicated with before are not
treated as spoof senders.
Define Trusted senders and domains , very much like a typical whitelist: whenever someone receives a message from one of the senders or domains, ATP bypasses anti-phishing
filtering.

Like other policies, you can also manipulate the policy with PowerShell using the Get-AntiPhishPolicy and Set-AntiPhishPolicy cmdlets. For example, to retrieve details of the current policy:
[PS] C:\> Get-AntiPhishPolicy

To enable mailbox intelligence for the default policy:

[PS] C:\> Set-AntiPhishPolicy -EnableMailboxIntelligence $True -Identity "Office365 AntiPhish Default"

If you enabled the anti-phishing policy while it was an ATP feature (back in April 2018), then you will find that a default policy called “Office365 AntiPhish Default ” was subsequently created
for you. In these circumstances, it’s probably best to remove your policy and let the default policy do the work.

Testing ATP features

Earlier, we used a URL to test the Safe Links feature. The URL is provided by Microsoft and causes the ATP Safe Link feature to treat the link as malicious. All you do is send a message which
includes the following URL: http://www.spamlink.contoso.com

To verify whether incoming attachments are scanned by ATP, it is enough to send a message that includes a regular (safe) attachment of a type that could include malicious code. This can be
anything from an executable, to a Word-or PowerPoint file or PDF file. Because the regular anti-malware feature will not detect anything in the file because it is clean, the message will be
rerouted through ATP for additional scanning. Ensure that the attachment type though is not blocked by the common attachment list.

When the message arrives, you can check the message headers. Among the headers, you should see the X-MS-Exchange-Organization-SafeAttachmentProcessing header as described above.
This header has no value, but if the header exists, you know the message attachments were checked by ATP for malicious content. As the latency introduced in message delivery by ATP Safe
Links processing is now not very long, a delay in message delivery is not always going to be caused by ATP processing.

Bypassing ATP
There are several ways to bypass ATP scanning. If you do not want messages sent to a specific recipient to be scanned, you can exclude recipients in any of the ATP rules. If that is not
possible for you, or if you want messages received from a specific domain to be exempted from scanning, you can create a Transport Rule to "Bypass spam filtering". You would then
configure the conditions of the rule to match the sender's domain ensuring that dmarc=pass or dmarc=bestguesspass is included in the header as well (don’t just safe list domains without
checking if they authenticate first). Alternatively, you can also add the sender, the sender's domain or sending server's IP addresses to one of the EOP safe lists which will automatically cause
the ATP features to be bypassed. For example, you can add a specific IP address to the IP Address Allow List (Connection Filter).

Messages sent between on-premises recipients and recipients in Office 365 in a hybrid deployment are automatically bypassed. This is because the connectors that are setup as part of the
hybrid configuration set the Spam Confidence Level (SCL) header to -1 which indicates EOP to bypass any additional anti-malware or anti-spam filtering.

Reporting on Safe Attachments

Other than the information provided by the message traces, Microsoft offers a few built-in reports that give statistical information on processed messages. You can access these reports from
the Reports > Dashboard in the Security and Compliance Center. The report with the ATP information is the Threat Protection Status report as shown in Figure 17-20 . The ATP Message
Disposition report also exists on the report dashboard but is not part of the Threat Protection Status report currently.

Figure 17-20: Safe Attachments reports

There used to be two reports available in the Exchange Management Console. These now open the Threat Protection Status as well.

In the Threat Protection Status report, you see the following:

A primary report showing how many malicious files where found and how they were detected. This is the report that shows ZAP emails that are moved after the fact as well.
Links to the ATP File Types report and the Malware Detections report. Each of these reports link to other reports, all of which are available in the report dashboard.

The previous reports that you could open directly from Exchange Management Console, are still available from the Security and Compliance center as top-level reports. These are:

Advanced Threat Protection Message Disposition Report which will show information about the number of messages, grouped by the action that was taken on the message. Actions
include monitored, blocked, bypassed and redirected messages.
Advanced Threat Protection File Types Report will produce a report in which all malicious attachments are broken down by the file type of the attachment.

The reports in the GUI are interactive and allow you to adjust the filters on the fly. For example, you can change which file types are shown or you can alter the date range. The information
that fuels these reports comes from underlying message trace information. Note that if you specify a custom date range that goes beyond the default 30 days, a historical search will
automatically be generated, and you will have the ability to download the CSV report later instead of viewing it interactively on screen. More information on message traces and historical
searches is provided in the Message Tracing section.

The reports are also available as tables showing the underlying data. To see this information and drill down further (such as message ID and to and from information) click View Details Table
to the top right of the report.

From the Reports section in the Security & Compliance Center you can pin the reports widgets to your SCC landing page in the admin portal. All reports in the Reports-section can be
scheduled to run at a specific time. When you create a schedule for a report, you will receive an email with the report, rather than having to navigate to the Reports-dashboard yourself. This
can be useful if you need to include recipients on the report that do not have access to the SCC.

You can also retrieve reports through PowerShell using the Get-AdvancedThreatProtectionTrafficReport cmdlet. The cmdlet provides additional filtering capabilities that go beyond what is
available through the GUI. For instance, you can report only for a specific domain as illustrated in the next example:

[PS] C:\> Get-AdvancedThreatProtectionTrafficReport –Domain Office365itpros.com

Over time Microsoft are adding new reports. For example, in the month of September 2018 an insight report that shows the number of spoofed domain pairs detected with high and medium
risk has been released and is currently shown at the top of the Security and Compliance Center reports dashboard and in Figure 17-21 . This report is shown where your MX record points to
Exchange Online Protection. Where it does not, the report cannot be utilized and is therefore not shown.

Figure 17-21: Spoof Insights

Clicking any of the numbers in the Spoof Insight report gives you a list of the domain the email came from (the infrastructure that sent it) and the domain that it claimed the email came from.
For any given domain pair, Microsoft determine if this is a high, medium or non-suspicious pairing. You can then tweak this information to allow domain pairs to spoof a given domain if you
think it is okay to do to. For example, one of the 40 suspicious domain pairs detected and linked from the report in Figure 17-21 is one that showed Outlook.com as the sending infrastructure
and gmail.com and the spoofed domain. Gmail emails don’t come from Outlook.com, and so this is a domain pair that I would not be marking as allowed to spoof.

Safe Links Reporting

Along with the ATP features, Microsoft introduced added reporting that help an administrator understand what is happening in the organization.

For instance, for the Safe Links feature, the report not only shows links that were received but, if enabled, it can also detect and show what links were clicked and by whom. This makes a
very compelling argument as it allows you to understand which of your users might need extra security training. To access the report, login to the EAC and navigate to URL trace under mail
flow . From there, you can start a new trace. Creating a URL trace is very like how a message trace is created. The URL trace report is still found in the Exchange Management Console and
not in the Security and Compliance Center.

The interface provides you with some built-in date ranges, but you can also specify a custom date range. The only warning here is that the report cannot go back more than 30 days in time
and the total date range cannot span more than 7 days. Next, you can specify the recipients to include in the report, but that is not mandatory. If you do not specify a recipient, the report will
show data for all messages.

If you are looking for specific information, you can also filter based on one or more criteria. This can be useful if you have already run a general report and would like more information about
whom received a specific malicious link. Figure 17-22 illustrates the URL trace results, where the report is scoped only to include the office365itpros.com domain:

Figure 17-22: URL Trace results

For each of the items in the report, some extra information is available by double-clicking the item. Amongst routing information, the information also includes the client IP address, if that
was available when the link was clicked.

The Threat Protection Status report that you can see in the Security and Compliance Center will show files blocked that are malware when the source of the file is a rewritten link, but it does
not show the data that the URL Trace shows.

PowerShell is often a more effective solution when the need exists to process very large data sets. The following example illustrates how to get the same information as shown in the above
picture. As before, make sure the PowerShell session is connected to Exchange Online:

[PS] C:\> Get-URLTrace –UrlorDomain office365itpros.com

If you want to narrow down the results to only show the URLs which were actually clicked through by the user, do the following:

[PS] C:\> Get-URLTrace –UrlorDomain office365itpros.com | ? {$_.UrlClicked –eq $True}

S/MIME
S/MIME stands for Secure/Multipurpose Internet Mail Extensions and is a protocol that uses X.509 digital certificates to digitally sign, encrypt or sign and encrypt messages. It helps people
who receive email to be sure that the message was not tampered with in transit. S/MIME also helps authenticating the sender; it proves the message is not spoofed.

Before sending the message, the sender chooses to sign and/or encrypt the message. The recipient then uses the PKI infrastructure to verify the certificate and confirm the signature of the
message. In case a message is encrypted, things are a little more complicated because it's not the certificate of the sender that is used to encrypt the message, but the certificate of the
recipient. For the sender to encrypt a message, he must have access to the public key of the recipient's S/MIME certificate. This needs the sender to import the recipient's certificate
information prior to encrypting the email. If the message is sent within the organization, an administrator can automatically push that information to the Global Address List without needing
user interaction. However, for external recipients this needs various certificates to be imported manually on the user's computers.

Office 365 supports S/MIME for signing and encrypting messages. You can use S/MIME signing for both internal and external communications. Encrypting external traffic is only possible
using the Outlook desktop client and needs the import of recipient certificates onto the user's local machine.

Real world: When configuring S/MIME, you can use an internal or external (third-party) Certificate Authority. In both scenarios, the entire certificate chain must be uploaded to Office 365.
To do this, you can export the chain from a local machine into an SST file and upload it to Office 365, as described in this procedure . After uploading the certificate, allow at least 30 minutes
while for the changes to replicate across EOP. Without uploading the certificate chain, you will see a message that says “An error occurred while sending this S/MIME message. The
certificate used to sign this message isn't trusted by your organization.”

If encryption to external recipients is needed, the Azure Information Protection based Office Message Encryption feature (Chapter 24) might be a better fit. Azure Information Protection is a
policy-based encryption service that is managed by the tenant administrator, which removes the need to fiddle with certificates on client computers. In August 2018, this feature will be
turned on in all tenants automatically if the administrator has not turned it on themselves.

Message Tracing
Most Exchange administrators know the following problem: a user calls the help desk to report "Person X has sent me a message a few hours ago, but I still haven't received it." Sometimes
messages are misaddressed, or the sender does not actually send the email, or a message does not arrive when expected, or the message was sent but never arrived in the recipient’s inbox.
For instance, a transport rule might forward the message to a different mailbox or the message went into the user's junk e-mail folder instead of the inbox.

The ability to trace messages that flow through Exchange has always been very helpful for administrators. Message tracing proves particularly helpful to troubleshoot scenarios like the ones
mentioned earlier as it allows you to figure out what happened to a message, like when it was delivered or why it was rejected, quarantined or deferred.

In an on-premises organization, an administrator controls how long message tracking logs are kept. Office 365 keeps message tracking data for 90 days and then purges the logs. Today, an
administrator can trace messages by:

Using the Mail flow > message trace section of the EAC or the Security and Compliance Center

Using PowerShell with the Get-MessageTrace and Get-HistoricalSearch cmdlets.

The older EAC method for message tracing is being deprecated. The version in the Security and Compliance Center is newer, more flexible, and is the preferred method. Only the newer
message trace interface is covered here.

Exchange Online separates traces into those done for recent messages (up to ten days) and those over a longer timeframe (from ten to ninety days). Recent message traces are done
interactively, and you see the results on screen. However, if you need to track messages older than ten days or if you need to create a report with extended information, you use a historical
search instead. Unlike Exchange on-premises servers, message tracking information only exists online for a limited period before it moves into the Office 365 reporting data warehouse
(Chapter 21). Historical searches execute in the background and the results are emailed to the administrator as a CSV file once the search is complete. Note that it might take several hours
for Office 365 to perform a historical search.

Tracing Messages
To use message tracing in SCC, go to the Mail flow > Message trace section. From there, an administrator can create new custom traces, choose from one of the pre-canned or previously-
saved trace reports and view the results of finished/pending traces. The pre-canned trace options are useful to quickly start a trace you use regularly with the same options. Most of the time,
you will create a new (custom) trace using one of the following parameters:

Date range . By default, a trace will go back 48 hours. However, you can edit the scope to span a several hours to multiple weeks up to a maximum of 90 days. Searches over ten days
old are performed as background searches and the results are emailed to you once they complete. You can also search a window time (2 days ago to 18 hours ago for example) by moving
the date range slider – this was not possible in the old search without doing a custom search.
Delivery status : allows the administrator to target messages based on what action was performed. Amongst other things, you can specify to search for messages that were
quarantined or filtered as spam or failed or messages that have yet to be cataloged with getting status . If you are unsure what happened to the message, the default choice of All is
enough to perform the search.
Message ID : each message has a unique message ID. You can find the id from the message headers and can help to scope down the message trace to a single message.
Sender and Recipient , allows you to select who sent/received the message. If you need to specify an external sender's address in the recipient picker, it suffices you to type it in
manually and then click OK .

Direction which specifies if the message was sent (outbound) or received (inbound).
Original client IP address . The IP address of the sender's messaging service/server.

When executing a historical or extended report search, you must also specify the following:

Report Title the title of the email that will be sent to the notification email address,
Notification email address of the person to whom the report needs to be sent to.


Real world: Office 365 automatically decides when to start a historical search. You can quickly distinguish between a regular search from a historical search when creating a new trace:
when the text on the Search button changes into Next , a historical search is created instead; you must then also give a report title and recipient email address, and you will have to wait for
the results to be emailed to you. This is, for example, true for all traces for which you request an extended report, regardless of the date range.

Viewing Trace Results

There are two ways to view Trace Results: interactively (when executing a regular search) or through a CSV report (when executing a historical or extended report search). The information
exposed through the CSV files can be great for reporting purposes. The format of the file makes it generally easy to quickly sort through the information and create your own views, though
extended reports are restricted to needing to include a sender or recipient or message ID. Standard searches can be time based only with no other restrictions. The interactive results, on the
other hand, are generally more interesting for troubleshooting purposes for a variety of reasons: they show you more information on the various events that apply to the message while in
transit, and you can quickly find related messages making troubleshooting easy.

Each trace result has details on the status of the message and the actual message events. The events include information about the message as it made its way through the various filtering
mechanisms in Exchange Online Protection. Although you can derive the same information from the raw message events, the way results are displayed on screen make it easy to find
information about whether a message was delivered successfully and, in case it was not, why not.

In addition to the message status, each trace has extra information about the message in a more human readable format. For example, it might tell you how to fix a problem when a message
was not delivered successfully, or it will tell you if a message was forwarded to another recipient.

Figure 17-23 shows an example where a message goes to a mailbox that is not able to accept the message. Note how the GUI makes it easy to find the status and cause of the problem and
how it suggests a solution to the problem.

Figure 17-23: Viewing trace results in the Security and Compliance Center

The interactive interface allows you to quickly find related messages. This is particularly handy if you are dealing with many messages and cannot quickly find the information you need.
Clicking the ellipsis at the far right of the entry in the result reveals the "find related records for this message " option. This is useful in scenarios where a trace reports a failed result for
a seemingly legitimate message. Sometimes this is because there is a transport (or inbox) rule that redirects messages to a different recipient. Although the failed message will show an entry
in the message event stating as much (Figure 17-24 ), the related record gives you access to the events for the forwarded message as well (Figure 17-25 ).

Figure 17-24: Message events for a redirected message

Figure 17-25: Related records for a forwarded message.

Non-Delivery Reports
Whenever a mail server cannot deliver a message because of a hard failure, it generates a non-delivery report (NDR) and sends the NDR to the originator of the message. This happens so
that the sender knows the message failed. Compare it to a letter being "returned to sender" because the post office could not deliver the letter to its destination.

NDRs hold information about why the message delivery failed. The following example illustrates the typical information found in an NDR, saying that the recipient does not exist in the
remote email system:

550 5.1.10 RESOLVER.ADR.RecipientNotFound; Recipient not found by SMTP address lookup

The receiving server generates the information in an NDR. Historically, the recipient's messaging system generates the NDR and sends it back to the originator. However, most modern
messaging systems now generate the NDR, based on the receiving server's information, at the originating system which allows to greatly enhance the user experience, as outlined below.

For most users, regular NDR messages are not very helpful and often leaves them wondering what to do next. On the other hand, NDRs typically also have other details which are useful from
an administrator trying to figure out what happened.

When a user reports an NDR, it is normal to investigate to try to rectify the problem. However, when a remote system responds that a user is not known, or that the remote mailbox if full,
there is nothing that you can do except wait for the remote server to fix the problem. To reduce support calls, Microsoft has put a lot of focus into the layout and text contained in the NDRs
generated by Office 365 that are sent back to internal users. Messages are more descriptive and helpful than is the norm with other SMTP-compliant email systems. In some cases, the advice
given helps the user to resolve the problem themselves. Figure 17-26 is a good example. The problem is detailed and the steps necessary to deliver the message to the intended recipient are
laid out for the user to take. In addition, the NDR still has information that could be helpful to the administrator like the original message headers and normal SMTP error code.

Figure 17-26: Office 365 layout for NDR message(s)

Tracing Messages with PowerShell

Tracing messages in PowerShell with very easy. The sole difference from the approach taken by the Security and Compliance Center is that you use different cmdlets to start a regular or
historical search. A regular search, which can trace messages for up to ten days (increased from seven days from July 2018) is started using the Get-MessageTrace cmdlet. For instance, to
view all messages from the past 48 hours, run the following command:

[PS] C:\> Get-MessageTrace –StartDate (Get-Date).AddDays(-2) –EndDate (Get-Date) | Select Received, SenderAddress, RecipientAddress, Subject

R eceived SenderAddress RecipientAddress

------- ------------- ----------------

9 Oct 2018 08:16:39 michel@eightwone.com tony.redmond@ office365itpros.com

9 Oct 2018 08:14:43 newsletter@bloomandwild.com deirdre. smith@office365itpros.com

9 Oct 2018 08:13:20 michel@eightwone.com ben.owens@office365itpros.com

The output of the command is truncated in the example shown above. Usually, running a similar command would return many, many results. Of course, you can use more parameters with
Get-MessageTrace to refine the scope of the search and return fewer results. Alternatively, you can use PowerShell's Out-GridView cmdlet, which displays the results in a GUI and allows you
to easily filter and sort the data. The results of this are shown in Figure 17-27 .

[PS] C:\> Get-MessageTrace –StartDate (Get-Date).AddDays(-2) –EndDate (Get-Date) | Select Received, SenderAddress, RecipientAddress,Subject | Out-Gridview

Figure 17-27: Results for a PowerShell message trace seen through Out-GridView

Note that a message trace often returns many entries for system messages. For example, the results shown in Figure 17-27 include messages for public folder mailbox hierarchy
synchronization. Other results visible include connectors linked to Office 365 Groups to import tweets into a mailbox.

Extra Detail in Message Traces

When you run a trace in the Security and Compliance Center, Office 365 automatically returns extra routing information with more details about what happened to the message while it was
in transit. When using PowerShell traces, to get the same results, you must you to pipe the results from the message trace to the Get-MessageTraceDetails cmdlet as shown below:

[PS] C:\> Get-MessageTrace –StartDate (Get-Date).AddDays(-2) –EndDate (Get-Date) | Get-MessageTraceDetail | Select MessageID, Date, Event, Detail, Data | Out-GridView

Historical Message Traces

The Start-HistoricalSearch cmdlet launches an historical search and is used in the same way as Get-MessageTrace . For instance, to start a search for messages from a certain sender in the
past 75 days and send the report to admin@domain.com, you would use the following PowerShell command.

[PS] C:\> Start-HistoricalSearch –ReportTitle "Historical Search" –NotifyAddress "admin@office365itpros.com" –StartDate (get-date).AddDays(-75) –EndDate (Get-Date) –ReportType MessageTraceDetail -Sender
Kim.Akers@office365itpros.com

JobId SubmitDate ReportTitle Status Rows

----- ---------- ----------- ------ ---- 1accb213-ef59-4710-81f3-f511e0bc2222 8 Oct 2018 16:14:04 Historical Search NotStarted 0
The ReportType parameter controls how much detail is returned. For instance, to include the full routing details, the ReportType is changed to MessageTraceDetail . The results from the
historical search are sent by email rather than being displayed on screen, so piping the output to the Out-GridView cmdlet will have no effect. The Get-HistoricalSearch cmdlet can be used to
check the status of searches.

[PS] C:\> Get-HistoricalSearch | ? {$_.Status –eq "InProgress"} | Select SubmitDate, ReportTitle, Status | Format-Table –AutoSize

SubmitDate ReportTitle Status

---------- ----------- ------

8 Oct 2018 16:14:04 Historical Search InProgress

With Get-HistoricalSearch it is not possible to include all message information for all senders. You need to refine the search by including a sender, recipient or messageID.

Message Tracing and Anti-Spam Headers

Often message tracing is used to understand why a message was marked as spam. If you run a trace through the EAC, you can opt to include extra routing details when you submit a trace
request for messages older than 7 days. To receive the same details through the SCC, you must request an Extended Trace report. The CSV file that is returned holds some extra data,
including the anti-spam headers that were explained previously in this chapter.

Once the search completes, open the CSV file and look at the custom_data field. Note that the example below was trimmed at the end for brevity.

S:AMA=SUM|v=0|action=|error=|atch=0;S:AMA=EV|engine=M|v=0|sig=1.193.3192.0|name=|file=;S:AMA=EV|engine=A|v=0|sig=201503191928|name=|file=;S:AMA=EV|engine=K|v=0|sig=19.3.2015
18:28:0|name=|file=;S:CFA=AS|sfv=NotSpam|rsk=Low|scl=0|bcl=0|score=|sfs=(601004)|sfp=0|fprx=|mlc=|mlv=|list=1|di=|rd=mail-db3on0089.outbound.protection.outlook.com|h=emea01-db3-...

The field contains a lot of useful information regarding the various filters that processed the message. For example, the data from the anti-malware agent is displayed after S:AMA (Anti-
Malware Agent). Similarly, the anti-spam information is displayed after the S:SFA (Spam-Filtering Agent). Information on mail flow rules is shown after S:TRA (Transport Rule Agent). For
example, information from the various anti-malware engines might look like this:

S:AMA=SUM|v=0|action=|error=|atch=0;S:AMA=EV|engine=M|v=0|sig=1.193.3192.0|name=|file=;S:AMA=EV|engine=A|v=0|sig=201503191928|name=|file=;S:AMA=EV|engine=K|v=0|sig=19.3.2015 18:28:0|name=|file=;

Once you have singled out the data, you can for instance use the information explained in the Anti-spam message headers topic to understand the decisions of the anti-spam agent. In this
scenario, the message was not marked as spam (SFV=NSPM ), the client IP address was not on a black list (IPV=NLI ):

S:SFA
=SUM|SFV=NSPM|IPV=NLI|IPV=NLI|SRV=|SFS=6039001|SFS=438002|SFS=69234005|SFS=199003|SFS=3905003|SFS=189002|SFS=50986999|SFS=122556002|SFS=16297215004|SFS=46102003|SFS=15975445007|SFS=19617315012|SFS=6806004|SFS=16236675004|SF
db3-...

Mobile Devices Need Management Too

Although email was the first cloud application used by many on their mobile devices, there’s lots more that Office 365 users can do on smartphones, tablets, and other devices. Those devices
need some management attention to, and that’s where we’re going with Chapter 18.
Chapter 18: Mobile Devices, EMS, and
Intune
Paul Robichaux

As our business workforce becomes more mobile every year, and security risks for corporate data increase, it’s
important to consider how you will manage mobility for your organization. Office 365 customers have a choice of
solutions that can be used for mobile device management (MDM) and mobile application management (MAM). Each
has different features available, with different strengths and weaknesses.

Some of the considerations that come into play include which devices and operating systems will need to be managed,
and who will own those devices (BYOD vs corporate). We also need to consider whether non-Microsoft applications
such as SaaS apps or custom business apps need to be managed. Diversity in the user population is also an important
consideration. For some organizations, a single approach to mobility is needed, while other needs to apply different
policies and configurations to different sets of uses. Specific compliance requirements are also important, as some
organizations fall under strict government or industry regulations.

The solutions that we can choose from are:

Exchange ActiveSync.
Office 365 MDM.
Microsoft Intune.

In addition to those Microsoft solutions there’s an extensive range of third party mobility solutions provided by other
vendors. For this chapter, we'll look at the Microsoft solutions, but as part of your own assessment you should also
evaluate third party MDM and MAM solutions.

Before we dive into the Microsoft solutions, let’s start with a quick definition to separate the two. Mobile device
management refers to controlling the entire device, whereas mobile application management is concerned only with
controlling behavior of specific applications on the device. This may seem like an obvious distinction, but there are
some subtle points to it. For example, is it better to enforce a requirement to use a PIN on the entire device, or just on
applications that contain sensitive data? Your security team might prefer the former, but your users might prefer the
latter. Microsoft’s solutions blur the line between MDM and MAM in some cases, so I’ll try to make it clear which
solutions offer which functionality.

Comparing the three solutions

Before we dig into the details of each of Microsoft’s three solutions, it is helpful to examine them at a high level to
give you some background. Consider this section somewhat of an executive summary to help you understand the basic
capabilities and tradeoffs of each.

Exchange ActiveSync costs you nothing. It works on a very broad range of devices, and it offers basic device
management functionality. It is only loosely integrated with the rest of Office 365; it is very much Exchange-centric.

Office 365 Mobile Device Management (MDM) offers a broader set of functions than Exchange ActiveSync, including
the ability to secure access to documents stored in OneDrive for Business and SharePoint Online. It also gives you
more control over which devices you want to allow to connect, and what they can do when they do connect. There’s no
additional cost for Office 365 MDM if you have an enterprise or business subscription.

Microsoft Intune is a full-fledged MDM and MAM solution. It does everything Office 365 MDM does, plus much more.
For example, Intune can also manage Windows PCs, and you can manage access to individual applications and their
features even for devices that are not enrolled in Intune for MDM. This is extremely useful in environments that have
“bring your own device” (BYOD) policies. Intune requires you to buy licenses, either through the Enterprise
Management + Security SKU or as a standalone license.

With that bit of perspective, let’s see how you can use each of these solutions to help secure your mobile devices and
data.

Exchange ActiveSync
Exchange ActiveSync (EAS) is Microsoft’s protocol to allow mobile devices such as smart phones to securely access
their email, calendar, contacts and tasks from remote networks. EAS provides customers with policy-based controls
over mobile devices and data. An administrator can block specific users or device types from connecting, and issue
remote wipe commands to erase devices that have been lost or stolen. EAS is a great example of the mixing of MAM
and MDM functionality—some of the policy controls it offers apply only to the email or calendar applications on the
device, but support for its PIN enforcement and remote device wipe policies are embedded into all modern versions of
iOS and Android and can thus be considered as MDM features.
EAS uses Direct Push to allow connected devices to be updated immediately when a new email message has arrived in
the user’s mailbox, rather than the device needing to poll the server at fixed intervals. Of course, users can still
choose a manual polling interval in most mobile email apps, but the convenience of instantly receiving new email to
your mobile device is what most people tend to want.

Direct Push works by having the client initiate a HTTPS connection to the server with a long timeout period of 15-30
minutes. If the mailbox has a new or changed item, the server responds to the device’s open HTTPS request. When
the 15- or 30-minute timeout elapses the device simply opens a new HTTPS request and the process continues. This
“hanging sync” process allows the device to idle its cellular or WiFi connection while waiting for a response from the
server, which greatly reduces battery drain while still providing quick arrival for mail.

Note: The current version used in Exchange Online is ActiveSync Version 16 .1 . Version 16.0 delivered enhanced
calendar reliability to reduce calendar sync issues, added support for calendar attachments, and added support for
syncing the Drafts folder. Version 16.1 introduces improved keyword search by adding Keyword Query Language
(KQL) support. Users can also propose new meeting times from their mobile devices. And administrators can issue an
account-only wipe request to mobile devices, which allows for selective wipe of corporate data while leaving the
personal data on the device intact. Of course, all these improvements require client-side support to be added as well.
Apple added support for ActiveSync v16.1 in iOS 10, which was released in the third quarter of 2016.

EAS has the advantage of being very broadly supported by the two dominant mobile operating systems. A user with an
EAS-capable device can quickly configure the device herself to sync Exchange data. However, since the introduction
of Outlook mobile, Microsoft has essentially lost interest in EAS, partly because new EAS features require the device
manufacturers to update their OS and built-in applications to take advantage. On the other hand, Outlook mobile has
steadily been receiving new features, including integration with the on-device OneDrive client, the ability to sync
Exchange contacts to the device through Outlook, and so on. It’s probably best to think of EAS as the lowest common
denominator for mobile Exchange access (and MDM/MAM) and prioritize supporting Outlook mobile and the other
mobile Office 365 applications as you proceed with your deployment.

Configuring mobile devices and applications for Exchange ActiveSync

Mobile devices are simple to configure for EAS connectivity to Exchange Online thanks to Autodiscover. If the
administrator has configured the correct DNS records for Autodiscover, the end user can simply enter their user
principal name (which is recommended to be configured to be the same as their primary email address) and password
in the mobile device’s email application and it will configure itself with the correct settings.

If for some reason Autodiscover is not working, then a manual configuration may be required. The server name for
manual configuration is “outlook.office365.com”.

Understanding device access state for ActiveSync clients

Every mobile device or application that connects to EAS is placed into one of five access states. These access states
are:

Device discovery – when a mobile device connects to the server for the first time it will spend up to 14 minutes
in this state while the server determines what access state should be applied.
Allow – a device in the allow state can synchronize email, calendar, tasks, and contacts as long as it remains
compliant with the mobile device policy that is applied to that mailbox user.
Block – a device can be in the block state for two reasons. The first is that a device access rule is blocking the
device based on one of the device’s characteristics, such as the make, model, or user agent. The second reason is
if the device is not compliant with the mobile device policy that is applied to that mailbox user.
Quarantine – like the block state, a device will be placed in quarantine if a device access rule is configured to
quarantine devices matching a specific characteristic. Another reason a device can be quarantined is if the
default access level for the organization is set to quarantine new mobile devices. When a device is in this state,
the user will see a temporary message in their mailbox indicating that the device has been quarantined, but they
will not get their normal email or calendar data until the device gets into the allow state.
Mailbox Upgrade – this is a temporary state when a mailbox is moved from an older version of Exchange to a
newer version. In Office 365 you will likely never notice a device in this state.

The device discovery and mailbox upgrade states are both temporary states and are only applicable under certain
circumstances. Since they are not states that you can directly control through configuration and policies we won’t be
looking any closer at them. However, we will explore methods of controlling the allow, block, and quarantine states
(known as ABQ) as we continue.

Device access is controlled by a 9-step workflow (Figure 18-1 ) that each connecting mobile device or application goes
through.

1. Is the mobile device authenticated?

2. Is the user enabled for ActiveSync?
3. Does the device comply with the mobile device mailbox policy in effect for that user?
4. Does the user have a personal exemption that blocks the mobile device?
5. Does the user have a personal exemption that allows the mobile device?
6. Is the device blocked by a matching device access rule?
 7. Is the device quarantined by a matching device access rule?
8. Is the device allowed by a matching device access rule?
9. Apply the default access level (allow/block/quarantine) specified in the organization settings.

Figure 18-1: The decision flow for ActiveSync device access

This workflow is important to understand because at several points through the process a decision can be made to
allow, block, or quarantine the device. Once that decision is reached, the evaluation process stops. For example, if a
user is not ActiveSync enabled then they will not be able to connect regardless of whether their mobile device can
connect. Or as another example, a user who has a personal exemption that allows their mobile device to connect will
be able to do so regardless of an organization-wide device access rule that quarantines or blocks that device type, and
regardless of the default access level configured for the organization.

Let’s step through the stages of determining device access state in a bit more detail, and explore some of the
configuration options that are available to you for controlling each stage of the process. As you can see from the
device access workflow in Figure 18-1 , administrators can control ActiveSync connectivity to mailboxes in a variety of
ways.

Mobile device authentication

Before any of the ActiveSync policy or ABQ controls can take effect, the user's device must first authenticate.
Authentication can occur in several ways:

The user authenticates through an application such as the native Mail app for iOS. The built-in mail and calendar
applications on iOS support Modern Authentication, as described in Chapter 3, so you can use them with MFA
and AD FS.
The user grants access to an application or service that will access the mailbox on behalf of the device using
OAuth for authentication. An example of this is the cloud service used to proxy ActiveSync connections from the
Outlook for iOS and Android apps, which is discussed later in this chapter.
Certificate-based authentication is used to authenticate the device. When certificate-based authentication is
used, the user's credentials saved on the device do not need to be updated when they reset or update their
password, because the certificate is used instead of the username and password.

Certificate-based authentication became available in preview for Office 365 customers in July 2016. It’s fully
supported by the service, but not every client supports it. Using certificate-based authentication involves running your
own Public Key Infrastructure (PKI) such as Active Directory Certificate Services or a non-Microsoft PKI product, or
purchasing certificates from a public certification authority (CA). The certificates need to be provisioned to users'
mobile devices, often achieved through the use of a mobile device management solution such as Microsoft Intune. In
addition, the CA that issues the client certificates must have a publicly accessible certificate revocation list (CRL) that
Azure can access to check certificate validity. A federation server, such as AD FS, is also required when certificate-
based authentication is used for Android clients. Overall, certificate-based authentication involves a significant cost
and operational overhead, so the decision to use it shouldn't be taken lightly.

At the most basic level you can prevent a user from authenticating to ActiveSync by disabling or removing their user
account or their mailbox. Of course, disabling a user account is not very helpful if you would like the user to be able to
continue using the account for other access, in which case you can consider implementing one of the controls that
we’ll explore next. For certificate-based authentication scenarios, you can revoke the client certificate to prevent
authentication from the mobile device.

Mobile device mailbox policies

An Exchange Online organization is configured with one default mobile device mailbox policy when the Office 365
tenant is first created. If you make no changes at all then the default policy will apply to each mailbox user
automatically. The default policy for Exchange Online is quite permissive, as it allows most device types to connect
without even requiring any basic security controls. In the default policy mobile device passwords are optional, no
device encryption is required, and mobile devices that don’t fully support policies can synchronize anyway. While this
might ensure that the small business customers that make up most of the Office 365 customer base can easily connect
their mobile devices without any assistance from an Office 365 technical consultant, the default policies are
inadequate for any company that wants basic security for mobile devices.

An administrator can take several approaches with mobile device policies:

A single, default policy that applies to all mobile devices.

Multiple policies, with the most restrictive being the default policy and other less restrictive policies being
assigned on a case-by-case basis.
Multiple policies, with the least restrictive being the default policy and other more restrictive policies being
assigned on a case-by-case basis.

Although having a single mobile mailbox device is a perfectly valid approach, and one that avoids any complexity that
managing multiple policies creates, demonstrating a multiple policy approach allows us to look at some important
considerations for changing the default policy.

Let’s take the example of a business that wants a default mobile device mailbox policy that has basic security controls,
and a more secure mobile device mailbox policy for the sales team who travel frequently and have a history of losing
their mobile devices in airports and taxis overseas. First, the default policy that is created by Office 365 needs to be
modified. Through the EAC, first select and then edit the Default (default) mobile device mailbox policy by clicking
Mobile in the left navigation bar and then selecting the Mobile device mailbox policies tab.

Notice that the name “Default” is just a name, and is not what determines the default mobile device mailbox policy. A
tick box labelled This is the default policy is what controls which policy Exchange uses as its default. As long as the
default policy is named “Default” this is unlikely to cause any confusion. If you decide to make a different policy the
default, it would also be a good idea to rename the policy named “Default” to something else to avoid confusing other
administrators.

Clear the tick box for Allow mobile devices that don’t fully support these policies to synchronize , which will
prevent devices from connecting if they don’t support your policy requirements (for example if you require device
encryption but the device is not capable of encrypting its storage). Next, navigate to the security settings and set the
policy settings to match those shown in Figure 18-2 . These settings provide a basic level of mobile device security
without being too inconvenient for the end user.

Figure 18-2: Editing the security details of a mobile device mailbox policy

Before you save the changes be aware that any existing mobile device users that are assigned this policy will not be
able to connect if their device does not comply with the new policy settings. Different mobile device operating systems
will have different user experiences for this situation, but most of them will display a message to the end user that
explains what change is needed to be compliant (e.g., setting a passcode on the device).

Next let’s look at creating a new policy. In the mobile device mailbox policies view click the [+] icon to create a
new policy. In this scenario, the sales team needs to be assigned a more secure policy than the default policy by
requiring a longer password, a shorter device lock timeout, and enabling device wipe for too many failed sign-in
attempts. After creating the new mobile device mailbox policy, you need to assign it to the mailbox users. Navigate to
the recipients list and select a mailbox user to assign the policy to, then click View details under mobile devices in
the right pane. When the Mobile Device Details window opens, click Browse and choose the mobile device mailbox
policy to assign (Figure 18-3 ).

Figure 18-3: Assigning the new policy to a mailbox

Mobile device mailbox polices can also be assigned using the Set-CASMailbox cmdlet.

[PS] C:\> Set-CASMailbox alex.heyne -ActiveSyncMailboxPolicy "Sales Team"

Set-CASMailbox would of course be the best way to assign policies to the members of the sales team—you wouldn’t
want to click through the steps required in EAC for more than a couple of users.

At some point you may find it necessary to change a mailbox user back to the default mobile device mailbox policy. In
that situation it’s natural to think you should just use Set-CASMailbox to assign them to the policy named “Default”
again.

[PS] C:\> Set-CASMailbox James.Ryan -ActiveSyncMailboxPolicy "Default"

Because of a quirk in the way policies are applied, this won’t work. To understand why that is the wrong action to take
compare two mailbox users; one with the default mobile device mailbox policy assigned, and one with a different
policy applied.

[PS] C:\> Get-CASMailbox Kim.Akers | Format-List activesyncmailboxpolicy*

ActiveSyncMailboxPolicy : Default

ActiveSyncMailboxPolicyIsDefaulted: True

[PS] C:\> Get-CASMailbox James.Ryan | Format-List activesyncmailboxpolicy*

ActiveSyncMailboxPolicy : Sales Team

ActiveSyncMailboxPolicyIsDefaulted: False

The important detail is the ActiveSyncMailboxPolicyIsDefaulted attribute. To set a mailbox user back to the default
policy (not necessarily the policy named “Default”), use Set-CASMailbox to null the ActiveSyncMailboxPolicy attribute.
This reverts the user to the default policy and ensures they will be updated to any different policy that is later marked
as the default.

[PS] C:\> Set-CASMailbox James.Ryan -ActiveSyncMailboxPolicy $null

[PS] C:\> Get-CASMailbox James.Ryan | Format-List activesyncmailboxpolicy*

ActiveSyncMailboxPolicy : Default

ActiveSyncMailboxPolicyIsDefaulted: True

Before you change the default mobile device mailbox policy you just need to be aware that all mailboxes where the
ActiveSyncMailboxPolicyIsDefaulted property is set to True will be re-assigned to the new default policy, and those set
to False will not. To see a list of mailboxes that will not be re-assigned when the default mailbox policy changes, you
can run the following commands to discover the name of the default policy, then filter the results of Get-CASMailbox
for those that are assigned that policy but have the ActiveSyncMailboxPolicyIsDefaulted property set to False .

[PS] C:\>$default = (Get-MobileDeviceMailboxPolicy | Where {$_.IsDefaultPolicy}).Name

[PS] C:\>Get-CASMailbox -ResultSize Unlimited | Where {$_.ActiveSyncMailboxPolicy -eq $default -and

$_.ActiveSyncMailboxPolicyIsDefaulted -eq $False}

Real World: Despite your best efforts to configure mobile device mailbox policies that suit your organization’s
security requirements, you should always be aware that mobile devices and applications can “lie” to Office 365 and
report compliance with policy items such as password complexity even though they do not meet the policy’s
requirements. An example of this was the Outlook for iOS and Android application, which launched without support
for enforcing device PIN/passcode requirements. A later update to the app added support for these policy items.

Allow/block exemptions
In some scenarios an organization may wish to block a specific make, model, or operating system of mobile devices
using a device access rule, but still need to allow some of those specific devices to connect on a case by case basis.

The unique device ID of a mobile device can be allowed or blocked for a mailbox user, and in doing so this will take
effect before any device access rules are assessed. Personal allow/block exemptions can be seen in the output of Get-
CASMailbox . By default, no device IDs are allowed or blocked.

[PS] C:\> Get-CASMailbox -Identity Kim.Akers | Format-List ActiveSync*DeviceIDs

ActiveSyncAllowedDeviceIDs : {}

ActiveSyncBlockedDeviceIDs : {}

The device ID for a specific mobile device can usually be found within the device operating system, or sometimes on a
label attached to the packaging for the device. If you need to pre-allow or pre-block a device, you will need to retrieve
the device ID before it connects to the user’s mailbox. Otherwise, you can determine the device ID of any device that
is already connected by running the Get-MobileDevice cmdlet.

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers | Format-List FriendlyName, DeviceID

FriendlyName: Outlook for iOS and Android

DeviceId : 63479921AD27F5AB

FriendlyName: Black iPhone 6S

DeviceId : ApplC39GQ8NNDTDL

To allow or block a device ID use the Set-CASMailbox cmdlet. Both the ActiveSyncAllowedDeviceIDs and
ActiveSyncBlockedDeviceIDs are multi-value attributes. This makes it easy for an administrator to accidentally
overwrite existing values when they are trying to add a device ID to either the allow or block list. In this example, a
device ID is added to the allow list using Set-CASMailbox .

[PS] C:\> Set-CASMailbox -Identity Kim.Akers -ActiveSyncAllowedDeviceIDs ApplC39GQ8NNDTDL

[PS] C:\> Get-CASMailbox -Identity Kim.Akers | Format-List ActiveSyncAllowedDeviceIDs

ActiveSyncAllowedDeviceIDs: {ApplC39GQ8NNDTDL}

Another device is added to the allow list. As you can see this overwrites the existing value if the attribute is not
updated correctly.
[PS] C:\> Set-CASMailbox -Identity Kim.Akers -ActiveSyncAllowedDeviceIDs 63479921AD27F5AB

[PS] C:\> Get-CASMailbox -Identity Kim..Akers | Format-List ActiveSyncAllowedDeviceIDs

ActiveSyncAllowedDeviceIDs: {63479921AD27F5AB}

The correct method of updating the allowed or blocked device IDs lists is to use the add or remove methods.

[PS] C:\> Set-CASMailbox -Identity Kim.Akers -ActiveSyncAllowedDeviceIDs @{add='ApplC39GQ8NNDTDL'}

[PS] C:\> Get-CASMailbox -Identity Kim.Akers | Format-List ActiveSyncAllowedDeviceIDs

ActiveSyncAllowedDeviceIDs : {ApplC39GQ8NNDTDL, 63479921AD27F5AB}

Any mobile device that has been allowed or blocked by an exemption will have a DeviceAccessStateReason of
“Individual”.

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers | Format-List FriendlyName,DeviceID,*AccessState*

FriendlyName : Outlook for iOS and Android

DeviceId : 63479921AD27F5AB

DeviceAccessState : Allowed

DeviceAccessStateReason: Individual

FriendlyName : Black iPhone 6S

DeviceId : ApplC39GQ8NNDTDL

DeviceAccessState : Allowed

DeviceAccessStateReason: Individual

With device access being allowed or blocked via a personal exemption it will ensure that the device is still able to
connect even if a device access rule or the default organization setting is configured to block or quarantine devices.
Although that may sound like a good thing it can sometimes be a problem. For example, if you decide to block a
specific version of iOS due to a bug, and you implement a device access rule to block that specific version, anyone
with a personal exemption for their device will still be able to connect because the personal exemption takes
precedence over the device access rule.

Device access rules

Device access rules allow an organization to allow, block, or quarantine mobile devices based on their characteristics
such as make, model, and user agent. This flexibility means they can be deployed to suit almost any scenario. For
example, if you wanted to block all Apple iPhone devices that run older versions of the operating system, while still
allowing the latest versions to connect, you can achieve that with device access rules.

Similarly, you could configure a default access level for the organization of block or quarantine, but then use a device
access rule to allow all iPhones. Because the device access rules take precedence over the default access level, an
iPhone user will be allowed to connect while any other device will be subject to the default access level for the
organization.

To configure a device access rule in PowerShell, use the New-ActiveSyncDeviceAccessRule cmdlet. The most
important parameters for this cmdlet are - QueryString and - Characteristic . The characteristic can be either:

DeviceType.
DeviceModel.
DeviceOS.
UserAgent.

If you already know the specific characteristic upon which you want to base the rule, then you can configure a device
access rule regardless of whether a device matching that characteristic has connected to the server or not.

The query string for the characteristics for device access rules are exact match only. There is no partial match or
wildcard allowed for the query string. This means that if you need to block all versions of iOS below a certain number,
you will need to configure a device access rule for every single possible version number below the minimum that you
wish to apply. The situation with Android devices is even worse, due to the enormous number of different devices that
all have unique device types, models, OS or user agent strings. If you’re not sure of the exact query string for the
characteristic, then you can look at the characteristics of devices that have already connected to the server using Get-
MobileDevice .

[PS] C:\> Get-MobileDevice | Format-List DeviceType, DeviceModel, DeviceOS, DeviceUserAgent

DeviceType : Outlook

DeviceModel : Outlook for iOS and Android

DeviceOS : Outlook for iOS and Android 1.0

DeviceUserAgent: Outlook-iOS-Android/1.0

DeviceType : Android

DeviceModel : Nexus 7

DeviceOS : Android 4.3

DeviceUserAgent: Android/4.4.3-EAS-2.0

DeviceType : Outlook

DeviceModel : Outlook for iOS and Android

DeviceOS : Outlook for iOS and Android 1.0

DeviceUserAgent: Outlook-iOS-Android/1.0

DeviceType : Outlook

DeviceModel : Outlook for iOS and Android

DeviceOS : Outlook for iOS and Android 1.0

DeviceUserAgent: Outlook-iOS-Android/1.0

DeviceType : iPhone

DeviceModel : iPhone4C1

DeviceOS : iOS 9.2.1 13D15

DeviceUserAgent: Apple-iPhone4C1/1202.466

Here is an example of a device access rule to demonstrate how they are configured. This rule will block a device that
has a “Model ” of “Outlook for iOS and Android”.

PS C:\> New-ActiveSyncDeviceAccessRule -Characteristic DeviceModel -QueryString "Outlook for iOS and Android" -AccessLevel Block

Notice that the rule has blocked two matching devices, but one is still allowed because individual exemptions take
precedence over device access rules.

[PS] C:\> Get-MobileDevice | Where {$_.DeviceModel -eq "Outlook for iOS and Android"} |

Format-List deviceaccess*

DeviceAccessState : Blocked

DeviceAccessStateReason: DeviceRule

DeviceAccessControlRule: Outlook for iOS and Android (DeviceModel)



DeviceAccessState : Blocked

DeviceAccessStateReason: DeviceRule

DeviceAccessControlRule: Outlook for iOS and Android (DeviceModel)

DeviceAccessState : Allowed

DeviceAccessStateReason: Individual

DeviceAccessControlRule:

It is a good idea to review existing mobile device associations to see which ones will not be impacted by a device
access rule you are implementing. For example, before adding the device access rule to block “Outlook for iOS and
Android” we can run the following PowerShell command to find all matching devices that are already allowed by
individual exemption.

[PS] C:\> Get-MobileDevice | Where {$_.DeviceModel -eq "Outlook for iOS and Android"

-and $_.DeviceAccessStateReason -eq "Individual"}

Device access rules can apply an access level of allow, block, or quarantine. An allow rule can be used to permit
specific device types that are considered “safe” to connect when the default access level of the organization is
restrictive (i.e. when it blocks or quarantines). For example, an organization may choose to quarantine all devices by
default so that they can be reviewed on a case by case basis, but then configure a device access rule to allow all
iPhones because they are pre-approved for connection according to that organization’s security policies. When a
device access rule allows a device to connect it does not add the device as a personal exemption for the mailbox user,
therefore the device will only be allowed until the device access rule is removed, after which they will be re-assessed
according to the allow/block/quarantine workflow.

Block rules behave similarly to allow rules except that they block devices instead of allowing them. Other than that,
there is one notable exception: a block rule will continue to block a device even if that device is later modified in some
way that it doesn’t match the rule’s criteria. For example, this user has an iPhone running “iOS 9.2.1 13D15”, which is
the OS version for an iPhone 6S running iOS 9.2.1.

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers | Format-List friendlyname, deviceos, deviceaccess*

FriendlyName : iPhone 6s

DeviceOS : iOS 9.2.1 13D15

DeviceAccessState : Allowed

DeviceAccessStateReason: Global

DeviceAccessControlRule:

A device access rule is added to block the device OS “iOS 9.2.1 13D15”.

PS C:\> New-ActiveSyncDeviceAccessRule -Characteristic DeviceOS -QueryString "iOS 9.2.1 13D15"

-AccessLevel Block

The device is now blocked by the device access rule. The same block would also apply to any other iPhone 6S device
that was upgraded to that exact OS version.

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers | Format-List friendlyname, deviceos, deviceaccess*

FriendlyName : iPhone 6s

DeviceOS : iOS 9.2.1 13D15

DeviceAccessState : Blocked

DeviceAccessStateReason: DeviceRule

DeviceAccessControlRule: iOS 9.2.1 13D15 (DeviceOS)

The device owner upgrades the device to a newer version of iOS, however the device remains blocked as long as the
device access rule still exists. In effect this means that a user can’t unblock a device themselves by modifying its
characteristics, such as by upgrading the operating system. However, if the device OS has been blocked and the user
installs a different email application on the device that presents itself as a different DeviceOS , then that application
will not be blocked by the same rule.

If a mobile device has been blocked by a device access rule and you want to allow that device to connect again
without deleting the device access rule, then the solution is to add the device as a personal exemption for that mailbox
user. As discussed earlier this can cause issues at a later stage because once the personal exemption is in place the
device will no longer be impacted by any other device access rule you create.

In some cases, a persistent block is not the desired outcome. Sometimes a mobile device OS is found to have a
security vulnerability, and organizations choose to block that version of the device OS only, but want the devices to be
able to connect again after the users upgrade to a later version. In those situations, a device access rule with a
quarantine action is more suitable.

[PS] C:\> New-ActiveSyncDeviceAccessRule -Characteristic DeviceOS -QueryString "iOS 9.2.1 13D15"

-AccessLevel Quarantine

The above example will quarantine iPhone 6S devices running iOS 8.1.3 only until they upgrade to a later version.
After the OS is upgraded, the device can connect again, without having to delete the device access rule or add a
personal exemption for the device.

Default access level

The last step of the allow/block/quarantine workflow is the default access level of the organization. Mobile devices
that have not already been allowed, blocked or quarantined based on an earlier stage of the process are controlled
using this setting. The default access level for mobile devices in Exchange Online is set to a value of “Allow” when the
Office 365 tenant is first created.

[PS] C:\> Get-ActiveSyncOrganizationSettings | Format-List DefaultAccessLevel

DefaultAccessLevel : Allow

The default access level can be set to either allow, block, or quarantine, and these settings have the same effect as
they do for device access rules. One exception is the quarantine setting, which has controls at the organization level
to add some additional text to the quarantine notification email that the mailbox user receives. This text is useful for
providing information to end users about how they should respond to their mobile device being quarantined. For
example, some organizations want the device owners to contact the help desk by phone to request approval for the
device, while others may prefer not to burden the help desk with those calls and will instead instruct the device owner
to visit an intranet page to submit a form requesting approval.

[PS] C:\> Set-ActiveSyncOrganizationSettings -DefaultAccessLevel Quarantine

-UserMailInsert "Please visit http://intranet/mobiledevices to accept the mobile device access terms and conditions and request
device approval."

Dealing with existing devices when changing default access level

If you are planning to change the default access level for mobile devices that are connecting to your Exchange Online
organization, you should first consider the impact that this will have on the mobile devices that are already
connecting. It would not be a wise move to suddenly block or quarantine hundreds or thousands of mobile devices, as
this will surely cause a sudden influx of help desk calls from confused and angry end users.

Any mobile device that is not being allowed, blocked or quarantine by a personal exemption or device access rule is
controlled by the default access level. These devices will have a DeviceAccessStateReason of “Global” when you
review the output of Get-MobileDevice .

[PS] C:\> Get-MobileDevice | Where {$_.DeviceAccessStateReason -eq "Global"}

The output of that command may be difficult to interpret within a PowerShell console sessions so I recommend you
pipe it to Export-CSV , or run a more comprehensive report such as the Get-EASDeviceReport.ps1 script , which also
shows additional information such as the timestamp of the last mailbox synchronization attempt by the device.

Managing Mobile Device Associations

When a user has attempted to connect a mobile device or application to their Exchange Online mailbox with EAS, a
device association is created. Device associations exist for any device that is allowed, blocked, or quarantined by
Exchange Online. You can list the device associations for a mailbox by running the Get-MobileDevice cmdlet.

[PS] C:\> Get-MobileDevice -Mailbox paul@office365itpros.com |

Select FriendlyName, Device*

FriendlyName DeviceId DeviceImei DeviceOS

------------ -------- ---------- --------

Outlook for iOS and Android 257B371BCF057A09 Outlook for iOS and And

iPhone 6s 4FQI7OF7V143F350CDPJBN9R7C iOS 9.2.1 13D15

PAUL-PC8 B60E81836F754126A509240EEEF5E599 Windows 10.0.10586

An Exchange Online mailbox can have a maximum of 100 mobile device associations. For the average user that is
more than enough, but it is foreseeable that someone who frequently tests different mobile devices might reach that
limit. If 100 device associations already exist and the user attempts to connect a new device to their mailbox,
Exchange Online will first check for any existing mobile device associations that have not synced in more than 180
days and can therefore be considered inactive. If inactive devices are found, they are removed from the mailbox. If the
attempted removal of inactive device associations reduces the count below 100, then the new device can connect. If
necessary, the user can remove mobile devices from their mailbox using the Mobile devices section of OWA Options
(Figure 18-4 ).

Figure 18-4: Managing mobile device associations through OWA Options

You should exercise some caution when working with mobile devices because the “Remove” and “Wipe device”
buttons are located directly next to each other. Administrators can also remove mobile device associations for users by
opening the properties of the mailbox in the Exchange admin center, choosing Mailbox features , then clicking View
details under the Mobile Devices section. Again, use caution when removing devices so that you do not accidentally
issue a remote wipe for the device instead. There is also a limit of 20 device association removals per mailbox each
month.

Reporting Remote Devices

As we have seen, the PowerShell module for Exchange Online includes several cmdlets to deal with ActiveSync
devices. It is possible to use those cmdlets to create reports about device usage. In this example, we look for devices
using the REST API to communicate with Exchange Online, which means we are interested in looking for devices
running Outlook for iOS and Android. After fetching information about the devices known to Exchange, we fetch the
latest statistics for each device. Finally, we output some information, including the version of the O/S running on the
device (so that we make sure it’s not too outdated), the last time the device successfully synchronized with the
owner’s mailbox, and the mobile operator. One of the devices has no operator listed, which means that it only uses Wi-
Fi to connect.

Although this script isn’t very sophisticated, it provides the basis for more complicated reports.

[PS] C:\> $OutlookDevices = (Get-MobileDevice -RestAPI | Select FriendlyName, DeviceUserAgent, UserDisplayName, DeviceMobileOperator,
DeviceId, FirstSyncTime, DistinguishedName)

$Report = @()

Write-Host "Checking" $OutlookDevices.Count "Outlook mobile clients"

ForEach ($Dev in $OutlookDevices) {

$DN = $Dev.DistinguishedName.Split(“,”)[2..10] -join “,”

$Mbx = (Get-Mailbox -Identity $DN)

$Licenses = ($Mbx.PersistedCapabilities -join ";")

$DeviceInfo = (Get-MobileDeviceStatistics -Identity $Dev.DistinguishedName | Select LastSuccessSync, DeviceUserAgent, DeviceOS,

Status, DeviceApplicationPolicyStatus)

$ReportLine = [PSCustomObject][Ordered]@{

User = $Mbx.DisplayName

Device = $Dev.FriendlyName

UA = $Dev.DeviceUserAgent

DeviceOS = $DeviceInfo.DeviceOS

Status = $DeviceInfo.Status

Operator = $Dev.DeviceMobileOperator

FirstSync = $Dev.FirstSyncTime

LastSync = $DeviceInfo.LastSuccessSync

DeviceAppStatus = $DeviceInfo.DeviceApplicationPolicyStatus

$Report += $ReportLine

$Report | Sort User, FirstSyncTime | Format-Table User, Device, DeviceOS, LastSync, Operator -AutoSize

User Device DeviceOS LastSync Operator

---- ------ -------- -------- --------

Jack Smith Outlook for iOS iOS 11.2.6 20/03/2018 22:10:33 vodafone UK

Jenny James Outlook for iOS iOS 11.2.5 20/03/2018 22:47:52

Tony Redmond Outlook for iOS iOS 10.3.3 01/10/2017 18:13:27 vodafone IE

Tony Redmond Outlook for iOS iOS 11.2.6 21/03/2018 14:47:37 vodafone IE

Remote Wipe
Eventually you will meet the situation where someone has lost their mobile device, or an employee leaves the
company under circumstances where there is a concern about corporate data being stored on the device. ActiveSync
provides the capability to perform a remote wipe of mobile devices. However, there are some caveats with remote
wipes that you need to be aware of.

For a remote wipe to be successful the device must connect to Exchange Online after the remote wipe request has
been created. Unfortunately, there are numerous ways in which a lost or stolen device may never contact the server
again:

the device is not configured for push email, so doesn’t automatically connect to the server.
mobile and Wi-Fi communications are disabled to prevent connections being made.
the mobile carrier disables the SIM card.
the user changes their password in Active Directory.
the device is blocked by a device access rule.

Some mobile applications such as Outlook appear to Exchange as a device themselves, unlike the native mail apps
that represent the entire hardware device. Remote wipes for those applications only removes the data from the
application, not the entire hardware device. The Outlook mobile app also supports “selective wipe” that only wipes
corporate data and leaves any personal email accounts intact (a good example of a MAM feature, since selective wipe
only applies at the application level). Unfortunately, the remote wipe result is not reliably reported back to the server.
No matter whether the remote wipe was request was successful or not, it will often remain in a “pending” state
forever, or until the wipe request is removed. Therefore, you could never know when and if the remote wipe was
successful.
In June 2016, Microsoft announced an update to the Exchange ActiveSync protocol (EAS 16.1). Among the
improvements in EAS 16.1 is support for account-only remote wipes to allow administrators to issue a remote wipe for
only the Exchange mailbox data on a mobile device. Previously, a remote wipe for an ActiveSync device would wipe
the entire device if the user used a native mail application to connect from the device. This puts these devices on a par
with Outlook mobile, in that an EAS 16.1-compatible client will only erase mail, calendar, and contact data from the
associated account when it receives a wipe request instead of erasing the whole device.

You can test the EAS capabilities of a mailbox by using the Remote Connectivity Analyzer to perform an Exchange
ActiveSync test. In the results, there’s a line called “MS-ASProtocolVersions ” which lists the EAS versions a mailbox
supports. For a mailbox where EAS 16.1 has not yet been enabled, the output looks like this.

MS-ASProtocolVersions: 2.0,2.1,2.5,12.0,12.1,14.0,14.1,16.0

For a mailbox where EAS 16.1 is enabled, the output looks like this.

MS-ASProtocolVersions: 2.0,2.1,2.5,12.0,12.1,14.0,14.1,16.0,16.1

You can also determine the EAS version in use by querying the mobile devices for a mailbox with the Get-
MobileDevice cmdlet.

PS C:\> Get-MobileDevice -Mailbox "Kim.Akers@office365itpros.com" | Select FriendlyName, DeviceType, ClientVersion, ClientType

FriendlyName DeviceType ClientVersion ClientType

------------ ---------- ------------- ----------

Outlook for iOS and Android Outlook 14.1 EAS

Outlook for iOS Outlook 161 REST

Outlook for Android Outlook 161 REST

TestActiveSyncConnectivity 12.0 EAS

iPhone 6s iPhone 16.1 EAS

Outlook for iOS Outlook 161 REST

iPad mini 2 iPad 16.1 EAS

In the example above, the iPad is connecting using the native mail app for iOS, and is running iOS 10 which is the
minimum requirement for EAS 16.1 compatibility. So, with all of that in mind, let’s look at the process for remote
wipes.

Warning: Remote wipe is a destructive process, unless the device supports EAS 16.1 and the account-only wipes
described earlier. When a mobile device is connecting using a native mail application such as those found on iOS and
Android devices, a full remote wipe will remove all data from the device and return it to its default factory condition.
In a BYOD scenario, this could result in the permanent loss of personal data from the device, such as photos and
videos. For mobile devices connecting using a third-party email application such as Outlook for iOS and Android, a
full remote wipe request will usually remove all data from the application but not the entire device. As there are
many third-party email applications for mobile devices it is possible they will not all behave the way you expect them
to, so you should always proceed with caution when dealing with remote wipes.

First, the device owner can initiate their own remote wipe if they want to. This can be performed by logging into OWA
and opening the Options panel. In the Mobile devices section, all the user’s mobile devices will be visible (Figure 18-
4 ). However, users can only perform full device wipes, not account-only wipes. If a device owner wants to remove
only the corporate data from the personal device, and they still have the device in their possession, then they can
simply remove the accounts from the mobile apps themselves. On the other hand, if they've lost their personal device,
they might consider it worthwhile trying to perform a full device wipe instead.

For administrators who use the EAC to perform a remote wipe, clicking the Wipe device icon will present the option
to perform an account-only wipe, or a full wipe of the data on the device. After answering Yes to the confirmation
prompt the remote wipe request is created and will have a status of “Wipe pending”. The next time the device
connects the wipe request will be completed, and the user will receive an email notification to let them know the
result.

If the device never connects the remote wipe request will remain in place with a “Wipe pending” status forever.
Administrators can initiate remote wipe requests in the EAC, and also by using the Clear-MobileDevice cmdlet. The
device Identity attribute is used to identify the device to be wiped. You can retrieve the Identity attribute of a device
using Get-MobileDevice .

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers |

Format-List FriendlyName,Identity


FriendlyName iPhone 6s

Identity Kim.Akers\ExchangeActiveSyncDevices\iPhone§UVEBE4V4K17NT0V1US7QBQIK68

FriendlyName: Outlook for iOS and Android

Identity : Kim.Akers\ExchangeActiveSyncDevices\Outlook§63479921AD27F5AB

When you issue the remote wipe request with Clear-MobileDevice you can also specify an email address to receive the
notification if the wipe is successful.

[PS] C:\> Clear-MobileDevice Kim.Akers\ExchangeActiveSyncDevices\iPhone§UVEBE4V4K17NT0V1US7QBQIK68

-NotificationEmailAddresses admin@office365itpros.com

To perform an account-only wipe using PowerShell, include the -AccountOnly parameter when using Clear-
MobileDevice . When using PowerShell, even if you choose to perform an account-only wipe, the cmdlet will present
the same warning that all data on the mobile device will be permanently deleted. However, if you try to issue an
account-only wipe for a mobile device associated with a mailbox that hasn't been enabled yet for EAS 16.1, or if the
device itself doesn't support EAS 16.1, you'll receive an error message to that effect. It's then up to you, your
corporate IT policies, and perhaps the device owner themselves, to decide whether the situation warrants issuing a
full device wipe instead.

When a remote wipe request is in a “Wipe pending” status, and before the device has actually been wiped, the request
can be cancelled. Mobile devices with remote wipes pending can be identified using the Get-MobileDeviceStatistics
cmdlet.

[PS] C:\> Get-MobileDevice | Get-MobileDeviceStatistics | Where {$_.Status -eq "DeviceWipePending"}

DeviceType : iPhone

DeviceUserAgent : Apple-iPhone4C1/1204.508

DeviceWipeSentTime :

DeviceWipeRequestTime : 28/03/2015 12:58:59 PM

DeviceWipeAckTime :

DeviceFriendlyName : iPhone 6s

Identity : APCPR04A001.prod.outlook.com/Microsoft Exchange Hosted Organizations/office365bootcamp.onmicrosoft.com/

Kim.Akers / ExchangeActiveSyncDevices/iPhone§UVEBE4V4K17NT0V1US7QBQIK68

IsRemoteWipeSupported : True

Status : DeviceWipePending

StatusNote : You'll receive an email message that lets you know when the remote device wipe is complete.

DeviceAccessState : Allowed

DeviceAccessStateReason: Global

DeviceAccessControlRule:

LastDeviceWipeRequestor: admin@office365itpros.onmicrosoft.com

To cancel the remote wipe, use the Clear-MobileDevic e cmdlet with the Cancel parameter. Again, this uses the
Identity attribute of the device to identity which mobile device to remote the remote wipe request for, however the
Identity shown in the output of Get-MobileDeviceStatistics is not the correct value to use. Fortunately, you can pipe
the output of Get-MobileDeviceStatistics back to Get-MobileDevice to see the correct attribute.

[PS] C:\> Get-MobileDevice | Get-MobileDeviceStatistics |

Where {$_.Status -eq "DeviceWipePending"} | Get-MobileDevice | fl FriendlyName,Identity

FriendlyName: iPhone 6s

Identity : Kim.Akers\ExchangeActiveSyncDevices\iPhone§UVEBE4V4K17NT0V1US7QBQIK68


PS C:\> Clear-MobileDevice Kim.Akers \ExchangeActiveSyncDevices\iPhone§UVEBE4V4K17NT0V1US7QBQIK68 –Cancel

After a remote wipe completes, the mobile device partnership for the mailbox user remains in place, and the mobile
device has a state of “Blocked” due to “Policy”. It will remain in this state forever, and if the user attempts to
reconnect the device the remote wipe will immediately be performed again.

[PS] C:\> Get-MobileDevice -Mailbox Kim.Akers | Format-List friendlyname,deviceos,deviceaccess*

FriendlyName : iPhone 6s

DeviceOS : iOS 8.2 12D508

DeviceAccessState : Blocked

DeviceAccessStateReason: Policy

DeviceAccessControlRule:

If you want to allow the user to reconnect that device the remote wipe request must be removed. To remove the wipe
request you simply need to remove the mobile device partnership for the mailbox using Remove-MobileDevice .

[PS] C:\> Remove-MobileDevice Kim.Akers\ExchangeActiveSyncDevices\iPhone§UVEBE4V4K17NT0V1US7QBQIK68

Establishing an ActiveSync policy for your organization

Let’s recap the allow/block/quarantine workflow for determining ActiveSync device access states.

1. Is the mobile device authenticated?

2. Is the mailbox user enabled for ActiveSync?
3. Does the device comply with the mobile device mailbox policy in effect for that user?
4. Does the user have a personal exemption that blocks the mobile device?
5. Does the user have a personal exemption that allows the mobile device?
6. Is the device blocked by a matching device access rule?
7. Is the device quarantined by a matching device access rule?
8. Is the device allowed by a matching device access rule?
9. Apply the default access level (allow/block/quarantine) specified in the organization settings.

Remember that the steps are followed in order, and if at any stage a decision is made to allow, block or quarantine a
device, then the remaining steps of the process are not performed. When you plan your configuration, you need to
take into consideration:

The process above;

Whether you want to be a permissive or restrictive organization;
How much administrative effort will be required for maintaining configurations for each step of the process.

For example, a permissive organization that wishes to keep administrative effort to a minimum may configure:

A default access level of Allow;

Device access rules to block/quarantine only under exceptional circumstances, for example a known security
vulnerability with a new version of a mobile operating system;
A single ActiveSync mailbox policy to enforce a few security settings such as device passwords and encryption.

A restrictive organization that wishes to keep administrative effort to a moderate level may configure:

A default access level of Quarantine;

Device access rules to allow known safe makes/models/operating systems of mobile device;
Some personal exemptions for users with devices outside of that known safe list.

While a highly restrictive organization that is willing to live with a higher administrative burden may configure:

A default access level of block;

Disabling ActiveSync on all new mailboxes when they are created;
A process for enabling ActiveSync and configuring personal exemptions for specific devices on a case by case
basis;
A variety of mobile device mailbox polices to apply different security requirements depending on the level of
access to confidential data that the user may have.

Real World: If you do nothing at all and leave the default settings in place, then you are choosing to operate in a
very permissive model in which even basic controls such as passcodes on mobile devices are not required at all, and
any user in your organization can connect any ActiveSync-capable mobile device to their mailbox.

Office 365 MDM

In addition to the ActiveSync management capabilities available in Exchange Online, Microsoft has Office 365 Mobile
Device Management to give Office 365 tenants additional built-in mobile device management capabilities including
employee-owned mobile devices in a “bring your own device” (BYOD) environment. The benefits of Office 365 mobile
device management (MDM) include:

Device management – enforce security policies such as PIN/passcode requirements, and prevent jailbroken
devices from accessing your corporate data.
Conditional access – allow access to corporate data only from devices that are managed by your company and
that are compliant with your policies.
Selective wipe – remove corporate data from employee mobile devices while leaving their personal data intact.

Given that Exchange ActiveSync offers MDM capabilities, you might wonder why Office 365 offers MDM features of
its own. The capabilities of ActiveSync and Office 365 MDM are different, and are designed to solve different
problems, but can also be used side by side in the same tenant.

ActiveSync takes a device-centric approach to mobile device management. ActiveSync is used to allow, block or
quarantine mobile devices that are accessing Exchange mailbox data such as email, contacts, and calendars. The user-
centric controls for ActiveSync extend only as far as enabling or disabling the ActiveSync protocol for a user’s
mailbox. Beyond that any allow/block/quarantine is based on device characteristics such as make, model or user
agent, even when you are approving or blocking a device for an individual user. And even though tenants can use
ActiveSync and mobile device policies to enforce security policies such as PIN/passcode requirements, those policies
can be circumvented by the connecting device or application as we saw with the first release of Outlook for iOS and
Android.

Furthermore, if a mobile device is not connected to a mailbox using ActiveSync then it is not subject to any policy
enforcement at all. With Office 365 this means that the same device could be used to access corporate data using
Office Mobile apps and OneDrive for Business without any device security policies being applied. Office 365 MDM
closes that security gap by allowing organizations to enforce device management and security policies on any mobile
device that is accessing corporate data. In addition, Office 365 MDM takes a user and group-centric approach for
targeting policies.

The lack of selective wipe in ActiveSync has also been a cause of conflict in the past for many organizations, with
employees not taking too kindly to IT administrators remotely wiping their entire mobile device including personal
data. The workaround for this has usually been to use a non-native email application, which sometimes means
incurring additional costs to purchase apps, or putting up with limited integration with the device OS and other apps.
Again, Office 365 MDM solves these issues by allowing selective wipe.

Device and Application Support for Office 365 Mobile Device Management
Office 365 MDM is generally supported on the latest Android, iOS and Windows Phone devices; however, the level of
application support on each device platform may vary. When a user is targeted by an Office 365 MDM policy, their
access to Office 365 resources using Exchange ActiveSync-capable applications (such as the Mail app on iOS),
OneDrive for Business and Office mobile applications (such as Outlook, Word, and Excel) can be controlled. Microsoft
continues to work on the list of supported devices and apps for Office 365 MDM, so it is a good idea to review the
current list of supported devices and apps periodically.

Activation and Initial Configuration

Before you can use the Office 365 Mobile Device Management features you will first need to activate it in your Office
365 tenant. Office 365 MDM leverages Microsoft Intune back end services. If your organization already uses Intune
you will be blocked from activating Office 365 MDM to avoid a conflict, however, you can contact Microsoft support to
be enabled for co-existence. If you enable Office 365 MDM and later begin exploring Intune’s capabilities, you will be
able to activate Intune on your own.

Log in to the Office 365 portal with your tenant administrator credentials, and open the Security & Compliance
admin center, select Data Loss Prevention , and then Device management. To begin the configuration, the wizard
will prompt you to create a default security group so that the default device security policy can be created
automatically.

A message will appear to let you know that MDM setup may take a few hours. When MDM activation has completed
you should see a red warning icon to let you know that some more settings need to be configured. Click the Manage
Settings link to see a report of the MDM setup steps that you need to complete. The required steps (Figure 18-5 ) are:

Configure domains for MDM.

Configure an APNs (Apple Push Notification service) certificate for iOS devices.
Figure 18-5: Completing MDM setup for Office 365

Real World: If you have previously turned off DNS checking for the domain name in your Office 365 tenant then
you may see a green checkmark for “Configure domains for MDM” even though you have not configured the domain
for MDM. It is recommended to check your domain configuration even if you see a green tick for that item.

Configuring Domains for Office 365 Mobile Device Management

To configure your domain, navigate to the Domains section of the Office 365 admin portal, select your domain name,
and click the Domain settings link. Click the link to Change domain purpose , and then check the box to enable
Mobile Device Management for Office 365 .

Some additional DNS records, “enterpriseregistration” and “enterpriseenrollment” will be presented for you to add to
the public DNS zone for your domain name (Figure 18-6 ).

Figure 18-6: DNS records for Office 365 MDM

Add the new DNS records to your public DNS zone. If they won’t immediately validate in the Office 365 domain
management area, and you’re sure they are correctly configured, you can still proceed with the other Office 365 MDM
setup tasks and try to validate the DNS records again later when DNS propagation has had time to complete.

Configure an APNs Certificate for iOS Devices

In the list of required steps for Office 365 MDM setup click the Set up link for configuring an APNs certificate for iOS
devices. This is required because Apple device push notifications go through a service maintained by Apple, not
directly from a service or application in the cloud straight to the device. Your web browser will be redirected to a page
where you can download the certificate signing request (CSR) for provisioning the new APNs certificate. Download
the file to a safe location on your computer and click Next .

Next you need to sign in to the Apple APNs Portal to request the certificate (Figure 18-7 ). An Apple ID is required for
this task. Do not use an Apple ID that is owned by an individual; instead, you should create a new one that is
associated with an email address in the company. If you don’t already have a company Apple ID, take a few minutes
now to create one on the Apple web site and then return to continue the setup tasks.
Figure 18-7: Configuring Apple Certificates

After accepting the license agreement click the Browse button and select the CSR that you downloaded from the
Office 365 admin portal earlier, then click the Upload button.

Note: Depending on the web browser used, the upload might return a JSON formatted file that the browser
prompts you to download. You can download and save the file if you like, but this is not the certificate file that you
need.

You will receive a notification to the email address for your Apple ID when the certificate has been created. If, after a
moment or so, it seems like your web browser is stuck on the “Uploading…” page, simply refresh the Apple portal
page to see your available certificates. Click the Download button to download your new certificate as a PEM file.

Return to the web browser tab or window for Office 365 where you are configuring the APNs certificate, and click
Next to continue to the page to upload your new certificate. Or, if your browser session has timed out, simply start the
APNs process again and skip past steps 1 and 2 to begin step 3 of uploading the APNs certificate. Click the Browse
button and select your PEM file to upload.

The certificate install takes several minutes to complete. If your web browser appears to hang or time out on the
upload you can log back in to the Office 365 portal again, start the process of configuring the APNs certificate, and
just skip through to the step 3 again to upload the certificate you already have instead of requesting another new
certificate from Apple. If everything is in order you should see a green tick for the Mobile Device Management
settings and you can start managing MDM policies for Office 365.

Managing Device Policies in Office 365 Mobile Device Management

To manage Office 365 Mobile Device Management device policies, you will use the Security & Compliance Center . In
its Data Loss Prevention section under Device Security policies , you will see a list of device policies that are
already configured. (As of July 2018, this link doesn’t work or appear reliably in all tenants, but you can use
https://protection.office.com/?rfr=AdminCenter#/devicev2 to access it directly.) There are no default policies created
when you enable Office 365 MDM, so if this is your first look at this page then it’s likely you’ll see an empty list.

There are two types of controls that you can apply:

Organization-wide default policies.

Device policies targeted at groups of users.

Real World: Office 365 Mobile Device Management policies are targeted at security groups. Before you begin
configuring additional policies, you should probably create security groups to serve as policy targets.

Configuring Organization-Wide Policies

Click the link to Manage organization-wide settings device access settings (Figure 18-8 ). Here you can decide
whether to allow or block unsupported devices and applications from accessing email when they are targeted by an
MDM device policy.
Figure 18-8: Deciding whether unsupported devices can access Exchange

If your organization is going to require mobile device management for all connecting devices then you can consider
configuring this to Block . However, this will cause unenrolled devices to lose connection to email and other Office 365
resources almost immediately after the user is added to a group that is targeted by device policies.

It is also quite reasonable that you would not want to immediately configure a block. Perhaps your organization is only
testing and evaluating Office 365 Mobile Device Management and don’t want to disrupt existing users, or perhaps you
are allowing a grace period for users to enroll their devices in MDM before you begin blocking unenrolled devices.

At the time of this writing the user experience on devices that are blocked is not very friendly, which may result in an
increase in support calls to your help desk. Microsoft has tried to improve that experience by sending the user a
system-generated email when they are blocked. The email explains why the device was blocked and guides them
towards installing the Company Portal app to enroll their device. Despite this, it is likely you will need to support your
users through the enrollment process when they are blocked. Even if you do want to block unsupported devices, you
might still have some scenarios where a block is not desirable. For example, you might have a VIP user who has
received an exemption for their preferred device that happens to be unsupported. It’s natural that this kind of thing
should happen, but you should also consider that VIP users are often targeted by attackers because their devices
carry the most sensitive and confidential information. In most cases, it is more sensible to insist that all devices are
included in mobile device management and to resist the attempts of those who want to be excluded from these
policies.

If you are forced to compromise, you can instead specify one or more security groups of users whose mobile devices
are excluded from access control, rather than allow everyone to bypass it.

Note: Before you can add a group to this exclusion list you need to have created the security group in Office 365
already. Also be aware that the group picker will show no groups at first even if you have groups in your Office 365
tenant, and you need to do a search on a keyword before any groups will appear.

Creating a Device Policy in Office 365 Mobile Device Management

To create a device policy, click the “+ Create a policy” button in the Device Security Policies section of the
Security and Compliance Center. Next, give the policy a name and description. You may find it makes administration
easier if you name the policy to match the name of the security group it will be targeted to.

The first set of policy items will look familiar to anyone who has configured ActiveSync mobile device policies before.
They contain the most common settings that organizations worry about such as requiring a PIN/passcode, requiring a
minimum length or complexity for the PIN/passcode, inactive lock timeout, and device encryption.

You’ll also notice the setting to Prevent jail broken or rooted devices from connecting . This setting is not available in
ActiveSync policies, only in Office 365 MDM device policies, and it is a good idea to enable it as many exploits for
mobile device platforms only work against jail broken or rooted devices. You also have options for requiring
management of the email profile on the device. This is available on iOS only at this stage, and is what enables
selective wipe (only wiping the corporate email data, not any personal data).

If the user has an existing email profile on the device that profile will stop working and will need to be deleted so that
the managed profile can be applied.
Figure 18-9: Deciding whether to block or allow access to devices

As shown in Figure 18-9 , the final setting is to choose whether to block or allow access for devices that do not meet
the requirements of your policy (for example a PIN/passcode that isn’t long or strong enough). If the policy requires a
managed email profile and the user has not removed an existing email profile from the device that will also result in
the device being non-compliant.

Remember that anybody in the exception group you configured earlier will not be blocked by this setting. Also keep in
mind that users will be blocked for any non-compliance with your new policy, no matter how trivial that setting may be
in the overall scheme of things. It is entirely possible that devices will be blocked for reasons you might not anticipate
when you plan your deployment of Office 365 Mobile Device Management.

If you want to get a sense of how compliant or non-compliant your users’ enrolled devices are before you start
blocking the non-compliant devices, then you should set the action to Allow access and report violation (see Figure 18-
9 ).

The next page contains a set of additional security policies that you can configure for device backups, data leakage,
and application installs. Requiring encrypted mobile device backups is a common requirement to protect sensitive
data that may be copied to users’ computers. Blocking screen capture is one method of making copying or data
leakage more difficult (but not impossible). Application stores (such as the iTunes Store and the Google Play Store)
can also be blocked, however this will prevent the user from installing applications such as Office mobile apps after
they’ve enrolled their device.

Next you can decide whether to apply this policy now or later. If you choose to apply it now you will need to select at
least one security group that will be targeted by the device policy. Again, be aware that the group picker will show no
groups at first even if you have groups in your Office 365 tenant, you need to do a search on a keyword before any
groups will appear.

After you finish creating the new device policy it will have a status of Turning on… for several minutes, before it
finally displays a status of On when it is ready and active.

Enrolling Devices for Office 365 Mobile Device Management

The enrollment process for mobile devices will vary slightly depending on whether the user currently has applications
on their device connecting to Office 365 resources and whether the device policies force enrollment. Different mobile
device platforms will display different screens throughout the process.

Once you put users into a security group that is the target of a policy, they will automatically be prompted to enroll
their device in Office 365 MDM, or they can enroll manually. The user experience for this process varies somewhat
also. For example, consider Rese, a user who already has Outlook mobile installed on her iOS device. Once her user
account is placed into a security group that is subject to an MDM policy, the next time Outlook mobile attempts to log
on, it will fail, and she will see an error message. When she tries to log back in to the Outlook application, it will
display a notification that she must enroll her device in management, along with a link to download the Intune
Company Portal app to enroll the device. Alternatively, Rese can download the Intune Company Portal app from the
Apple App Store (Figure 18-10 ) and begin the enrollment process manually.
Figure 18-10: Intune Company Portal apps in the Apple (left) and Android (right) app stores

The exact enrollment process varies depending on the user’s mobile device operating system. Examples of the
enrollment process are available at the following links:

Office 365 Mobile Device Management enrollment for Apple iOS devices .
Office 365 Mobile Device Management enrollment for Android devices .

Removing or Wiping Mobile Devices

There are two ways for a mobile device to be removed from Office 365 Mobile Device Management:

The device is retired.

The device is wiped.

Devices can be retired by the end user by opening the Company Portal app on their mobile device and removing it
from management. Retiring a device wipes all corporate data and any management profiles from the device. After
opening the Company Portal app, the steps are slightly different for each mobile device platform:

On Windows Phone select the device then tap the trash can icon to remove the device.
On Android select the My Devices tab, select the device, and tap the trash can icon to remove the device.
On Apple iOS select the device and then click the “…” icon and then tap the Remove button to remove the
device.

Administrators can also remove devices by issuing a full wipe (or “factory reset”) or selective wipe (“remove company
data”) from the Mobile Devices section of the Office 365 Admin center (Figure 18-11 ). Note that this is not the
mechanism you use to request a remote wipe through Exchange ActiveSync. The effects of a full Office 365 MDM
remote wipe are
Figure 18-11: Selecting a device to be wiped

A full wipe is destructive, removing all data from the mobile device including the user’s personal data, and restoring it
to factory default settings. A full wipe would usually be used only in situations where a device has been lost or stolen.
On the other hand, a selective wipe only removes the corporate data from the device, leaving the user’s personal data
intact. This is the preferred option for common scenarios such as a staff member leaving the organization.

Renewing the APNs Certificate for Office 365 Mobile Device Management
The Apple Push Notification service (APNs) certificate that is issued by Apple will expire after 1 year. The email
address you use for the Apple account that requests the certificate will receive a notification email as the expiry date
is approaching. As mentioned earlier, you should ensure that the email address that is used for the certificate is one
that your organization owns (i.e. do not let anyone use their personal Apple account for this purpose), and should be
one that is monitored so that the expiry notifications are read and actioned. If you’re not sure about the expiry date
for your certificate, you can view it in the mobile device management section of the Office 365 Admin center.

A bug in the new admin center prevents the certificate renewal wizard from being accessed. If you’re using the new
admin center and experience this issue, you can revert to the old admin center to be able to perform the certificate
renewal. After accessing the mobile management section of the admin center, click Manage settings , and then
Set up for the APNs certificate.

The wizard will step you through the process of generating a new CSR, which is then uploaded to the Apple Push
Certificates Portal. Apple will issue you the new certificate, which will be a .PEM file type. If your browser prompts
you to download a .JSON file (which may have also happened to you during initial setup of MDM) you can ignore that
file. Once you have the .PEM file, complete the wizard in the Office 365 admin center.

Microsoft Intune
ActiveSync and Office 365 Mobile Device Management both offer useful mobile management capabilities for Office
365 customers. Many customers are happy with the mobile management functionality included in Office 365 as they
allow a reasonable level of control over mobile device features and configuration settings. Building on basic
ActiveSync capabilities, Mobile Device Management enables group-based targeting of policies, detection of jailbroken
devices, control of access from mobile devices and applications to non-email data such as documents in OneDrive for
Business, and minimal provisioning of configuration settings on devices.

Intune is the next step up in mobile management, and it is a large step at that. While ActiveSync can be used to apply
some control to email connectivity, and Office 365 MDM offers some added features, it is Microsoft Intune that is
designed to give comprehensive mobility management. In fact, Office 365 is actually a limited version of Intune,
running on the Intune engine and with only a small subset of Intune’s full feature list.

You can buy Intune as a standalone license, or as part of the Enterprise Mobility and Security (EMS) suite of services.
Microsoft claims over 45,000 customer organizations are using Intune, and that more than 40,000 customer
organizations have licensed EMS, and although Microsoft do not break down exactly how many of those customers
use Intune specifically, they claim to be 2x bigger than Airwatch and 3x bigger than MobileIron. Perhaps more
significant is their claim that less than 1% of Office 365 customers use a non-Microsoft solution for mobility
management.

Intune can configure mobile devices with email profiles, wireless network, and VPN settings, and can deploy
certificates to mobile devices as well. The certificate deployment capability is important for customers who want to
make use of certificate-based authentication for ActiveSync. Intune can also manage Windows PCs, which is
particularly useful for organizations that use a BYOD approach to mobility.

Intune's capabilities also extend into mobile application management. Both ActiveSync and Mobile Device
Management can prevent a device or application from accessing data. However, once the device or application can
access data, MDM solutions cannot exert any control over what happens to the data. For example, a user on a mobile
device can freely copy a Word document that they opened from OneDrive for Business and save it into their personal
Dropbox account. Intune's Mobile Application Management (MAM) can prevent the sharing of corporate data to
personal applications (Figure 18-12 ).

Organizations that use System Center Configuration Manager (SCCM) to manage their on-premises devices, must
decide whether to enable Intune as the mobile device management authority, or whether to integrate Intune into the
existing SCCM infrastructure. The integration of Intune and SCCM is referred to as hybrid MDM. Using Intune as a
standalone service has some advantages. As a cloud service, there is no infrastructure to deploy, making it easier to
get up and running with Intune than with a hybrid MDM solution. Standalone Intune also receives new features and
updates sooner than hybrid MDM.

In comparison, thanks to the integration with SCCM a hybrid MDM solution has more advanced reporting, and gives
the "single pane of glass" management for both mobile devices and regular PCs. Hybrid MDM is also able to manage
clients that connect only to the corporate network and do not have access to the internet. The SCCM user CALs are
included with your Intune subscription, but deploying the server infrastructure still requires a greater effort than
enabling Intune. However, Microsoft does recommend hybrid MDM if management of more than 50,000 devices is
required.

Figure
18-12: Configuring a MAM policy using the Azure-based Intune portal

The Intune management console originally began life as a Silverlight-driven interface. Thankfully, in 2017 Microsoft
moved Intune management functionality to the Azure portal. This change provides unified management of Intune with
other Azure services, which is becoming more important for Microsoft customers these days as they expand their
usage of cloud services. Microsoft has also added support for the Graph API, allowing customers to automate Intune
administrative actions and integrate them into other workflows and third-party systems. Microsoft uses the Graph API
to integrate Intune and the OneDrive admin portal. The device access policy that is configurable in the OneDrive
admin portal uses Intune application management capabilities to enforce restrictions such as blocking the copying of
data to other applications. Configuring a device access policy in the OneDrive admin portal does not need an Intune
license for your end users, and it is a single policy that will be applied to the entire organization. To get more granular
policy deployment you will need to upgrade to Intune licenses instead.

Preparing Intune for an organization

Before you can use Intune to manage devices and applications it must be configured as the Mobile Device
Management authority. The MDM authority defines how you will be managing devices and applications, and is
configured to one of these values:

Intune Standalone
Intune Hybrid (with SCCM)
Office 365 MDM

If your organization has not previously used an MDM then you can enable Intune as the authority without impact. If
you have been using Office 365 MDM then the Mobile Device Management authority can be changed to Intune
without affecting existing enrolled devices. In the Intune blade navigate to Device Enrollment and select Overview
(Figure 18-13 ).
Figure 18-13: Configuring the MDM authority.

If the MDM authority can be changed you will see a button to set the MDM authority. After you have configured the
MDM authority to Intune, whether it be standalone or hybrid, you will not be able to change it again without opening
a support case with Microsoft.

As with Office 365 MDM, an Apple APNs certificate is needed to manage iOS and MacOS devices. If you are
transitioning from Office 365 MDM and the APNs certificate is still valid, then no further steps are needed. For new
Intune deployments, you will need to start the APNs certificate provisioning process from the Intune console, found in
the Device Enrollment section under Apple Enrollment . The steps involved are the same as described earlier in
this chapter for Office 365 MDM. Also keep in mind that if you are transitioning all your users from Office 365 MDM
to Intune, you should remove your Office 365 MDM policies at the end of your transition project to ensure no conflicts
exist between the two different configurations.

Managing Intune users and groups

Intune uses the same identity model as other Office 365 services. To add a user to Intune, the user account must first
exist in Azure Active Directory, either by creating it as a standalone identity, or by synchronizing it from an on-
premises Active Directory. To enable a user account for Intune you need to assign an Intune license to the user.
License management is covered in chapter 7.

Intune uses groups for targeting of policies and application deployment. Azure AD security groups are used for a wide
range of cloud services and have enhanced capabilities such as dynamic group membership, and Microsoft wisely
moved Intune away from its original approach of using separate groups to using Azure AD groups. In general, you
don’t have to do anything special to manage users or groups from Intune: manage your users as you normally would,
then either select existing groups as Intune targets or create new groups, in the cloud or synchronized from on-
premises, and add users to them.

Managing Intune administrative permissions

Office 365 Global administrators are automatically made full administrators of Intune. There are two Intune-specific
administrative roles that you can assign to users through the Azure AD portal:

Intune Service Administrator holders have full control over all Intune features; they can also create and
manage Intune groups and manage users and devices, unless you prevent them from doing so by restricting the
underlying Azure AD permissions
Conditional Access Administrator holders can view, create, modify, and delete conditional access policies only.
They cannot manage users, groups, or devices.

Besides these top level permissions, in the Azure portal you can assign six more granular Intune roles from the
Intune roles > All roles blade of the Intune portal:

Policy and Profile manager - creates and deploys policies and profiles.
School Administrator – manages Windows 10 devices when you’re using the Intune for Education product.
Help Desk Operator - perform remote tasks for devices, such as unlocking, resetting, or wiping a device, and
view user and device information.
Application Manager - administer applications and app assignments, and read device information.
Read Only Operator - view only access to areas of the Intune portal in Azure.
Intune Role Administrator – allows holders to manage custom Intune roles and assign both custom and built-in
roles to users. This is the only role that can assign Intune administrator permissions.

When you assign a pre-defined role to an Azure AD security group you also choose a scope for the assignment. For
example, you can create an assignment that grants a group of IT staff in a branch location the ability to manage
application deployments only to users in that branch location, by limiting the scope to an Azure AD security group that
contains those user accounts. Custom roles can also be defined to suit your needs, choosing from a list of granular
permissions to grant to users who need to complete specific tasks.

Normally, a single user can only enroll five devices (see below for more on this limit). Intune supports a separate
Device Enrollment Manager (DEM) role that has no special permissions other than the ability to enroll up to 1,000
devices. This role is intended to ease bulk enrollment for schools or other environments that have very large numbers
of devices that can’t be self-enrolled, so most Office 365 customers won’t use it.

Device Enrollment
Intune has multiple policy types that have a cumulative effect on the ability of a user to access applications from
mobile devices. At the most basic level, different device platforms can be allowed or blocked based on the enrollment
restrictions that you configure. Enrollment restrictions are found in the Device enrollment section of the Intune
blade in the Azure portal (Figure 18-14 ). By default, all platforms are permitted, and you can choose to block any
platforms that you do not intend to support for your organization.

Figure 18-14: Device enrollment restrictions

You can also configure device limit restrictions, which defaults to a maximum of 5 devices per user. More than 5
devices may be required in some edge cases, but a sensible limit should always be applied. For large scale device
rollouts where one or a few individuals are provisioning devices for other users this limit should not be increased.
Instead a Device Enrollment Manager (DEM) account should be used to provision the devices. Also keep in mind that
increasing the device enrollment limit for users may result in them having more enrolled devices than they are
allowed to install Office 365 client applications on, which is limited to 5 installs per user.

Each device platform also has additional enrollment settings that can be configured. for example Apple enrollments
can be managed using Apple’s own Configurator tool, and Android devices can have Android for Work profiles
configured. These are optional approaches to corporate device enrollment and are not strictly necessary.

Devices are enrolled so that they can be managed and reported on. Although access to applications is possible without
enrolment, the management capabilities for a non-enrolled device are nearly non-existent. For example, a device that
has not enrolled can't be automatically configured using a device configuration profile to meet the organization's
security requirements.

Users can enroll with Intune using the Intune Company Portal app on iOS and Android devices, or by adding a
workplace account to a Windows 10 PC. For organizations with Azure Active Directory Premium licensing, Windows
PCs can be automatically enrolled when they are joined to Azure AD. For corporate owned devices, the enrollment can
be managed in bulk using a DEM account, or by using platform-specific bulk provisioning tools such as Apple's Device
Enrollment Program (DEP).

Relying on users to enroll devices is unfortunately going to create a support burden for your IT teams. You can create
instructions and provide them to users, but the enrollment steps tend to be confusing even though they are a linear
process. Some users will naturally be concerned about the privacy implications of enrolling a personal device in
Intune. The enrollment screens do spell out what access the IT admins get to a device that has been enrolled, but that
may not be enough to calm the fears of every user. At the very least you should be prepared to discuss the level of
control that enrollment provides (e.g. ability to deploy apps, enforce restrictions, and report on device compliance)
and some of the sensitive things that enrollment doesn't grant access to (e.g. the user's personal text messages or
other non-corporate app data).

Device Compliance Policies

Device compliance policies allow you to configure sets of rules for each platform that define what is considered a
compliant device. Compliance policies do not change the configuration of a user's device, they simply define the
criteria for compliance. If a device is non-compliant, the compliance policy itself doesn't enforce access restrictions.
Restricting access based on a device's compliance is the role of conditional access policies in Azure Active Directory.
Without any compliance policies in place you can configure conditional access rules that require compliance. But no
actual settings (such as a device PIN) will be required for an enrolled device to begin accessing applications until you
create a compliance policy.

Compliance policies are created in the Device compliance blade of the Intune portal. (Figure 18-15 ). This blade also
gives you a useful quick snapshot of the overall compliance state of your organization.

Figure 18-15: The Device compliance blade.

Each compliance policy is configured for a specific platform, such as iOS, Android, macOS, or Windows. Compliance
policies can define rules such as whether jailbroken devices will be blocked, minimum and maximum operating system
versions, and security settings such as PIN and passcode requirements (Figure 18-16 ).

Figure 18-16: Compliance policy settings for iOS device passwords.

After creating a compliance policy, it needs to be assigned to a security group in Azure Active Directory. When an
enrolled device does not meet the compliance policy that is assigned for that user, the device will prompt the user to
correct the settings that are not in compliance, such as setting a device PIN.

Device Configuration Policies

Intune can apply configuration policies to enrolled devices by enforcing profiles that define specific settings, block
apps or built-in functionality, or limit the user's ability to customize options. These profiles include a wide range of
options such as disabling camera apps, forcing the use of a PIN or passcode, or locking down a device so that it can
only run a single approved application. For a corporate owned device, the use of configuration policies makes sense,
because the organization is quite entitled to apply whatever restrictions they feel are necessary for those devices. For
BYOD devices, such enforcement may not be well received by the device owner. If the user makes their own personal
decision not to secure their device to the level required by the organization then they simply lose access to corporate
data from that device.

Configuration policies are created in the Intune blade of the Azure portal. You can configure profiles for:

Android and Android for Work

iOS
Mac OS
Windows Phone 8.1
Windows 8.1 and later
Windows 10 and later

Each platform has a list of profile types that can be created. For example, iOS allows you to create profiles for device
features, device restrictions, email configuration, certificates, VPN settings, and WiFi networks. Windows 10 has a
similar range of profile settings that includes features such as Cortana, the Edge browser, BitLocker encryption, and
Windows Defender. You can think of configuration policies as being similar to Group Policy. But unlike Group Policy,
Intune configuration policies can be enforced on non-Microsoft platforms, and can be enforced on devices that are not
joined to Active Directory domains.

Deploying Applications
One of the oldest uses of device management systems such as System Center Configuration Manager (SCCM) is to
automatically deploy software to end user devices. Software deployment worked well for agent-based systems like
SCCM that could download and run code on the device to install and configure the necessary software. Mobile devices
did not initially provide that same level of control, however these days it's possible for MDM solutions like Intune to
interact with devices, even those that are not domain-joined, to make app deployments possible.

App deployment for mobile devices saves users time by either automating the deployment of required applications, or
by providing lists of recommended applications for the user to install at a time of their choosing. All of this is possible
without the user needing to open an app store and search for the application itself. Aside from the convenience, this
also ensures that users install the correct version of the necessary applications and don't in advertently install
unwanted third-party clones or malware instead.

Intune provides a catalog of Microsoft mobile apps that are pre-configured and ready for assignment to end users.
When you assign a mobile app to a security group you can configure whether the app is available, or whether it is
required (Figure 18-17 ). The installation of a required app is initiated automatically on the device, although the user
is prompted to confirm the install.

Figure 18-17: Assigning a mobile app to users in Intune.

Of course, it's not only Microsoft apps that are used by businesses today. You might have a series of third-party
applications such as Dropbox or Slack that your organization uses instead of the Microsoft alternatives. You might
also have custom line of business (LOB) applications that need to be deployed (or side-loaded) onto mobile devices.
Each of those scenarios is also catered for in Intune app deployment. You can add an app to the catalog and either
provide the details of the LOB application, or search an app store for the application you want to add.
Figure 18-18: Adding a third-party application to the app catalog in Intune.

From the end user's perspective, the mobile app assignments are visible in the Company Portal app on their device.

Mobile Application Management

Although device enrollment provides the best experience for organizations who want to control and manage the
security and configuration of mobile devices, it's also possible to use Intune to enforce requirements for non-enrolled
devices. This is achieved by creating mobile application management (MAM) policies in Intune. MAM is sometimes
referred to as MAMWE (MAM Without Enrollment) because of the ability to control application access for Office 365
services without users having to enroll their device. This is appealing for users in BYOD scenarios who want to access
corporate data but not allow the organization to manage their personal device.

There are actually two parts to the full MAM process. One is that you can apply restrictions on which apps may
connect to Office 365 services. You apply these restrictions through Azure AD conditional access policies, which are
described in Chapter 3. The second is that you can control which apps users may install on their managed devices,
and what any MAM-aware application can do once it has connected to your Office 365 tenant.

Instead of enrolling, the Microsoft Authenticator app (for iOS) and the Company Portal app (for Android) are used as
authentication brokers. The broker app registers the device in Azure AD, which creates a device record in Azure AD
and a certificate for the device so that authentication tokens can be issued.

MAM only works with cooperating applications. Understandably it is Microsoft's core mobile apps such as Outlook
and OneDrive that are MAM capable out of the box. That doesn't mean that only Microsoft apps will be able to
function when MAM policies are in effect. Microsoft makes available the Intune App SDK to allow developers to build
Intune app protection capabilities directly into their apps. As an alternative, the Intune App Wrapping Tool can be
used to add the management layer that is required for MAM policies to be applied.

Controlling application access through Azure AD conditional access

At present, you can only use Azure AD conditional access to restrict access to Exchange Online and SharePoint
Online. In both cases, the mechanism you use to enforce control is to create a conditional access policy using the
Conditional access blade in the Azure AD management blade. As described in Chapter 3, you use this blade to
create a policy specifying which users and groups the policy should apply to, which cloud apps (in this case, Exchange
Online and/or SharePoint Online) it applies to, which conditions (such as device type or location), and so on. In the
case of MAM enforcement, the most important step is to ensure that the Require approved client app checkbox is
set on the Grant section of the policy. This will allow access to the specified applications only through approved
applications, which as of July 2018 means only Microsoft-branded applications (including OneDrive, Outlook, Planner,
PowerPoint, Skype, Teams, Visio, and Word). Once you create a conditional access policy with that checkbox set, and
then assign it to a group of users, those users will only be able to access the selected services through the MAM-
managed Microsoft applications.

Note: The list of allowed apps when configuring conditional access will sometimes be missing the latest applications
that Microsoft has released. Since most apps connect to Exchange Online or SharePoint Online in some capacity, this
can lead to newer apps being unable to connect when conditional access is configured. Microsoft generally releases
conditional access support for new apps within a few months of their full release.

A user who is not licensed for Intune and is not the target of a MAM policy will be subject to the ActiveSync policies
that are configured in the organization. After the user is licensed and added to a MAM policy the ActiveSync policies
will then be ignored for that user, and only the MAM policies will apply. For Exchange Online access Intune uses
ActiveSync to quarantine apps that are not permitted to connect so that the user can be delivered a system message
in their inbox notifying them of the quarantine action and advising them to use an approved application.
You can combine the use of ActiveSync and Intune policies to achieve your overall mobile strategy, for example by
configuring ActiveSync policies to block all device access to email, and then configuring Intune MAM policies to only
allowing Intune-licensed users to connect using apps that support MAM policies. While the cost of an Intune license
for the user might seem excessive for just that one feature, you should consider that Intune can then also be used to
manage any corporate issued devices for that user, including their Windows PC, as well as manage access to the
broader Office 365 service and not just to email. Furthermore, since Intune is often purchased as part of the overall
Enterprise Mobility and Security suite, you gain additional features such as Azure Active Directory Premium, Azure
Multi-Factor Authentication, and Advanced Threat Analytics.

Managing application settings through Intune

In the Azure portal, the Intune MAM policies are located in the Mobile apps blade in Intune. This area of the portal
has undergone a great deal of change over the last year or two; it is now supposed to be the single location for
managing applications through Intune. The most relevant tools in this blade for us are the ones that allow you to
publish applications and to create and manage app configuration policies and app protection policies.

Let’s start with publishing applications. You don’t literally publish them through Intune; instead, you specify which
apps from the platform-specific app stores for iOS, Windows, and Android you want users to see as featured or
required applications in the Intune Company Portal application. You can assign applications to AD security groups so
that users in those groups will be required to download them through the portal. Of course, that doesn’t force users to
actually use the applications, but there are other policy types that can do so.

App configuration policies allow you to force configuration settings onto applications that have been written to use the
Intune management SDK. When you create an app configuration policy, you pick one or more applications and then
specify a set of key/value pairs. The application is supposed to honor the settings you supply. However, as with making
changes in the registry, if you misspell a key (or make one up), or specify an incorrect value, the app will happily
ignore your changes.

App protection policies are much more interesting. You manage them through the Mobile apps blade of the Intune
portal. When you define an app protection policy, you specify a targeted set of users or groups that the policy applies
to, a set of targeted apps that the policy controls, and then a group of policy settings that you want enforced. The
policy settings are not application-specific, but their implementation is. For example, if you set the policy control
“Prevent ‘Save As’,” when that policy is applied to Word or PowerPoint it will do what you expect, but when you apply
it to the Skype for Business client, it will be ignored.

Figure 18-19 shows some of the policy settings that are available. For example, you can prevent data transfer into or
out of an app; when you apply this setting, the targeted app will try to honor it by restricting copy/paste, drag-and-
drop, and other methods of data transfer.

The fact that app protection policies allow you to require a PIN (and set strength requirements for it) showcase the
difference between MDM and MAM. If you set a PIN policy using MDM, it applies to the device. If you set it using an
app protection policy, it only applies to the targeted application, so you can apply it and know that people will (e.g.) be
forced to set and use a PIN with Outlook and OneDrive even if, for some inexplicable reason, they have chosen not to
put a PIN on the device itself. The security result is roughly the same: unauthorized users can’t get into Outlook or
OneDrive. However, the implementation and user impact may be significantly different.
Figure 18-19: Adding a third-party application to the app catalog in Intune

Implementing Intune Solutions

Microsoft Intune has a broad set of capabilities for managing mobile device and application access. As such, there is
not a single implementation approach that will work for all organizations. Intune is definitely a solution that needs to
be designed to meet the needs of the organization, rather than applying a generic approach.

Intune deployment projects can be scoped by answering just a few questions during the planning stage:

Who are the users that need access to corporate data, and what are their use cases?
What are the devices that users will be using to access corporate data? This includes decisions about which
device platforms will be used, as well as corporate vs BYOD device ownership.
What are the data access requirements for corporate data, for example whether corporate data can be copied to
unmanaged applications?
How will corporate data access be handled for devices that are unmanaged, or that are managed by a third-party
MDM solution?

By answering those questions, you can develop the overall Intune solution to meet your needs. Not every feature of
Intune is going to be necessary for every organization. For example, if only managed devices will be allowed to access
corporate data, then MAM app protection policies will not be required, and you can simply enforce conditional access
for enrolled and compliant devices instead. In contrast, an organization that wants to allow basic email access from
BYOD devices can use MAM policies to require unmanaged devices to use managed apps, and then configure
conditional access to require full device enrollment to access other Office 365 services.

You should also make use of test and pilot groups to ensure that the policy configuration you have designed delivers
on the goals that you have for the organization. When making changes it's also important to remember that Intune is a
complex system that has multiple overlapping policy areas. From the user's perspective, it's often unclear whether
access is being prevented due to an app policy, a conditional access rule, or another setting within Intune. Similarly,
you should take care when making changes to ensure that enough time passes for the result of the change to be
assessed. Many of the configurations within Intune do not have an immediate effect on users and devices due to
various caching and propagation delays. Administrators who are familiar with System Center Configuration Manager
(SCCM) will already have the requisite patience for working with Intune.
Moving on from mobility
A major reason why organizations deploy of mobile device management is to protect corporate data from misuse or
loss. Data governance is a broad topic that extends well beyond mobile access, so let's explore it in more detail in
chapter 19.
Chapter 19: Office 365 Data Governance
Tony Redmond

Compliance means different things to different people. In the world of Information Technology, compliance is either a state
of following established guidelines, specifications, legislation, or the process that a company takes to achieve compliance.
Legislation varies from country to country. Well-known examples in the United States include the Sarbanes-Oxley Act (SOX
2002) or the Health Insurance Portability and Accountability Act (HIPAA – 1996). Companies working in these industries
must ensure that their IT systems store and keep data in compliance with the regulations laid down in legislation. The
General Data Protection Regulation (GDPR) is another example of how regulation exists to compel companies with
businesses in the European Union to protect personal data properly.

Microsoft began its journey to deliver compliance features in Exchange 2010. Since then Microsoft has delivered an array
of features broadly grouped into four categories as shown in Figure 19-1 . Although not every Office 365 workload supports
the compliance functionality, coverage is spreading. The aim is to cover all Office 365 workloads where users generate
information that organizations might need to keep for business or regulatory purposes. Collectively, these categories give
organizations a framework for “data governance.”

Figure 19-1: Aspects of compliance

A data governance framework helps organizations to satisfy regulations and to apply standards. Technology helps users to
be compliant, but only if that technology is easy to use and unobtrusive. Experience proves that trying to implement
difficult-to-use or complex compliance technology is not a recipe for success. Building compliance into systems like Office
365 in an intuitive and unobtrusive manner is challenging, especially as the volume, complexity, and sources of the data
that needs governance increases all the time. However, without good governance, organizations cannot meet compliance
and regulatory requirements and expose themselves to the consequences of data leakage or external threat.

Data Governance in Office 365

When Microsoft launched Office 365 in June 2011, the different applications in the suite offered differing and limited
compliance capabilities. Because each application had its own approach to compliance, simply bringing the on-premises
functionality to Office 365 resulted in a fragmented and disjointed situation. Some data have protection, and some do not.
Some data are subject to retention policies and some are ignorant of policies. Some data comes within the scope of
eDiscovery and some are invisible to searches. And so on.

Another complication is that some of the Office 365 applications (like Yammer) have never had a good compliance story.
Still another comes through the introduction of new applications like Teams and Planner, which create new sets of data
that might be of interest from a compliance perspective. In short, the old approach of depending on workload-specific
compliance functionality had run its course and Office 365 needed a new cross-service architecture. Microsoft began
rolling out the new Office 365 Data Governance framework for Office 365 in April 2017. To emphasize its cross-service
focus, administrators access the functionality that makes up the framework through the Security and Compliance Center
instead of the workload administrative portals.

Data Governance is more than archiving email. In the context of Office 365, data governance means the application of a
set of consistent policies to keep necessary data and remove unwanted data across all Office 365 workloads – Exchange,
SharePoint, OneDrive for Business, Teams, Yammer, Sway, Skype for Business, and so on. In short, any item that comes
into Office 365 or users create within Office 365 should be subject to the governance policy set by the organization.
Depending on the business requirements that exist within an organization to preserve and protect information, the
individual compliance features available in Office 365 will be less or more important. Fitting them together into a coherent
plan is a major task that needs input from Information Technology, Human Resources, and Legal personnel in addition to
overall direction and support from senior management. As the regulatory and legal frameworks that govern compliance
vary extensively from country to country and industry to industry, the treatment of the topic found in this chapter is by
necessity at a reasonably high-level. However, it should be enough to give guidance as to what is and what is not possible
when you work with Office 365.

The solution is the implementation of a new data governance framework for Office 365. The new framework can handle
the core workloads (Exchange and SharePoint), some variations of those workloads (Office 365 Groups and OneDrive for
Business) and creates the foundation to extend into other workloads like Yammer when their developers do the necessary
engineering work to support data governance. As Microsoft says, the idea is to “keep what you want and get rid of what
you don’t,” which is a good way to summarize what data governance is all about.

Workload-specific functionality still exists within Office 365. Customers still need functionality like the SharePoint
eDiscovery Center and Exchange mailbox retention policies to manage information stored on-premises servers, and you
can transfer the Exchange mailbox retention policies and retention tags from an on-premises organization to Exchange
Online using the Hybrid Configuration Wizard to ensure that the same policies are used on-premises and in the cloud.
However, the smarter longer-term approach is to move away from workload-specific processing to policies that apply to all
workloads if and as you can. Microsoft’s focus for Office 365 is on the Data Governance framework and not on workload-
specific processing, so that’s where you can expect new features and functionality to appear.

Keeping Content in Place

Traditionally, when a company needed to preserve email or documents for compliance purposes, they move copies of the
items to a separate archiving system. Veritas Enterprise Vault is a classic example of such an archival system. When
necessary, administrators run eDiscovery or retrieval activities to find content held in the archival system. This approach
was acceptable when products like Exchange and SharePoint did not have their own archiving capabilities. Today, the
focus is on leaving data in place within its native repository and integrating compliance technology inside applications.
Advantages of this approach include:

Copies of messages, documents, and other content do not have to be transferred to other systems to be searched,
analyzed, or otherwise interrogated for legal reasons.
Cost for software and hardware is avoided by not having to use added systems. In addition, complexity is removed
from the IT infrastructure.
It is easier to prove the “ chain of custody ” and show that no one can tamper with email and documents before they
are used in legal cases.
Not moving content from its native repository means that security is continuously applied and no opportunity exists
for content to be compromised as it is transferred to a new repository.

Removing the need for copying files from one system to another makes it easier to meet chain of custody requirements
and prove that no one could have interfered with an item to compromise its contents. Guaranteeing the chain-of-custody is
important when data is used to prove facts during litigation. In-place archiving means that information stays in its original
locations but is protected against interference for as long as the hold persists. This creates data that can be proven to be
immutable as neither administrators nor users can interfere with it after a hold is in place.

Keeping everything in place inevitably means that repositories become larger. Software therefore must be more intelligent
and more capable to deal with the larger volume of data. Searches must be able to find information quickly and accurately
when needed for compliance purposes. The work done by Microsoft Research to improve and make search technology
smarter together with the acquisitions of FAST and Equivio are examples of how Microsoft has responded to this need.
FAST (otherwise known as the Search Foundation) indexes information as soon as it is in Office 365 via user activity or
bulk imports while Equivio is part of Advanced eDiscovery (part of the E5 plan) and is capable of processing millions of
messages or documents to find the desired results very quickly. Another advantage is that when you have a single set of
data, it is much easier to build the necessary infrastructure around that data to secure and audit its use. The Office 365
Audit Log, for instance, records user interaction with data across multiple workloads which administrators can then
search using the Audit log search in the Security and Compliance Center, Office 365 Advanced Security Management, or
third-party tools such as Quadrotech Radar for Security and Audit.

The Value of Unity : Applications like Delve, MyAnalytics, and anything else built on top of the Office Graph are much
more productive and valuable when data stays in place inside Office 365. These applications have access to the full
spectrum of user activities rather than a subset, which might be the case when some interactions flow through external
systems. Overall, Microsoft’s position is that their approach ends up with a single set of data, accessed by clients like any
other data using an infrastructure that is both cheaper and easier to manage. It is a compelling vision in the eyes of some.
Others might consider that it is just too much of putting all your eggs in one basket.

Principles of Data Governance

Office 365 takes a cross-service approach to data governance based on common policies that apply across multiple
workloads. This approach is obvious the way that content searches, eDiscovery, and Data Loss Prevention work, and is
rolling out for data governance in the form of Office 365 labels, retention policies, and associated features. Although you
could be critical of Microsoft and wonder why it has taken them since 2011 to figure out how to manage data for
compliance purposes within Office 365, it is fair to say that an overall Data Governance strategy has been coming together
since 2015. The Security and Compliance Center is a key part of that strategy because it offers a single administrative
interface to compliance functionality that is deliberately designed to work across all the Office 365 applications rather
than just a single workload.

Office 365 Data Governance is a policy-driven framework to control the retention and removal of content across all Office
365 applications. We can break the framework down into four parts:

Import : Bring information from multiple sources, including from non-Microsoft sources like Facebook and
Bloomberg messaging, into Office 365 through the Import Service or third-party products. The aim is to allow
customers to gather all the information they need to manage in Office 365 repositories such as archive mailboxes
(PST replacements) or SharePoint and OneDrive sites (file server replacements). Once the data is collected, it can be
managed in a consistent manner without the need to duplicate policies or ever move the data again.
Retain : Provide customers with the ability to state what data they need to retain for compliance purposes, for
whatever period is required. The policies used to enforce retention must apply across all parts of Office 365 (or as
many as possible).
Delete : It is equally important for customers to be able to remove data that they no longer need. Methods need to
exist to remove information from all Office 365 repositories according to policy.
Classify : Some data is more important or secret than others. The framework must allow users to classify information
as they work. Applications should highlight information classified by users to raise awareness of the importance of
the information. The system should recognize the sensitivity of the information and potentially restrict what users can
do with that content.

Of course, when data is kept for retention purposes, it must also be immutable. The Office 365 Import Service is the
cornerstone for “Import,” Office 365 retention policies and retention labels handle make sure that tenants can keep or
remove content as they need and classify information to mark important items to be kept for specific purposes. Other
aspects of Office 365, like Data Loss Prevention, handle the need to protect against the misuse or inadvertent disclosure of
data.

The Influence of GDPR

From May 25, 2018, companies with business operations inside the European Union must follow the General Data
Protection Regulation (GDPR) to safeguard how they process personal data “wholly or partly by automated means and to
the processing other than by automated means of personal data which form part of a filing system or are intended to form
part of a filing system. ” The penalties set for breeches of GDPR can be up to 4% of a company’s annual global turnover.
For companies like Microsoft that have operations within the EU, making sure that IT systems support GDPR is critical.

Because many Office 365 applications can store data that might come under the scope of GDPR, the regulation has a
considerable influence over how tenants deal with personal data. The definition of personal data is “any information
relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be
identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number,
location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental,
economic, cultural or social identity of that natural person .”

GDPR goes on to define processing of personal data to be “any operation or set of operations which is performed on
personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation,
structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or
otherwise making available, alignment or combination, restriction, erasure or destruction .”

In effect, individuals have the right to ask companies to tell them what of their personal data a company holds, to correct
errors in their personal data, or to erase that data completely. Companies need to know what personal data they hold,
make sure that they obtain consents from people to store that data, protect the data, and notify authorities if data
breaches occur. They also need to be able to handle obligations under GDPR articles, such as data subject requests (article
15) and requests for erasure (article 17).

The definitions used by GDPR are quite broad. To move from the theoretical to practicality, an organization needs to
understand what personal data it holds for its business operations and where they use the data within software
applications. According to Microsoft, over 90% of an organization’s data is stored in Office 365 is authored in Word, Excel,
PowerPoint, OneNote, and Outlook (Exchange). It is easy to imagine examples of where personal information might be
stored in Office 365, including:

Annual reviews written about employees stored in a SharePoint or OneDrive for Business site.
A list of applicants for a position in an Excel worksheet attached to an email message.
Tables holding data (names, employee numbers, hire dates, social security numbers, salaries) about employees in
SharePoint sites.
Word documents holding applications for employee work visas.

Given that personal information can be found in almost any application, the work to locate the 10% of personal
information stored outside Office documents is likely to take the most effort (for example, see this guidance for PowerApps
). Organizations must know what applications hold personal data and how that personal data is processed to comply with
GDPR. Knowing where to look is only the start of the process and it might not be possible to automate many of the steps
involved in responding to data subject requests.

Fortunately, the work done inside Office 365 in the areas of data governance and compliance help tenants to satisfy the
requirements of GDPR and prepare to be better able to meet their commitments under the regulation. These features
include:

Sensitivity labels and policies to protect (encrypt) and mark content holding personal data.
Auto-label policies to find and classify documents holding personal data as defined by GDPR. Office 365 includes
sensitive data definitions for many European country-specific personal identity cards and passports. In addition, a set
 of five common European sensitive data types (passport, national identification number, driving license, debit card
number, tax identification number, and social security) are available. You can create an auto-label policy to find and
label documents and messages holding these sensitive data types. Retention processing can remove items stamped
with a GDPR label after a defined period, perhaps including a manual disposition step.
The same sensitive data types can be incorporated into Data Loss Prevention (DLP) policies to stop people sharing
sensitive data outside the tenant. A General Data Protection Regulation DLP template makes it easy to deploy
protection for all the defined European Union sensitive data types.
Content searches to find personal data marked as coming under the scope of GDPR. You can search for content
stamped with retention labels used to mark items holding personal data.
Alert (or advanced alert) policies to detect actions that might be violations of the GDPR. For example, multiple
downloads of documents from a SharePoint site holding HR information. You can also search the Office 365 audit log
to discover and report potential GDPR issues.
Azure Information Protection to protect documents and spreadsheets holding personal data so by applying RMS
templates to encrypt the content, ensuring that unauthorized parties cannot read the documents.
A GDPR dashboard within the Security and Compliance Center gives extra visibility to different components that
tenants can use to meet GDPR requirements, including the ability to create special eDiscovery cases for data subject
requests (DSRs).

In addition, Microsoft has published information about how various aspects of Office 365 align with GDPR (as an example,
see this post covering SharePoint Online and OneDrive for Business ).

This chapter covers retention labels and retention label policies. Chapter 20 covers content searches, Chapter 21 covers
auditing and reporting, Chapter 22 covers Data Loss Prevention, and Chapter 24 covers sensitivity labels and rights
management. Having technology available to help satisfy the requirements of GDPR is helpful. However, it is only one step
to achieving full compliance with GDPR. Tenants still need to protect data through a mixture of user education and
technology. See this paper for information comparing the terms used in GDPR with those used in U.S. eDiscovery contexts.

Right of Erasure
A further challenge is posed by article 17 of GDPR (the “right of erasure”), which says: “The data subject shall have the
right to obtain from the controller the erasure of personal data concerning him or her without undue delay.” In other
words, someone has the right to demand that an organization should erase any of their personal data that exists within the
company’s records. Content searches can find information about someone using their name, employee number, or other
identifiers as search keywords. However, the risks of erasing information not relevant to the data subject is so high that a
manual check of all content retrieved by searches is invariably necessary to ensure that the tenant removes or redacts the
right data and just that data.

Compliance Manager
Microsoft’s Compliance Manager is a tool to guide compliance managers through the often-confusing array of rules
involved in frameworks like the GDPR. Compliance Manager is a static tool in that it does not have the necessary features
to organize the work needed to achieve compliance with complex regulations like GDPR. However, Office 365 has many
different tools that tenants can use to organize and manage the work set down in Compliance Manager. For example,
Planner can track progress to completion for the tasks assigned to those working on GDPR. To gain some collaboration
capabilities, including document management, Planner can be integrated with Teams or Office 365 Groups.

Retention Policies and Label Policies

Two types of policy exist within the data governance framework that you can mix and match to satisfy the specific
requirements of a tenant:

Label policies publish retention labels to users. Settings within the retention labels control what happens to content
marked with the labels. The typical scenario is that users mark items that they wish to remove or keep because of a
certain business need. For example, you might apply a “Keep for Audit” label to a document to keep it for ten years.
In many respects, a label works in the same way as a personal retention tag in an Exchange Online mailbox. You can
apply retention labels to items using automatic processing, but only if tenants have E5 licenses. See the section about
Auto-label policies later.
Retention labels are precise because users select and apply the labels to specific items. Office 365 retention
policies deliver broader coverage by applying retention settings to all items stored in selected locations (mailboxes,
sites, or groups). Background processes such as the Exchange Managed Folder Assistant then process the retention
settings defined in the policies. Tenants can use retention policies to preserve or remove content for the entire
organization or for up to specific groups of up to 1,000 accounts. Typically, tenants use retention policies to ensure
that a company meets the compliance requirements set out in legislation such as the Sarbanes-Oxley Act. Unlike
Exchange retention policies, you cannot use Office 365 retention policies to move items from primary mailboxes into
archive mailboxes.

You create and manage retention labels under the Classifications section of the Security and Compliance Center. When
ready, you publish sets of retention labels in label policies, which show up as retention policies under the Data Governance
section of the portal. Switching between sections in the Security and Compliance Center seems a tad confusing, but it all
comes together in the framework. Think of it this way: retention labels control content at a precise, item-specific level.
Retention policies offer broad-brush coverage of content at volume. Together, the mixture of specific and general control
affords tenants flexibility in how they build a data governance strategy for the organization. The same background
processes make sure that the settings specified in retention labels and retention policies are executed for the content to
which they apply.

In the rest of this chapter, we will explore the details of the Office 365 data governance framework and how to create and
manage retention labels, retention label policies, and retention policies. We will also examine how Exchange Online
retention policies work because of the ongoing need to support hybrid environments.

Rules of Retention
The many types of data used within Office 365 create a complex environment for retention management. Multiple ways
exist to mark content for retention or deletion. Therefore, a need exists for a mechanism to resolve the conflict that can
occur when several policies apply to a mailbox or site, especially when items might be removed following the end of the
set retention period. Microsoft applies the following four rules of retention to decide how to process content. The rules
work from top to bottom and are used as tie-breakers to set precedence if multiple policies apply to content.

1. Retention wins over deletion: You could call this the “keep safe” principle. In practical terms, it means that when
Office 365 checks policies that apply to content, it respects the longest retention period. Take the example of where a
mailbox comes under the scope of two retention policies. The first removes all messages after they are four years old.
The second, perhaps applied to a subset of mailboxes belonging to senior managers, mandates that all messages
should be kept for seven years. The solution is to move messages into the Recoverable Items structure after four
years and to keep them there for three more years. Both policies seem to be respected because users do not realize
that the messages are still available, but the longer retention period ensures that the messages stay indexed and
discoverable for the full seven years.
2. Longest retention period wins : If multiple policies specify different retention periods, Office 365 always keeps
items for the longest period. This principle ensures that content is kept for as long as it might be needed. If you want
to deliberately remove content after a certain period, deploy a policy that explicitly removes the content after that
period elapses and make sure that the location holding the content does not come within the scope of any other policy
(Exchange mailbox retention policy or Office 365 retention policy). Note that Office 365 will keep content for longer if
a user applies a label with a longer retention period to it. This is because the longest retention period always wins.
3. Explicit wins over implicit : Explicit means that a user or administrator has selected specific content for special
retention. This can happen when a user applies a label with a retention action to an item in a mailbox or site, or when
a location is included in a “non-org wide” policy (one that only applies to some content within the tenant). The logic
here is that the user or administrator has made an explicit decision about retention and that should always have
precedence over a policy that applies catch-all retention for a complete mailbox or site, including when retention
labels are applied automatically by reference to a query or because content holds a sensitive data type. This principle
has long existed in Exchange retention policies where a personal retention tag applied manually by a user always has
precedence over a folder tag or default tag.
4. Shortest deletion wins : Office 365 retention policies allow administrators to actively remove content after a certain
period (which is still called the retention period). If multiple policies apply with different retention periods, Office 365
uses the shortest period and removes the content when that period expires. The logic here is that an administrator
decided that content needed to be removed after a certain period. The presumption is that good reason guided this
decision. It would therefore not make sense if another policy, perhaps created by another administrator, interfered
with the decision to remove content after that period. Again, this is a reason to consider how the retention policies in
a tenant interact with content.

Remember that holds always take precedence over deletion. If an eDiscovery case places a hold on content for a set
period, Office 365 keeps that content until the hold period expires or the hold is lifted, even the retention periods of all
applicable retention policies are long gone.

It often takes time and some experience of working with retention policies across different Office 365 workloads to
understand the effect of the retention principles and how these principles can be best used to support the data governance
strategy for the organization.

Office 365 Retention Policies

Since Exchange 2010, mailbox retention policies have let administrators configure and apply policies to on-premises and
cloud mailboxes to help users control items through a mixture of system-controlled tags and personal tags. Actions
specified in the tags control how long items are kept in the mailbox and what happens once their retention period expires.
SharePoint has its own variation on the theme in document deletion policies, which remove items from sites after a certain
age. The workload-specific policies work well in their own way. Microsoft has gained a lot of experience in how customers
use policies to manage content since 2010. All of which leads to the introduction of Office 365 retention policies to deal
with

Exchange (mailboxes and cloud-based public folders).

SharePoint Online (including files owned by Yammer, Teams, and Office 365 Groups).
OneDrive for Business.
Skype (IM conversations).
Teams (compliance records created in group or personal mailboxes).
Office 365 Groups (conversations in the group mailbox).

Retention policies apply settings to control how Office 365 removes or keeps content. The basic settings are a retention
action, which defines what Office 365 does, and a retention period, which tells Office 365 when to execute the action.

Broad and Narrow Retention

Retention policies can be broad or narrow in scope. A broad retention policy applies the same retention settings to a set of
containers or locations. Each workload has its own locations: Exchange has mailboxes (user, shared, and group),
SharePoint has sites, OneDrive for Business has accounts, and Teams has teams. The broadest retention policy is one that
applies the same retention settings to all the locations managed by all workloads in a tenant. As described below, this is an
example of an org-wide policy.

Narrow retention policies make retention labels available to workloads. Each retention label has its own retention settings
and a retention policy for labels can have just one label or include many labels. Publication means the process of making
workloads like Exchange and SharePoint aware of the existence of the labels in a retention policy. Following publication,
the workloads expose the retention labels in client interfaces so that users can then apply the labels to selected content.
The application of a retention label to a message or document is a much more precise way to keep information. As such,
retention labels trump the settings applied by broad retention policies. For instance, you create a broad retention policy to
assign a five-year retention period to all SharePoint sites. All documents stored in a document library inherit the policy
and SharePoint will keep documents until they are five years old. If a user then assigns a retention label with a ten-year
retention period to a set of documents, SharePoint will keep those documents for that period.

Retention Policy Scope

The scope of a retention policy dictates the set of locations to which Office 365 applies the retention settings, two types of
Office 365 retention policies exist:

Org wide : An org-wide policy applies to all supported locations across Office 365. There can be up to 10 of these
policies in a tenant. You should use Org-wide policies sparingly as it is easy to create a policy that has unintended
consequences. For instance, if you create a policy to keep all content for two years and then remove the content
afterwards, the policy will remove everything from the tenant that is more than two years old and does not come
under the control of another policy. That could lead to the removal of most content from user mailboxes and sites,
which might not be what you want to do. Org-wide policies are also called “entire location” policies because they
cover all the locations within the selected workloads (for example, all Exchange Online mailboxes).
Non-org wide : These policies apply to a selected subset of the locations available within a tenant. Selection can
happen by picking specific mailboxes, sites, or groups to include in the policy or by applying a query based on
keywords or sensitive data types to find the content to which the policy applies. You can have up to 1,000 of these
policies in a tenant, each of which can span up to 1,000 mailboxes and 100 sites. If you include Skype for Business in
a policy, you should be aware that you need to add each account to the policy. You can include up to 1,000 accounts in
a policy.

With a thousand focused and ten organization-wide policies to play with, you can get very creative with your retention
strategy. In general, it is best to simplify retention processing by limiting the number of active policies as this will help to
avoid situations where the actions and retention periods of policies clash. See the Principles of Retention section earlier in
this chapter to understand how the Office 365 decides what to do when several policies apply to an item.

Automatic Policies
Office 365 also supports automatic processing for retention settings with both broad and narrow scopes. You can define:

Advanced processing for retention policies applied to locations.

Auto-label policies to apply retention labels to selected items.

These are both Office 365 E5 features.

Automatic processing relies a content search or selected sensitive data types to find items. A content search uses
keywords and qualifiers to find items, while if you use sensitive data types to find items, you select from the set of
sensitive data types known to Office 365 (like credit card and passport numbers). In either case, once Office 365 finds
data, it applies the retention settings from policy or assigns the label contained in the policy to the items. For example, a
policy set up to use advanced processing could have a content search to look for any message or document holding a
certain phrase (keyword) and apply a five-year retention label to any items returned by the search.

The Effects of Retention Policies

It's important to understand how Office 365 content might come under the influence of retention settings through broad
policies or the more precise application of retention labels:

Broad retention policies that define retention settings for locations (selected or org-wide) are invisible to users. Office
365 processes these policies in the background and users are unaware that retention is in force. This assignment is
implicit because an item inherits the retention policy because of where it is stored instead of having a retention label
assigned by a user.
Narrow retention policies that publish retention labels to locations make those labels available to users when they
work with content in those locations. After publication, a user can assign one of the published retention labels to a
document or mail message. This assignment is explicit because it takes a deliberate action by a user to assign the
label.

An item in a location can come under the control of a retention policy in these ways:

Someone assigns a retention label to an item. This explicit assignment has the highest-priority. Remember that a label
used purely as a visual marker does not have a retention action or period.
A location comes within the scope of a broad org-wide or non-org-wide policy, including policies that use advanced
processing to find content. The items take the retention settings from the policy unless someone assigns a retention
label to the item.
The owner of a SharePoint site defines a default retention label for a library. The label is inherited by new items
created in the library and can be assigned to all existing items. This label overrides any org-wide or non-org-wide
retention policy assigned to the location.
 An auto-label policy finds items based on keyword or sensitive data type and assigns the label contained in the policy.
An auto-label policy never replaces a label if one is already assigned to an item by a user or because it is the default
label for a library.

Users can change the retention label assigned by themselves or another user or inherited because it is the default label
defined for a library or assigned by an auto-label policy. They do not see or cannot affect the retention policy assigned to a
location.

If multiple retention policies apply to an item, Office 365 always favors an explicit assignment over an implicit assignment.

Permissions
Only members of the Compliance Administrator role group for the Security and Compliance Center can create or manage
retention policies. By default, this includes tenant Global Administrators.

Where did preservation policies go? Released in 2016, preservation policies were the first generation of retention
policies to cover Exchange Online and SharePoint Online. Retention policies are more powerful than preservation policies
because they function across all Office 365 locations to keep and remove content based on policy settings. Office 365
converts any preservation policies that existed in a tenant to retention policies. Converted policies only keep content, but
you can update them to remove content if this is necessary.

Planning an Office 365 Retention Policy

Before beginning the process to create a new retention policy, it is sensible to write down the basic structure of the policy,
broken down into four headings:

Broad or Narrow : Do we need to apply default retention settings to some or all locations, or publish retention labels
to allow users to dictate how content is kept? If the policy uses retention labels, what are those labels?
Scope : What Office 365 locations will this policy cover? If the policy targets a subset of locations within a workload
(for instance, ten mailboxes or five SharePoint sites), write down those locations. If you want to exclude a subset of
locations from the policy’s scope, you should also note those locations.
Purpose : Will the policy keep or remove content? If so, how long will the retention period be?
Type : Is this an org-wide or non-org wide policy? If non-org wide, is the targeted content found using a query or
because it is a certain sensitive data type?
Lock : Is the content that comes under the scope of the policy considered to be a formal record for the company and
if so, is a preservation lock needed? (See the section on Preservation Locks).

You can use an email distribution group or a mail-enabled security group (but not an Office 365 group) to define the subset
of mailboxes that a policy covers. However, the expansion of the group into individual members is a one-time operation.
For the example used here, the conditions listed in Table 19-1 apply.

Heading Policy setting

Scope The policy applies to mailboxes and OneDrive for Business sites belonging to senior managers, defined as being
members of the Senior Leadership Team (SLT) email distribution list. It also covers the content held in the SLT
group. This policy also covers the Skype for Business IM conversations stored in SLT mailboxes. Teams
conversations are stored in the group mailboxes owned by the relevant teams.

Purpose The policy keeps all information in the targeted locations for ten years and then removes the content.

Type This is a “non-org wide” policy. The group membership dictates the accounts to which the policy applies.

Lock Preservation lock is not needed.

Table 19-1: Planning an Office 365 retention policy

It is bad to find yourself in a situation where you create and deploy a retention policy only to discover that the policy
removes needed information. For example, it is very easy to apply an aggressive retention policy to “All Exchange
mailboxes” that removes items after 30 days. This is the equivalent of a default delete tag in an Exchange retention policy
to remove content after 30 days. When stamped on a mailbox, the Exchange Mailbox Folder Assistant (MFA) applies the
“Delete and Allow Recovery” retention action to all items in the mailbox that do not have a more explicit tag. After items
reach 30 days old, the MFA moves them to the Recoverable Items folder and preserves them for a further period (the
deleted items retention period). When that period expires (usually 14 days), MFA permanently removes the items from the
database. Exchange Online does not use backups, so you cannot recover the items at this point. Because retention policies
affect the content stored in user mailboxes, it is only sensible to consider and understand exactly what will happen when a
retention policy is active.

It is also possible that an Office 365 retention policy will clash with an Exchange mailbox policy or another Office 365
retention policy. It is a good idea to take some of the target locations and work out what policies have those locations
within their scope to figure out if a clash occurs.

No Change for Retention Action or Period

A crucial factor to consider when planning the implementation of Office 365 retention policies and labels is that you
cannot change some of the important settings that control how the policy functions after creation. For example, you can
alter the retention period for a policy, add new locations to its scope, and alter the KQL query to find content for the policy
to apply. However, you cannot change its basic operation. For instance, you cannot change a policy that keeps content into
one that simply removes content.

The logic is that users expect consistency in how their data is processing. If you can change the fundamental operation of
how retention works inside a tenant, users will not know whether their data will be kept or removed or when this will
happen. For this reason, it is wise to take time to chart out how retention will work across the tenant for all workloads
before you create any policies. Fools rush to implement retention without thought!

Naming a Retention Policy

With our structure in mind, we can go to the Data governance section of the Security and Compliance Center, select
Retention , and then Create . The first step is to assign a name and description (which only administrators see) to the
new policy. Some tenants insist that administrators include their name and a pointer to supporting documentation in the
description of a new policy. It’s more useful to include notes about what the policy does. For example:

“This is a retention label policy to publish a set of general-purpose labels to every location in the tenant.”

“This policy publishes the Highly Confidential label to the Senior Leadership Team locations.”

“This policy searches for content with the “Kazaa” keyword and applies the Ten Year retention label.”

Keeping or Removing Content

The next step is to define what the policy does to keep or remove content (Figure 19-2 ). When a retention policy removes
items, it uses the equivalent of a “delete and allow recovery” action in an Exchange retention tag. Users can recover items
later if needed from Exchange’s Recoverable Items structure or the SharePoint recycle bin. In either case, we must know
the length of the retention period and how Office 365 will calculate the age of an item). You can keep content forever, but
it is more usual to set a period like ten years. To remove content, you must tell Office 365 how to calculate the age of the
content. For mail messages, the creation date is used, but when a policy spans both documents and other items, it is best
to choose the last modification date as shown here as this accommodates the fact that documents are often changed well
after their creation date.

Figure 19-2: Defining the actions a retention policy takes to process content

Administrators who are accustomed to working with Exchange retention policies see an immediate difference here.
Exchange retention policies let you remove or archive items after the retention period lapses. While Office 365 retention
policies do not support an archive action, they let you say that you want to keep information for a set period. When the
retention period lapses, you can choose to leave the content alone (in which case another policy could apply) or remove it.
Alternatively, if you are more concerned about cleaning out locations to remove information that the organization no
longer needs, you can instruct Office 365 to remove items after they reach a certain age.

Immediate Evaluation : After you publish a retention policy, workloads make it active. This means that items within the
scope of the policy are evaluated and processed per the policy settings. If you’re not careful, the introduction of a new
policy can have a significant effect on users. For example, let’s say that you create a policy to remove items after three
years and set the scope of the policy to be all OneDrive for Business accounts. After publication, Office 365 begins to
evaluate items in all accounts and will remove anything more than three years old. If users aren’t prepared for this clean-
up to happen, the sudden removal of many items could come as a nasty surprise.

Setting the Scope for the Retention Policy

Next, we set the scope for the policy. You can decide to apply to all supported locations across Office 365 (an org-wide
policy) or you can select specific locations (a non-org wide policy). The simplest form of retention policy is one that
includes every available location, but quite often the need exists to focus in on a select group of individuals and the data
with which they work. Office 365 retention policies allow you to include or exclude subsets of locations. When you enable a
retention policy for a location, you can choose the scope to be:

Org-wide : Cover Exchange mailboxes, public folders, Office 365 Groups, SharePoint Sites and OneDrive for Business
accounts. The policy applies to all content in all locations. As new locations are added to the tenant, they come under
the control of the policy.

Non org-wide (choose specific locations): You can select individual mailboxes, sites, and so on. You can select
everything from a workload, like all Exchange mailboxes, or you can select a subset of the locations available within a
workload, such as only a few SharePoint sites. You can also exclude a selected subset from the policy. This means that
the policy will not apply to the mailboxes or sites that you select.

The exceptions are for Skype for Business and Teams. For Skype for Business, you must choose the users to include in the
policy, and public folders, which are either covered by the policy or not. You cannot select individual public folders for
inclusion in a policy. In addition, you cannot mix inclusions and exclusions for a location in a policy. If you exclude some
sites or mailboxes from a policy, it means that the policy applies to all other sites or mailboxes but not to those selected for
exclusion. While you can include Skype for Business in a retention policy spanning other workloads, the same is not true
for Teams. A retention policy for Teams can only cover Teams channel messages, Teams personal chats, or both. You
cannot include information another workload in a retention policy for Teams. However, you can have multiple retention
policies for Teams that cover different subsets of users, or separate policies for channel messages and personal chats.
Table 19-2 explains the various location types and how you input the selected locations.

Location Identified by
type

Exchange Mailbox alias or user name. You can also input the name or alias of a distribution group or mail-enabled
email security group.

SharePoint URL for the site. Example: https://tenant.sharepoint.com/Projects/

site

OneDrive URL for the site. Example:

for
Business https://tenant-my.sharepoint.com/personal/kim_akers_office365itpros_com/
accounts

Office 365 The name or alias for the group. You can add groups that use Yammer to store their conversations. In this
Groups case, the retention policy ignores the conversations stored in Yammer because Yammer does not yet support
the data governance framework, but the documents in the group document library are covered.

Skype for Input the account names of the users whose IM conversations you want the policy to cover. You cannot use a
Business distribution group or mail-enabled security group to input the names. You can include up to 1,000 Skype users
in a non-org wide policy.

Exchange Choose “All” to extend the policy to cover every public folder in the hierarchy. The default is “None.” You
public cannot select a sub-set of public folders.
folders

Teams Choose All to cover messages posted to all channels in every team in the tenant or select the individual teams
channel to come within scope of the policy.
messages

Teams Choose all to include all personal chats sent by users in the tenant or select the users to come within scope of
chats the policy.

Table 19-2: Office 365 locations supported by retention policies

Hybrid Governance
Office 365 can only apply retention policies to locations under its control. It cannot apply policies to on-premises locations
such as Exchange mailboxes, public folders, or SharePoint sites. The same issue occurs for content searches and
eDiscovery cases. If you have a hybrid environment, it’s a good idea to try to have the same retention policies (or as close
as possible) apply in both on-premises and cloud locations and be prepared to perform eDiscovery processing on both
platforms to capture all information necessary to satisfy eDiscovery searches.

Teams Retention
When we discuss retention for Teams content, it is important to understand that retention policies cover the compliance
records that Office 365 creates in group and user mailboxes to record messages posted in channel and personal
conversations. Teams also creates compliance records for messages posted in channels by Office 365 connectors.

When a Teams retention policy is in place, the Managed Folder Assistant (MFA) processes the mailboxes covered by the
policy to remove compliance records according to the policy criteria. MFA processes compliance records in the same way
that it deals with other mailbox items (for example, a mailbox holding compliance records is not processed if the mailbox
size is under 10 MB). The date used to determine whether items exceed their retention period are the creation dates of
compliance records in the Team Chat folder.

When the Managed Folder Assistant removes compliance records, Office 365 initiates a background job to remove the
source items in the Teams chat service (in Azure) and later, through server-to-client synchronization, from client-side
caches. Depending on the load on different components within Office 365 and how often clients connect to the Teams
service, the end-to-end removal process usually takes three or four days to complete and can take up to a week.

System messages posted to channels (for example, the addition or removal of a member) are not removed by retention
policies. In addition, some older messages posted by guest or hybrid users might not be removed.

If you want retention policies to apply to the content posted in the SharePoint document libraries used by Teams, you must
include those sites in the SharePoint section of the retention policy. A retention policy cannot process data stored in other
locations used by Teams such as third-party applications accessed through tabs or bots.

Retention policies for Teams messages cannot use the advanced features to search for content based on keyword queries
or sensitive data types. In addition, the minimum retention period for a Teams policy is 30 days.

Choosing Locations
If your policy covers a subset of workloads and locations, some up-front work is necessary to list the locations and gather
the information to input each location. In our example, it is easy to input the mailboxes because we can use a distribution
group. Office 365 expands the group to extract the individual mailboxes to add them to the policy. This is a one-time
operation and if people leave or join the distribution group thereafter it will not affect the retention policy. You will have to
edit the policy separately to ensure that it continues to cover the correct individuals.

While mailboxes are relatively simple to add, the same is not true for SharePoint and OneDrive for Business as you need to
input the URLs for sites that you want to add to the policy. As it is sometimes difficult to know exactly what the right URL
is for each site, it is a good idea to collect the URLs beforehand in a text file and cut and paste the URLs from the file into
the policy. You can add up to a hundred individual sites in a non-org wide policy.

Eventually, after adding the locations to the policy, we have something like the situation shown in Figure 19-3 . Note that
the Exchange email location reports only one recipient. This is because we input a distribution group and Office 365 has
not yet expanded the membership to extract the individual mailboxes. Click Next to continue.
Figure 19-3: Choosing a set of locations for an Office 365 retention policy

Reviewing Policy Settings

Aside from the decision whether to enable preservation lock for our policy (see next section), the last step is to review its
settings (Figure 19-4 ). If all looks to be in order, click Create this policy to instruct Office 365 to begin the publication
process to the locations covered by the policy. The Managed Folder Assistant enables retention policies for Exchange
locations (including Skype for Business Online and Teams) while SharePoint and OneDrive for Business use scheduled
background jobs to process policies. You can also opt to save the policy for later, which means that the policy is in a draft
state that you can update later (perhaps to add some extra locations) before making the policy live.
Figure 19-4: Reviewing policy settings before creating a retention policy

It can take some time before a retention policy becomes fully effective across all locations. The Managed Folder Assistant
must process each mailbox that comes under the scope of the policy, including group mailboxes and SharePoint
background jobs must run to make the retention policy effective for sites. You can check the distribution status of the
policy by selecting the policy and viewing its properties. The following values are available:

On (Success): All locations know about the policy and are processing content as per the policy. However, the policy
might still not be effective everywhere because background jobs might not have completed their processing to enable
all locations.
On (Pending): Office 365 is enabling the policy in the target locations.
On (Errors): The policy is active, but some errors have occurred in its distribution.
Off (Success) : The policy is disabled and is not being applied to content. You can reenable the policy at any time.
Off (Pending): Office 365 is disabling the policy is. All locations are stopping the processing of content. However,
the process is incomplete.
Off (Errors): The policy is off, but due to some errors, it might still be active in some locations. You can click the
error to get more information. In many cases, the fault will disappear if you leave it alone for an hour or so.

The Effect of Retention

In terms of what you might have done previously with Exchange retention policies or SharePoint document delete policies,
the Office 365 retention policy we just created has the following effects:

A default retention tag lasting seven years is placed on every item in the mailboxes that are not otherwise stamped
with an explicit tag (directly placed on the item or inherited from the folder) or label. In addition, Exchange places an
in-place hold on the mailboxes (both primary and archive) for seven years. During this period, Exchange preserves
any item that a user removes from the mailbox in the Recoverable Items structure. Office 365 indexes the preserved
items so that they stay discoverable. When an item’s retention period elapses (in this case, seven years after
creation), the Managed Folder Assistant applies a “Delete and Allow Recovery” action and moves the now-expired
item into the Recoverable Items folder. The user can recover the item until the deleted items retention period set for
the mailbox expires (usually 14 days).
An in-place hold also applies for documents and lists stored in SharePoint or OneDrive for Business sites. In these
cases, when a site comes within the scope of a retention policy, Office 365 creates a special Preservation Hold library
for the site (if one does not already exist). Users cannot see this library because it is only visible to the site collection
administrator. Any attempt to remove or change content in the site thereafter causes SharePoint to copy the original
document to the Preservation Hold library, including versions of documents (Office 365 now preserves up to 100
versions of documents stored in SharePoint Online and OneDrive for Windows). Figure 19-5 shows items in a
Preservation Hold library where the names of the items kept by policy are formed by combining the original name
with a GUID to create a unique value. Site administrators can access the Preservation Hold library by going to Site
Contents and then selecting the library. However, a site administrator cannot remove or change items kept in the
Preservation Hold library. When the retention period elapses for items, background timer jobs remove expired items
from the document library or list. The job also cleans up the Preservation Hold library by removing any changes and
versions found there. If users did not update or remove documents during the retention period, copies do not exist in
the preservation hold library. Removed items go into the recycle bin from where they are permanently removed after
the normal 93-day recycle lifetime expires.
 Contents of group mailboxes are treated like user mailboxes and contents of the group document library are treated
like other SharePoint sites.
Kept copies of Skype for Business IM conversations cannot go into the Conversations History folder of user
mailboxes. This folder and its contents are visible to mailbox owners, so the copies kept by policy go into the Purges
folder in the Recoverable Items structure (Teams hides its compliance records from user view, so the same problem
does not arise).
Although our policy does not cover public folders, if they were, any copies kept by policy stay in the public folder
mailboxes to which they belong until the retention period expires. A process like that used to assess content removed
from user mailboxes ensures that any attempt to remove held content from a public folder cannot succeed.

Figure 19-5: Items in the Preservation Hold library for a SharePoint site

Office 365 Retention Policies and Inactive Mailboxes

As explained in Chapter 6, an inactive mailbox is a mailbox kept for compliance or some other reason that belonged to a
now-removed user account that was placed on hold before deletion. Including a mailbox in the locations processed by an
Office 365 retention policy ensures that a mailbox becomes inactive when its account is removed, but only if the policy
settings keep content. A policy configured to remove content does not transform a mailbox into inactive status upon
deletion. This is logical because if you deploy a policy to remove content from mailboxes, you do not want Office 365 to
keep mailboxes after you delete their accounts.

It is a good idea to have a retention policy to hold the complete contents of user mailboxes for a period after deletion. To
make this work, you must add the mailboxes to the policy (up to 1,000 mailboxes can be included in a single policy) before
the user account is removed. Alternatively, you can use an org-wide policy that covers the content in all mailboxes. In
either case, when Office 365 processes the account for deletion, it recognizes that one or more hold exists on the mailbox
and puts the mailbox into inactive status.

Advanced Retention Settings

When you define content settings for a retention policy (shown at the bottom of Figure 19-2 ), you can choose to use
advanced retention settings. This means that you want to create a policy to find content based on a KQL query or DLP
sensitive data types. For instance, you might want to find all content that has the keyword “Project Moonshot” and keep
that for ten years. Or you could find all content that has sensitive data covered by the U.S. Patriot Act (credit cards, bank
account numbers, taxpayer identification numbers, and social security numbers) and make sure that Office 365 removes
these items after five years. Figure 19-6 shows an example of advanced retention settings. In this case, we use three
keywords to find content that we want to keep. Office 365 will use these keywords to query the locations supported by
content searches and will apply the policy to any matching items (messages, documents, etc.).
Figure 19-6: Using keyword queries to find content to keep through a retention policy

If you change the policy settings later to update the keywords used in the query to match content, the target locations
evaluate the keyword queries to decide whether to keep or remove content. Retention policies that use queries are
distributed and enforced in the target locations in the same manner as other policies.

The Managed Folder Assistant has no part to play in retention policies based on keyword policies when they are applied to
Exchange mailboxes. Instead, the indexing service checks messages as they are added to content indexes and applies
retention policies as needed.

Figure 19-7 shows how to include a DLP sensitive data type (credit card number) in a retention policy. The definitions used
for sensitive data types are the same as for unified DLP policies, so they support all locations except Exchange public
folders, Teams compliance records, and Skype for Business conversations. It is also worth noting that Exchange applies
DLP checking to messages as they transit through the transport system, meaning that no checking against sensitive data
types happens for items already in situ within mailboxes. The transport service begins to check email as soon as Office 365
lets Exchange know about the new policy. SharePoint and OneDrive use different mechanisms to find content having
sensitive data and will report violations when content is added or changed or when a full index occurs of sites.
Figure 19-7: Using a DLP sensitive data type in a retention policy

Behind the Scenes with Advanced Retention Policies

When you create a retention policy with advanced settings, Office 365 also creates a retention label. Later, when matching
items are found, Office 365 stamps the that retention label on those items. Office 365 also creates a retention rule to tie
the policy to the retention label. You don’t see this through the user interface, but you can with PowerShell.

For example, if we examine the settings for the Office 365 for IT Pros Files retention policy, we see the keyword query the
link to the retention label (called a compliance tag here).

[PS] C:\> Get-RetentionComplianceRule -Policy "Office 365 for IT Pros eBook Content" | Format-List ContentMatchQuery, ApplyComplianceTag

ContentMatchQuery : "Office 365" NEAR(10) eBook

"Office 365" NEAR book

"Office 365 for IT Pros"

"Office 365" NEAR (20) Chapter

ApplyComplianceTag : 0b460d18-f6d2-401c-99e2-5594d711f579

If we run the Get-ComplianceTag cmdlet to look up the retention label, we can see the retention duration and retention
action:

[PS] C:\> Get-ComplianceTag -Identity 0b460d18-f6d2-401c-99e2-5594d711f579 | Select Retentionduration, Retentionaction

RetentionDuration RetentionAction
----------------- ---------------

3650 Keep

Thus, we know that any item found matching this query will be kept for 10 years.

A retention rule and retention label are also created for retention policies based on sensitive data types. To retrieve the
sensitive data settings, look at the ContentContainsSensitiveInformation property in a rule. In this example, we see three
sensitive data types are used to find items.

[PS] C:\> Get-RetentionComplianceRule -Identity ctaptr-907ee81b-7c08-4b88-9db7-3882d1df7f83 | Select -ExpandProperty

ContentContainsSensitiveInformation

Name Value

---- -----

maxconfidence 100

id e55e2a32-f92d-4985-a35d-a0b269eb687b

minconfidence 75

rulePackId 00000000-0000-0000-0000-000000000000

classifiertype Content

name U.S. Individual Taxpayer Identification Number (ITIN)

mincount 1

maxcount 9

maxconfidence 100

id a44669fe-0d48-453d-a9b1-2cc83f2cba77

minconfidence 75

rulePackId 00000000-0000-0000-0000-000000000000

classifiertype Content

name U.S. Social Security Number (SSN)

mincount 1

maxcount 9

maxconfidence 100

id 178ec42a-18b4-47cc-85c7-d62c92fd67f8

minconfidence 75

rulePackId 00000000-0000-0000-0000-000000000000

classifiertype Content

name U.S. / U.K. Passport Number

mincount 1

maxcount 9

Knowing That a Retention Policy Works

It can be difficult for an end user to see any signs that a retention policy is in force. It can be equally difficult for an
administrator. Here are some ways to verify that all is well.

For user mailboxes , you can check to see whether the retention tag assigned by a policy is stamped on folders and
messages that you expect. Clients display these tags like older Exchange Online retention tags, so if Outlook or OWA
displays the tag, you know that the policy is working.
Conversations in group mailboxes do not display retention tags. You must check that the items in the Inbox folder
are what you expect based on the retention policy in force. You can also check that the Managed Folder Assistant is
processing the group mailbox and removing items.
Retained Skype for Business Online conversations and Teams compliance records are stored in special hidden
folders. You can use the MFCMAPI utility or PowerShell to check that folder to ensure that conversations are kept as
expected. An easier method is to use the Get-MailboxFolderStatistics cmdlet to check the number of items in the
folder before the Managed Folder Assistant processes it and the number of items afterwards (see the section later for
information about how to know when the Managed Folder Assistant processes a mailbox). For example, the command
 below shows that the oldest item in the team (any channel) is from 20 Aug 2017. You know the retention period set in
the policy, so it is easy to calculate what the date of the oldest item should be if the retention policy is working.

[PS] C:\> Get-MailboxFolderStatistics -Identity Team1 -FolderScope ConversationHis

tory -IncludeOldestAndNewestItems | ? {$_.FolderType -eq “TeamChat”} | Format-Table Name, Itemsinfolder, Newestitemreceiveddate,

Oldestitemreceiveddate

Name ItemsInFolder NewestItemReceivedDate OldestItemReceivedDate

---- ------------- ---------------------- ----------------------

Team Chat 513 21 Aug 2018 16:10:41 20 Aug 2017 08:17:09

Documents stored in SharePoint Online and OneDrive for Business sites, including the sites used by Office 365
Groups, show no trace of retention because the principle is to keep everything in-place until user actions trigger the
need for SharePoint to capture copies of documents. If a policy dictates that an item should be kept, and an attempt is
made to remove it, a copy of the item will be in the special Perseveration Hold library for the site. The capture
happens even if a retention label stops the user removing an item. The site collection administrator can access the
Preservation Hold Library through Site Contents to verify the capture of changes or deletions for files under the
control of retention policies are there.
Check the Office 365 audit log to look for FileDeleted events logged for deletions of SharePoint and OneDrive items
because their retention period expired, and the retention action forces a deletion. FileDeleted events capture
information about what account removes an item. In the normal course of events, the data captured in the user
property is the User Principal Name of the account. When a retention policy or label causes a deletion, the audit
event captures the name of the policy or label. For example, this command searches for all FileDeleted events logged
during a month:

[PS] C:\> Search-UnifiedAuditLog -Operations FileDeleted -StartDate 4-Nov-2018 -EndDate 5-Dec-2018 -ResultSize 2000

After retrieving the audit events, you can use the techniques explained in Chapter 21 to
interpret them and generate data for analysis. A review of the information reveals if
retention processing deleted files. In this example, a user deleted two files and a retention
policy deleted the next three files.

TimeStamp User File

--------- ---- ----

2018-11-28T15:15:22 tony.redmond@office365itpros.com Board Meeting Agenda 12 Sept 2018.docx

2018-11-28T15:22:17 tony.redmond@office365itpros.com Privacy Policy for Website.docx

2018-12-04T03:51:45 Preservation Lock - Mailboxes and Sites Encoding Time.csv

2018-12-04T03:51:46 Preservation Lock - Mailboxes and Sites TeamsNotebook(Shared).onetoc2

2018-12-04T03:51:43 Preservation Lock - Mailboxes and Sites Ch25Skype for BusinessRewriteV1.docx

Applying Retention Policies to Office 365 Groups

If you apply a retention policy to an Office 365 Group, the same settings are used for both the group mailbox and group
document library. You cannot, for instance, remove conversations after a month and keep documents for a year.

The Managed Folder Assistant (MFA) must process the group mailbox to remove or keep items based on the settings in a
retention policy. MFA will not process items against the policy unless more than 10 MB of data exists in the mailbox, so
some groups that hold several hundred conversation items might not be processed. The information held in the group
team site does not count against the 10 MB threshold. You can use the Get-MailboxStatistics cmdlet to check the current
storage for a group mailbox:

[PS] C:\> Get-MailboxStatistics -Identity Office365TenantServiceHealth | Format-Table DisplayName, TotalItemSize, ItemCount

DisplayName TotalItemSize ItemCount

----------- ------------- ---------

Office 365 Tenant Service Health 11.29 MB (11,836,393 bytes) 154

In this case, MFA will process the group mailbox. See the section about Logging the Managed Folder Assistant later to
understand how to see a summary of the actions taken by the MFA to remove items from a mailbox per the settings of a
retention policy.

Removing Retention Policies

To remove a retention policy, select it in the Security and Compliance Center and take the Delete Policy action. When you
remove a retention policy, Office 365 informs the affected workloads that the policy no longer exists so that they cease to
implement it. The next time a background process reviews content, it might apply a new retention policy to items. The
time taken for Office 365 to switch retention policies depends on how quickly workloads cease processing the original
policy and how soon afterwards the content covered by the original policy is reevaluated.

Retention labels applied by automatic label policies stay in place after the removal of that policy.

Preservation Locks
Some regulatory regimes require that after an organization implements a retention policy, administrators cannot turn the
policy off or make it less restrictive. To meet this need, you can apply a preservation lock to a retention policy after it is
created. Once a policy is locked, an administrator cannot disable the policy or remove locations from the policy. It will stay
in force and active for all locations under the scope of the policy until the retention period expires. All the content subject
to the policy cannot be removed from Office 365 or changed during this period either. In fact, the only options open to an
administrator is to add locations to the policy or extend its duration.

You can only lock a retention policy by setting its RestrictiveRetention property through PowerShell. For example:

[PS] C:\> Set-RetentionCompliancePolicy -RestrictiveRetention $True -Identity "Management Preservation Policy"

Because not even Microsoft can remove a locked retention policy, it is wise to pause and think before enabling
preservation lock on a retention policy. You might need to implement such a policy to satisfy a legal or regulatory need, but
in most cases, tenants do not need to lock down content in this way. In short, make sure that you need a preservation lock
before turning it on for a policy. Apart from waiting for the policy to expire, there is no way back if you make a mistake and
enable the lock in error.

To discover if any policies include preservation locks and the workloads they cover, run this command:

[PS] C:\> Get-RetentionCompliancePolicy | ? {$_.RestrictiveRetention -eq $True} | Format-Table Name, Workload

You can also see whether a preservation lock applies to a policy by viewing its details through the Security and
Compliance Center.

Note that if you move a mailbox that is subject to a preservation lock back to an on-premises Exchange server, Office 365
keeps a copy of the mailbox to satisfy the lock. The copy held in Office 365 is a point-in-time copy and no mechanism exists
to synchronize the two copies after the mailbox moves.

Office 365 Labels

Office 365 has two forms of labels.

Retention labels : Mark content for specific treatment, like keeping certain documents for longer periods because
they hold specially-valuable information, such as accounting records, sales records, and so on. Users apply labels to
messages or documents to mark the items so that they can be more easily found, to show the importance of the
content to users, or to assign a retention period to the item. Retention labels were previously called “classification
labels” and it is possible to find older articles and blog posts referring to retention labels as classification labels.
Sensitivity labels . Apply protection to content. Sensitivity labels are how Microsoft has extended the functionality
of the original Azure Information Protection labels into Office 365 as part of their Microsoft Information Protection
initiative. They refer to these labels as “unified” because they bring together the work done in Azure Information
Protection to make it easy for users to self-classify content through the application of labels within the Office
applications with the data governance framework inside Office 365. Applying a sensitivity label to an Office 365 item
can protect it through encryption or add visual indicators such as watermarks to show users the importance or
sensitivity of the information. See Chapter 24 for more information about how to define sensitivity labels and deploy
them to users through sensitivity label policies.

When we refer to the two types of labels in general, we say “labels” or “Office 365 labels.” Otherwise we refer to the
specific type. The information presented in this chapter covers retention labels.

Retention Label Concepts

A retention label can be passive, meaning that it serves as a marker for content of a certain type but takes no further
action because it doesn’t have any retention settings. Passive labels are useful to mark content that needs to be dealt with
in the future or to find items belonging to a project. Mostly, retention label are active, which means that the label has a
defined retention action and period. The action tells Office 365 to keep content for the period or to remove content after
that period. For instance, you might decide that all documents marked with the “Confidential” label should be kept for five
years and then removed afterwards.

If your tenant has E5 licenses, you can create auto-label policies to have Office 365 apply retention labels based on a
keyword search or sensitive data type. Applying retention labels in this manner is often easier for end users because they
do not have to think about what label they should apply when they create or update content. In addition, if content is auto-
labelled, less risk exists that some important content is overlooked. On the other hand, a label that is explicitly applied is
more precise and has precedence because someone goes through a decision process to select and apply the label. The
combination of manual (explicit) and auto-applied (implicit) classification gives tenants great flexibility in how they
manage important content stored within Office 365.

Items such as a document or message can only ever be assigned one label. Apart from the precedence of manually-applied
retention labels, the other rules are:
 Anyone with write access to content can change the label that exists on that content whether that label was applied
manually or automatically. In other words, if an email is labelled as “Confidential”, the user can go ahead and change
the label to any other available within the mailbox.
Retention labels applied automatically never overwrite labels manually applied by users.
If multiple auto-label policies exist that can match content, Office 365 selects and applies the label belonging to the
oldest policy. When a label policy is created, Office 365 assigns it a priority number that is incremented from 0 (zero)
as more policies are created. Thus, the policy with the lowest number is the oldest policy. You can discover the
priority order for policies by running the Get-RetentionCompliancePolicy cmdlet as show below:

[PS] C:\> Get-RetentionCompliancePolicy | Format-Table Name, Priority, WhenChanged

Name Priority WhenChanged

---- -------- -----------

Management Preservation Policy 0 13 Apr 2018 13:02:36

Company Confidential Policy 1 29 Apr 2017 03:07:38

Clean up Office 365 Groups with Connectors 9 29 Apr 2017 03:07:40

Preserve Office 365 for IT Pros Files 10 29 Apr 2017 03:07:38

Formal Company Records 13 29 Apr 2017 03:07:39

Senior Leadership Team (SLT) Retention Policy 15 29 Apr 2017 03:07:40

Office 365 for IT Pros eBook Content 16 29 Apr 2017 03:07:39

GDPR Personal Data 17 8 May 2018 16:30:32

Exchange GDPR test 18 13 Sep 2017 12:43:11

Senior Leadership Team (SLT) retention policy II 19 13 Apr 2018 13:38:42

Teams policy for ExchangeGoms 20 13 Apr 2018 13:52:31

Mailboxes on Legal Hold 21 16 May 2018 21:55:27

New Hydra Product Team Retention Policy 22 2 Aug 2018 17:02:05

Project Condor Policy 23 9 Nov 2018 10:16:26

General sensitivity policy 24 9 Nov 2018 10:14:52

Black Matter Policy 25 9 Nov 2018 13:07:21

SharePoint Online Retention Policy 26 28 Nov 2018 14:39:39

Preservation Lock - Teams 27 30 Nov 2018 18:41:00

Preservation Lock - Mailboxes and Sites 28 30 Nov 2018 18:50:43

You cannot change the priority order of retention policies. Not all the retention policies in a tenant are used to publish
retention labels to workloads. In fact, the Get-RetentionCompliancePolicy cmdlet returns all the policies used for retention
and sensitivity labels, including those used to apply broad-brush retention to a set of locations. To isolate these policies,
we can check what policy rules exist that apply retention labels and tie that information back to the retention policies.
Here is some code to find the set of retention policies used to publish retention labels and then tell us what retention
labels are published by each policy:

[PS] C:\> $Policies = (Get-RetentionCompliancePolicy -RetentionRuleTypes | ? {$_.RetentionRuleTypes -eq "Publish"} )

ForEach ($P in $Policies) {

Write-Host "Processing" $P.Name

$Tag = $Null

$Rules = Get-RetentionComplianceRule |? {$_.Policy -eq $P.Guid}

ForEach ($R in $Rules) {

If (-Not [string]::IsNullOrEmpty($R.PublishComplianceTag)) {

$Tag = $R.ComplianceTagProperty -Split(",")

$TagValues = Get-ComplianceTag -Identity $Tag[0]

Write-Host $P.Name "includes the retention label" $TagValues.Name }

}}

Processing Company Confidential Policy

Company Confidential Policy includes the retention label Office 365 for IT Pros eBook Content

Company Confidential Policy includes the retention label Contractual Information

Company Confidential Policy includes the retention label Confidential…

Retention labels are published to workloads through label policies. A label cannot be used within a location unless it is
published to that location as part of a label policy. You can have multiple label policies in use within a tenant. Separate
policies exist for retention and sensitivity labels.

Retention Label Policies

Broad retention policies assign retention settings to locations. Retention label policies make one or more retention labels
available to end users through a publication process that informs Office 365 workloads about the labels and their settings.
It is then up to the workload to decide how best to reveal the labels in the different clients supported by the workload. A
label policy is composed of one or more retention labels. A retention label can be linked to multiple policies. For instance,
the “Confidential” label referred to above to keep content for five years could be in the policies assigned to different
departments along with other labels that meet the specific needs of each department. The members of the legal
department might have a policy that includes labels called “Case Review”, “External Counsel”, and so on while people
working in Accounts might have labels for “Collections”, “Audit Records”, and “Tax”. Being able to create policies with a
mixture of generic labels and work-specific labels supports flexibility in data governance. After all, not everyone who
works in a company needs to deal with information in the same way.

Apart from making retention labels available to users, the policy defines the locations where the label can be used. When
complete, administrators publish the policy to make the labels available to end users. After a period, the labels included in
the policy become available to clients and users can then apply the labels to email, documents, and other content to mark
those items as being of interest for business reasons. Behind the scenes, background processes like the Exchange
Managed Folder Assistant make sure that the instructions contained in the label settings are respected. For instance, any
item stamped with the “Archive Retention” label might be kept for 10 years and then removed from wherever it is stored.

Planning Retention Labels

The first thing to decide is what labels are needed to fill out the retention strategy. Broadly speaking, you can divide
retention labels into the following categories:

Specific : Retention labels needed by certain departments and used for specific purposes. For example, “Project
Documentation,” “Board Minutes,” or “Patent Material.” These labels usually keep information that is of high
importance to the organization and might be published to a select group of locations.
Generic : Retention labels used anywhere in the business. Often, these labels are named after the length of the
retention period, as in “Keep Five Years.” They might also have names that describe the business purpose, like
“Commercially Sensitive” or “Required for Audit.” These labels are usually published to all locations in an org-wide
policy.
Special -Purpose . Retention labels created for a specific purpose. For example, to mark and keep a set of
information needed for a merger and acquisition project.

After consulting with business units (including the IT department) to gather suggestions for retention labels, you can
rationalize the set to a manageable number. Each label should serve a distinct and obvious purpose that can be defined in
clear and easily understandable terms. In addition, you should be able to say where the records marked by labels are
stored. For example:

“We are required to preserve financial records for five years because we can be audited during this period. We need a
label to mark these records and ensure that Office 365 keeps them for at least five years. Items needed for audits include
messages and documents across all Office 365 mailboxes and sites.”

It is sensible to write down each of the retention labels that you plan to use before creating anything. It is much easier to
delay the release of a label and the training of users to use the label properly than it is to launch a label into general
circulation only to discover that you later need to withdraw it. Another thing to consider is how easy it is for users to
decide between different retention labels when the time comes for them to apply a label. Too many labels, misleading
names, or too much choice can lead to frustration and bad decisions.

Other points to consider include:

An item can only ever have a single retention label. To change a label, you must remove the first label and replace it
with another. Sometimes, you might have to remove a label from an item before you can remove the file.
A retention label applied by a user always has the highest priority. Office 365 never applies a retention label to
content if a label applied by a user already exists. On the other hand, a user can always replace a label applied by an
automatic policy with one that they choose. The logic here is that the user understands the full import of the content
and can therefore make the best decision about its importance.
If you use automatic label policies to apply labels to content found using a keyword query or because the content has
sensitive data, you might find yourself in a situation where some documents come within the scope of multiple
policies. Office 365 needs to have a tiebreaker to know what policy it should apply. That tiebreaker is the age of the
policy, so Office 365 always applies the oldest policy. The logic here is that companies usually create their most
important policies first, so it makes sense to apply those policies first. This is a key factor in the planning process as
you must decide what your most important policy is and make sure to create it first. No way exists to reorder the
 priority of automatic label policies as the only factor is a policy’s creation date. Therefore, if you make a mistake with
your priority, you must remove policies and recreate them in the correct order.
Although auto-label policies cannot replace labels assigned by users, they can replace labels previously automatically
assigned to items by other policies.

In summary:

Make sure that every retention label has an obvious purpose.

Try to have a small number of retention labels so that it is easier for users to make good choices about how long
content should be kept.
Create retention labels and policies in priority order.
Deploy auto-label policies after users have had a chance to apply retention labels manually.

Removing a Site : Before you can remove a SharePoint Online site, you must remove any documents that have retention
labels from the site. Normally, this means that you must remove the label from the documents and then delete them.

Creating New Retention Labels

After understanding the labels necessary to implement the data governance policy, we can create them through the
Classifications section of the Security and Compliance Center. Click Labels and then Create a label . You can then start
by entering the name of the label and some text to describe the purpose of the label for administrators and a separate
description that is visible to users when they browse labels as they classify material.

Naming a Retention Label

The name given to a label is important because this is what users see when they use the label to classify a message or
document. One issue for multilingual tenants is that no facility exists to translate labels. Whatever name you give to a label
shows up in clients, no matter what language they use. For this reason, it is best to give labels names that are simple to
understand and unambiguous in their intent as this will make it easier to communicate how to use the labels to classify
information.

Figure 19-8: Entering the name for a new label

For example, if we publish the label shown in Figure 19-8 to OneDrive for Business and SharePoint Online, people can use
the label to classify a document stored in a site by amending the document properties and selecting the label. As you can
see in Figure 19-9 , the user gets two visual hints about the meaning of the label: from its name and the descriptive text
that appears when they hover over the label.
Figure 19-9: Selecting a retention label to apply to a SharePoint document

Retention Actions and Periods

The next step is to decide whether the label has a retention action. The default is to turn retention Off, which means that
the label is just that – a label to show the importance or otherwise of some content. Apart from not having an associated
action, you can use these labels to mark content in the same way as any other label. For instance, you could create a
“Draft” label to allow users to mark items that have not yet reached the point where the content is interesting or valuable
enough to justify its retention. Another way of using labels without actions is as a convenient way to find information with
content searches (for instance, find all documents for “Project X”). Labels that do not have a retention action show up with
a “Never” expiration date when viewed through Outlook or OWA.

If you do want to use a label to enforce content retention or deletion, move the Retention slider from Off to On to display
the page to define the retention settings (Figure 19-10 ). The same retention actions that are available in Office 365
retention policies are also available with labels. If you have worked with Exchange retention policies previously, these
actions look like those used for a retention policy tag. However, as we already know, some significant differences exist.
Figure 19-10: Defining how Office 365 uses a new label

Do not select the choice to use the label to classify content as a record unless you understand what this means. We review
the concept of record labels later in this section.

When all the details are complete for the new label, click Next to review the settings and then Create this label .

One chance to get label settings right : Be careful with the settings you specify when creating a new label as you can
only change the retention duration afterwards. The logic is that allowing other changes after label creation will disrupt
how the label behaves when applied to content. For example, if you change a label that keeps content for ten years and
does not remove it afterwards to now remove the items, users might lose information that they expect to have.

Comparing Retention Labels and Retention Policies

The combination of retention policies and retention labels gives Office 365 administrators a lot of flexibility to plan a
retention strategy for content. However, policies and labels support different retention settings, and this can be confusing
at times. Let’s define what each can do.

You can apply retention actions and periods through both policies and labels.
A retention label can also be used purely for visual marking (a label). In this case, it has no retention settings.
Both policies and labels support the ability to retain content for a set period or remove content after a set period and
to use the created date or the last modified date as the date when the retention period begins.
Once the retention period is over, both policies and labels allow the content to be left in place for the user to decide
what to do with it. An analogy is government papers that have restricted access for ten years after creation. When the
ten-year period expires, Office 365 does not remove the papers. Instead, they become available for public access. This
subtle differentiation is important in the world of records management.
Because they are designed to be more explicit, retention labels offer extra control over what happens when a
retention period is over. A retention label can invoke manual disposition or be used for event-based retention. We’ll
get to these topics later in this chapter.
A retention label can be used to classify an item as a formal company record.
Unlike Exchange Online retention tags, neither Office 365 retention policies nor retention labels support a “move to
archive” action. If moving mailbox items to the archive is important, you can continue to use mailbox retention
policies in conjunction with Office 365 retention policies.

Figure 19-11 shows the retention settings for a retention policy (left) and label (right). The extra flexibility available in
retention labels is obvious.
Figure 19-11: Retention settings in an Office 365 policy (left) and retention label (right)

In summary, retention labels offer more flexibility, but they must be assigned manually or through auto-label policies. The
retention settings in policies are simpler, but they are easier to apply because you can deploy retention across the entire
organization or for a selected set of locations with just a few policies.

Publishing Retention Labels Through a Label Policy

To make the new label available to users, you must publish it in a label policy. You start this process by going to the Labels
section under Classifications to view the set of labels defined in the tenant. You can then select a label and click either
Publish Label or Auto-apply label, both of which result in the creation of a label policy to select how the label is made
available to Office 365 workloads or selected locations managed by those workloads. Alternatively, you can select several
labels and take the action to publish the set of selected labels in a single policy (bulk publish). We will explain how to
create an auto-applied policy in the next section. After deciding on the labels to include, you select the locations where the
labels are available to users. The options are:

All locations across Office 365 : This means that the labels in the policy are available to all users in the following
workloads. This is called an org-wide policy.

Exchange Online mailboxes . Labels appear in clients in the same way as Exchange retention tags and can be used
as retention tags. For example, you can create a rule in Outlook desktop to apply a label to messages, such as all
those that come from a specific address.
SharePoint sites . Labels can be used to classify individual documents or as a default label that applies to all content
in a document library. When a default label exists, all items stored in the library inherit the label unless another label
is explicitly assigned to an item.
OneDrive for Business sites . Labels can be used to classify individual documents stored in OneDrive folders.
Office 365 Groups . Labels can be applied to conversation items in Outlook Groups. They cannot be used to classify
conversations in Yammer-based groups. Because labels can be used with SharePoint, they can be used to classify
information in the document libraries belonging to either Outlook Groups or Yammer Groups.

Let me choose specific locations: For each of the supported workloads, you can opt to include or exclude certain
mailboxes, sites collections, OneDrive sites, or groups. For mailboxes and groups, you enter the names of the objects
you want to include or exclude. For site collections, you enter the URL in the form https://mytenant.sharepoint.com/.
For OneDrive for Business sites, enter the URL for the site. Figure 19-12 shows the locations available for label
publication. We can see that 50 Exchange mailboxes (or distribution lists) are selected together with all SharePoint
sites, all except one OneDrive account, and 100 Office 365 Groups.
Figure 19-12: Selecting specific locations for a label policy

Label policies with excluded or included locations are called non-org wide policies. You can have up to 1,000 non-org wide
policies in a tenant. This figure includes both retention and label policies.

When you exclude or include locations in a label policy, certain limits exist in the picker used to select target locations:

For Exchange , if you don’t enter anything in the search box, the picker shows the first 50 mailboxes and distribution
lists in the directory. Enter a search phrase to find the mailboxes to include or exclude. The easiest way to add many
mailboxes at one time to a policy is to use distribution lists. Office 365 expands the lists and adds all the mailboxes in
the membership to the policy.
For SharePoint , you can include or exclude up to 100 sites. These are sites, not libraries within sites. Use this
location to publish labels to traditional SharePoint sites that are not connected to Office 365 Groups.
For OneDrive for Business , you can include or exclude up to 100 accounts by specifying the URLs for the target
accounts.
For Office 365 Groups , the picker shows the first 100 groups in the tenant, including those hidden from Exchange
clients (used by Teams). You can add up to 1,000 groups to a label policy. Labels published to Groups apply to the
chosen group mailboxes and the group team sites. Use this location to publish labels to modern team sites connected
to Office 365 Groups.

Workloads like Teams and Planner do not currently support labels, so there is no need to include them in the publication
process.

Click Next after selecting all the target locations. You now name the policy and give some optional information to explain
is purpose. Figure 19-13 shows typical name settings for a policy. Click Next to review the settings for the policy. If any
issues are detected at this point (for instance, you enter a SharePoint site instead of a site collection), Office 365 will
refuse to publish the policy and you must fix the problem before you can continue and save the policy.

When everything is ready, click Publish labels to begin the provisioning process that makes the policy available to the
target locations. You know when the provisioning process is complete when the policy status changes from “Pending” to
“Success.” Once published, the new label is available to the target workloads defined in the policy. This process can take
up to one day to complete as, in some cases, the new labels must be published to clients. For example, the XML data used
to make retention policy information known to Outlook desktop clients must be updated with details of the new label and
then downloaded by clients before the label appears in Outlook’s user interface. Web clients typically pick up new labels
faster, but it is reasonable to expect that the entire end-to-end publication and provisioning process might take one or two
days before a new or updated label is available throughout the tenant.
Figure 19-13: Naming a label policy

Applying Retention Labels Through Auto-Label Policies

Publishing a set of labels for people to apply to documents and messages is certainly one way to solve the need for data
governance. The problem with this approach is that it depends on human beings to be very precise, consistent, and
persistent on how they classify material. As noted in this book, we know that humans are very inventive, but they also tend
to lose interest in boring technicalities after a while. On the other hand, computer systems are all about processing the
same steps time after time until told to stop. Auto-label policies, which are part of the Office 365 E5 plan, help
organizations to satisfy their data governance needs by finding content needing to be kept and applying the right label to
that content. Auto-label policies work alongside standard label policies so that users can explicitly label content while the
bulk of the retention assignments happen automatically.

Take our “Audit Material” label for example. We know that we want users to use the label to classify any content that the
organization needs to keep for audit purposes. We have reasonable confidence that people will classify new documents and
messages, but tens of thousands of pertinent documents exist in SharePoint and OneDrive for Business sites that we need
to keep. If we know how to find the content, an auto-label policy is the right tool for this job.

To begin, go to Label policies under Classifications and click Auto-apply label . The first step is to select the labels to
apply automatically to content. Select Choose a label to auto apply to pick a single label from the list of labels defined
for the tenant (Figure 19-14 ). When you have selected the label to apply, click Add and then Next to confirm that the
chosen label is correct.
Figure 19-14: Selecting labels for an auto-label policy

Two options exist to find content for the policy to apply the label:

Select content that matches a sensitive information type like credit card numbers or social security numbers. Because
Office 365 uses the sensitive data types defined for Office 365 Data Loss Prevention (DLP) to find the content, you
can apply labels to any location supported by Office 365 DLP policies. When you opt to use sensitive data types with
auto-apply label policies, you first select a template from the standard DLP set and then customize the rule set to
match your needs. For instance, you might want to only look for credit card numbers, in which case you could start
with the U.S. PII template and remove the other sensitive data types (like bank account numbers) to arrive at what
you need. You can also increase or decrease the standard values used as thresholds for DLP rules (for example, fire if
Office 365 detects more than six instances of credit card numbers in a document). See Chapter 22 for more
information about DLP.
Alternatively, you can create a custom policy and select any of the sensitive data types supported by Office 365 (over
80 at the time of writing). For instance, to create an auto-label policy covering GDPR information, you can select the
sensitive data types for individual European Union countries, like the Irish Personal Public Service (PPS) and
Netherlands Citizen’s Service (BSN) numbers.
Select content that matches a search query. Like the content searches used in Office 365, the query is expressed in
KQL syntax (this page is helpful to understand how to construct queries). You can use queries to apply labels to
Exchange, SharePoint, OneDrive for Business, and Office 365 Groups.

Board meeting minutes are a good example of the kind of record that companies must keep for compliance and audit
purposes. The query shown in Figure 19-15 looks for Word documents created since 1 January 2015 that include the
keyword “board minutes.” Obviously, this is a very simple query and the kind of queries that used in practice are likely to
be more complex.

Click Next to continue to name the policy and Next again to select the locations where Office 365 will look for content.
After selecting the locations, click Auto-apply to begin the provisioning process. Note the warning that it can take up to
seven days before the policy is fully effective and Office 365 has stamped the labels onto relevant content. This delay is
necessary to allow Office 365 to perform a first scan across all the selected locations to find matching items and apply the
label. Once active, Office 365 background processing finds new items matching the search query and stamps those items
more quickly than in the original step. As the background process finds and stamps items with the label, it records its
activity in a set of ComplianceSettingChanged audit records in the Office 365 audit log. You know that these activities are
for an auto-label policy because Office 365 writes the process identifier (a GUID) in the User property in the audit records
(audit records that have the label name in the User property are the results of an item receiving the default label assigned
to a SharePoint library). Auto-label policies ignore Items that already have a label on the basis that if a label already exists,
there is no need to apply an automatic label.

Figure 19-15: Entering a search query for an auto-label policy

Auto-applying labels is an effective way to achieve broad coverage of content that a query can find using a keyword or
because the content holds some sensitive data. Later, users can review items with the label and decide whether the label is
correct, or another label is a better fit. Unlike other labels, the labels included in auto-apply polices do not show up in
SharePoint, OneDrive, or Exchange as labels that users can apply manually.

If you deploy several auto-label policies, it is possible that documents will end up with labels that you think are incorrect.
Apart from using bad search queries to find the items, the problem might be because more than one policy applies to
documents. When this happens, the oldest policy (the one with the lowest priority number) wins and once a policy applies
a label to an item, other policies will ignore that item because it already has a label. Because auto-labeling works in this
manner, you need to be careful with the queries used to find content and with the order that you deploy policies.

Using Retention Labels

We now know how to set up, publish, and manage labels and understand the basic principles that only one label can exist
on an item at any time, whether a user assigns the label to an item manually or a background process assigns the label
automatically. With those thoughts in mind, we can now go ahead to discuss how to use labels within the different Office
365 locations.

Using Retention Labels with Exchange

Publication of a label means that workload-specific mechanisms make the new label available to users of an application.
For Exchange Online, that mechanism is to insert the label into the set of retention tags available to a mailbox through its
assigned retention policy. The Managed Folder Assistant (MFA) handles the publication of the unified set of retention tags
and labels to user and group mailboxes (group mailboxes only see labels and not tags). Only labels that have a retention
action are included in the set published to Exchange.

Inside Exchange Online, MFA runs on a workcycle to process mailboxes at least once every seven days. Therefore, it might
take up to a week before new labels become available to user and group mailboxes. See the “Logging the Managed Folder
Assistant” section later to understand how to extract and interpret MFA diagnostic logs for a mailbox to know when MFA
last processed the mailbox.

Mailboxes must hold more than 10 MB of content before the MFA processes the mailbox and labels show up. This
restriction exists to ensure that the Managed Folder Assistant does not waste cycles to process unused mailboxes. When it
opens a mailbox for processing, the Managed Folder Assistant makes sure that the current set of labels are known to the
mailbox before it begins to check items against their retention status.
Integrating labels with retention tags allows OWA and Outlook desktop to handle the two kinds of tags in a common way.
In effect, OWA and Outlook treat labels in the same way as personal retention tags and make the labels available to users
to tag individual messages or folders. Figure 19-16 shows how Outlook 2016 presents a mixture of retention tags and
labels to the user when they want to apply a policy to a folder.

Figure 19-16: Outlook lists retention tags and labels

Because labels are treated in the same way as personal retention tags, users should not see any different between the two
types. If a user assigns a label to a folder, all items in the folder inherit that label, unless they are already explicitly
stamped with a personal tag or another label. Likewise, if you change a label at the folder level, the next time the
Managed Folder Assistant processes the mailbox, it updates the items in the folder with the new label unless they already
have an explicit label or tag.

Because Exchange public folders do not support retention policies, they also do not support labels.

Integration with Exchange Retention Policies

Although OWA and Outlook present retention labels to users in the same way as they see personal retention tags from
mailbox retention policies, we already know that labels function differently to retention tags. Table 19-3 lists some of the
differences between the two methods used to mark mailbox items for retention processing.

Feature Mailbox Retention Tag Office 365 Retention Label

Remove Yes, when retention period expires. Yes, when retention period expires.
content
from
mailboxes

Archive Yes, when retention period expires No. Labels do not support an
content archival action.
from
mailboxes

Expiration No. A retention tag stays with an item until it is removed from the Yes. The effect of a label ceases
of control mailbox. when it expires.

Policy- Yes. Retention policies can include folder tags for any of the Exchange No. A label functions like a
driven default folders (like Inbox). personal retention tag and can be
tagging of applied to any other folder except
default default folders.
folders

User- Yes. Retention policies can include personal tags for users to stamp on Yes. A label can be used in the
driven non-default folders and individual items. same way as a personal tag to mark
tagging of items or folders.
items and
folders

Policy- Yes. Retention policies can include default tags that apply to all items in No. It is possible to create an auto-
driven the mailbox that are not stamped with a more explicit tag. A policy can label policy that applies to all items
default include default tags to define removal and archival actions and you can in the mailbox, but that is different
retention have a specific default tag for voicemail. from a default tag.

User- Yes. Users can select personal tags (through OWA options) that are not in No. Users do not have access to
selectable the assigned retention policy and use them to tag items and folders. labels that do not apply to their
tags mailbox.

Target Limited to Exchange mailbox folders, items within a conversation, As for retention tags, with the
individual items, or complete mailboxes. addition that labels can be used
elsewhere within Office 365.

Table 19-3: Differences between Exchange retention tags and Office 365 retention labels

The last point is the most important. Retention policies and tags only cover Exchange content. OWA and Outlook desktop
clients integrate Office 365 labels with the tags published through Exchange mailbox policies. Meaning users can apply
labels to email items in the same way as they use retention tags. Of course, the big difference between labels and tags is
that labels are available in other Office 365 workloads.

Using Retention Labels with Office 365 Groups

When you publish retention labels to an Office 365 Group, they become available in both the group mailbox and the group
document library. At the time of writing, OWA is the only client that supports the application of labels to conversations in
Office 365 Groups. Only group owners can use labels to classify conversations as OWA hides the labels to ordinary group
members.

However, all group members can assign retention labels to files and folders in the group document library and any group
member can overwrite a label previously assigned by another group member, except if that label is a formal record (in
which case only the site collection administrator can update the item). The reasons why labels behave differently in a
group’s mailbox and document library are because of the different ways that Exchange retention tags and SharePoint
permissions work. An Exchange mailbox is typically under the sole control of its owner while a SharePoint site is designed
to support multiple levels of shared access.

Yammer-based groups do not support the application of labels to their conversations, but you can apply the labels to files
in the document library belonging to the Yammer group.

Using Retention Labels with SharePoint and OneDrive for Business

Any user in the default members group for a SharePoint site (with the Contribute permission level) can apply labels to
documents, folders, or items in a list within the site. If the site belongs to an Office 365 Group, any of the members can
apply labels because they all have equal access to the site. Obviously, the owner of a OneDrive for Business site can apply
labels to content in their site. When a label is applied to a folder, all items in the folder inherit the same label, unless a file
already has a label. Applying a label to a folder holding thousands of files can take a little time to complete. A label
inherited from a folder stays with a document even when the document is moved to another folder. Also, if you upload a
file to replace an existing document that has a label, the uploaded file inherits the label.

To apply a label to an item in OneDrive for Business or SharePoint, select the document or folder and then open the
Details pane. Go to the Apply Label section and select a label from the set published to the location. Placing a label with a
retention action on a document or folder has some consequences:

Users cannot remove the document or folder from the library. Before the user can remove the document or folder, the
site administrator (or user, if they have permission) must remove the label or replace it with another label that does
not have a retention action. The logic here is that someone made an explicit decision to place a label with a retention
action on a document and another explicit action is necessary to remove the label.
 Users cannot edit documents marked with a label that classifies items as a formal record. They can update documents
marked with labels that have retention actions but are not formal records. Site administrators can remove the label
or replace it with another if users need to edit the item.
SharePoint records document updates and deletions as items in the Preservation Hold library for the site. Only site
collection administrators can access the preservation hold library (by adding /PreservationHoldLibrary to the URL for
the site). SharePoint only records these items when someone removes a file as otherwise the file and any versions are
available in the regular document library. A single Preservation Hold library holds records for activity in all folders
within the library.
If a document is subject to a retention policy, users can remove it, but a copy of the item must still be kept. In these
instances, Office 365 moves the deleted document into the preservation hold library and keeps it there until the
retention period expires.

The behavior is different for files held in OneDrive for business because these sites are personal rather than shared and
the owner of the site has the right to remove files from the site. When a user removes a file from their OneDrive for
Business site, the file goes into the site recycle bin. Thereafter, when the retention period for the recycle bin expires (93
days), a timer job examines expired items and moves copies of any item with a label into the site’s Preservation Hold
library.

A label is a managed property that SharePoint indexes along with other document attributes. Indexing occurs when the
crawler accesses new or updated documents. It can take up to an hour before a newly-assigned label is in the index. When
indexed, you can search for documents that have a certain label by using the compliancetag property. For example, you
can input “compliancetag:”GDPR Personal Data ” in the SharePoint search box to find documents stamped with the GDPR
Personal Data label. You can also use the property in content searches.

The Effect of Retention on OneDrive Synchronization

When you assign a label with a retention period to a document, SharePoint or OneDrive will not allow users to remove the
document from the library. Any attempt to remove the document results in the error “the label that’s applied to this
document prevents it from being edited or deleted .” In fact, this error relates to items marked as records rather than
items kept by policy. However, the same outcome results and the user cannot remove the item until they clear the label
(apply “None”) or assign a label that does not have an associated retention period.

Because SharePoint blocks the removal of any label with a retention period, the synchronization of file deletions from the
OneDrive sync client fail. The OneDrive sync client does not understand labels and a user can remove the files in its local
cache. However, when the sync client uploads the deletion to the server, OneDrive declines to remove the file because the
label is in place and resynchronizes the file with the client. To solve the problem, you must remove the labels from the
affected files and remove them using the browser.

Displaying Retention Labels in a Site

Because labels are not one of the default fields shown for SharePoint document libraries or lists, you can make the labels
more obvious by customizing the view of items in the library or list to include the “Labels” and (if necessary), “Item is a
record” fields. Figure 19-17 shows how label information appears in a document library. Unlike when you update
properties like a document’s name or title, SharePoint does not treat applying a label as a modification. Instead, it is more
of an administrative event. SharePoint therefore does not update the “Modified By” property with the name of the person
who applies the label.

Figure 19-17: Viewing label information in a SharePoint document library

An organization might have many retention labels in active use at any time. It can be confusing for users to have to choose
between multiple labels when they classify documents. In addition, some labels might be inappropriate for contents of
certain sites. With these points in mind, it is sensible to consider what sites should receive a label when you publish it.
Some labels are general purpose and are useful in all sites while others are better if restricted to specific sites.
Audit Records Generated When Retention Labels are Applied
Every time someone assigns, changes, or removes a label for a document or folder in a SharePoint Online or OneDrive for
Business library or list, an Office 365 ComplianceSettingChanged audit record captures the event. SharePoint also
generates audit records when label policies apply labels to documents. Records captured for label actions in a library
include the site, the document name, the user, and the name of the label. If the document was previously classified with a
label, the audit record captures both the original (SourceLabel ) and new (DestinationLabel ) classifications. For label
actions involving items in a list, SharePoint writes the item number into the audit event, so you see values like:

https://office365itpros.sharepoint.com/sites/GDPRPlanningMarkII/Lists/Things to do/2_.000/2

Retention labels also show up in Exchange Online, where they appear as personal retention tags. However, Exchange
doesn’t generate audit records when users apply retention tags to messages.

Because many organizations use labels to track documents holding personal data that come under the remit of GDPR, the
GDPR dashboard in the Security and Compliance Center includes several widgets to display information about different
aspects of label usage within the tenant. For example, you can find out how many labels have been applied recently, the
top labels used (and the people who apply labels), and even a “Risky labels activity” widget that tracks changes to labels
applied to documents or folders. You can add these widgets to the Security and Compliance home page or the Data
Governance dashboard.

Label Activity Explorer

The Label Activity Explorer (Figure 19-18 ) uses the audit records generated when labels are applied to documents,
folders, and lists to give a snapshot of the labels in use and who is using labels. Because the data comes from the Office
365 audit log, it takes about fifteen minutes before the audit log processes and displays label events from SharePoint.
Using a 30-day sliding window, the explorer gives an insight into:

How labels were applied (manually by a user or by a label policy).

Who applied, changed, or removed a label. If a label was changed on an item, you see details of the old and new label.
The workload (SharePoint or OneDrive for Business).
The target document or folder (the data does not show labels inherited by files after being placed on a folder). For list
items, you see the number of the item in the list.

You can filter by date range, label activity (all actions or just label changes), user, workload, or label to focus in on specific
activity. To see added information, click on an individual record to open the details pane.

Figure 19-18: The Label Activity Explorer reveals who’s using labels

Sometimes it’s good to create your own view on this data, so Chapter 21 includes an example of using PowerShell to
extract and interpret Office 365 audit records that capture details of how users assign labels.

Default Assignment of Retention Labels

The Outlook and OWA clients include the necessary user interface to make retention tags available to users, including
those published by Office 365 retention policies. This is valuable because users get a clear visual understanding of how
long Office 365 will keep an item. However, retention tags do not exist inside SharePoint or OneDrive for Business, so the
situation exists that users might not realize that the organization has a retention policy for information. Unless they
receive some coaching on how and when to apply labels, users might overlook the importance of keeping valuable
documents. While this coaching happens, we can help users by making sure that documents receive labels without user
involvement.

If you have Office 365 E5 licenses, one solution is to classify items by using an automatic label policy to find documents
using searches based on keywords or sensitive data types. This is a good way to make sure that documents with certain
characteristics (like HR personnel files) are classified no matter where they are stored within SharePoint.

To ensure that all documents in a SharePoint or OneDrive for Business library are classified, you can edit the library
settings, select Apply label to items in this list or library, and then choose the default label from the set of retention
labels in the tenant (Figure 19-19 ). In addition to assigning a default label to new items, you can choose to have
SharePoint or OneDrive for Business apply the selected label to existing items. If you need to add default retention labels
to multiple libraries, it’s possible to use PowerShell to script the assignment of a default label to a SharePoint library.

Figure 19-19: Applying a default retention label to a SharePoint library

You cannot use a retention label that marks items as records as the default for a SharePoint site. If any item is already
labelled, it does not get the default retention label for the library. This is because an explicitly-assigned label always takes
precedence over an auto-assigned label. In the same way, if you change the default retention label for a library, the items
in the library inherit the newly assigned label unless someone has already assigned them an explicit label. If users move
items stamped with the default retention label to another library that has a different default retention label inherit the
default label from their new location. If items move to a library that doesn’t have a default label, SharePoint sets the label
to “None.”

Finally, you can apply a retention label to a folder. In this case, all the files in the folder inherit the same label unless they
already have a label. If you need to change the label for specific files, you can remove the default label and replace it with
a better label. This feature also works for OneDrive for Business accounts. A small delay occurs after you apply a label to a
folder before the labels show up for the items in the folder to allow SharePoint or OneDrive to update the items with the
label.

OneDrive for Business Library Settings : If you want to set a default retention label for a OneDrive for Business
account, you must use the old-style OneDrive interface to access library settings. Library settings are available when you
expose the ribbon, select the library tab, and then library settings. You can then select a retention label as described for
SharePoint above.

Record Labels
Retention policies and labels play a key role in the ability of Office 365 to meet the requirements of Rule 17A-4 of the U.S.
Security and Exchange Commission (SEC), which says that companies that employ brokers, dealers, and other workers in
the financial industry must keep records of their electronic communications for between three and six years, depending on
the type of communication. Among the requirements set out are that the records should not be amendable or erasable by
an administrator.
You can use labels to mark specific items as records. To do this, you must opt for “use label to classify content as a record”
setting when you create or update a label (the choice to make a label a record is visible in Figure 19-10 ). Choosing this
setting transforms the label from a simple label to be a record label. Administrators can extend the retention period for a
record label and can add new locations to the scope of the label policy, but they cannot remove the label from any policy
where it is present.

When you assign a record label to an item or folder, a user cannot remove or change the item until the retention period for
the label expires. SharePoint shows the locked status with a small padlock on the item or folder icon (Figure 19-20 ).

Figure 19-20: Documents in a SharePoint library marked as permanent records

From a user perspective, marking content as a record means that they cannot:

Permanently remove the item. If you try to remove an Exchange item, Exchange copies the item to the Recoverable
Items structure and kept there until its retention period expires. If you try to remove an item marked as a record in
SharePoint, an error is flagged, and the item is unchanged. If you try to remove a record from a OneDrive library,
OneDrive moves the item into the special Preservation Hold library where it stays until its retention period expires.
Edit the item.
Change the properties of the item.
Remove the label.

Only a deliberate action taken by a user can mark an item as a record. You cannot use an auto-label policy to scan for an
apply record labels to items. If you apply a record label to an Exchange folder, all the items stored in the folder
automatically become records, even if the user later moves those items out of the folder.

Owners of OneDrive for Business sites can apply and remove record labels to any item under their control. They can also
remove items marked as records. However, for SharePoint content, while site members can apply a record label to an
item, only the site collection administrator can change the label or update the properties of the item when it has a record
label.

See this white paper for more information about the ability of Exchange Online to meet the data retention requirements of
SEC rule 17A-4.

Processing Manual Dispositions

labels are good for marking Office 365 content for retention or removal, however, sometimes you do not want an
automated process to function without supervision. For instance, you might have a label to mark documentation for
customer projects. Usually the projects finish in a few months and it is certainly safe to remove them after five years.
However, in some more complex or extended projects, you need to keep the documents for longer. A label that removes all
documents classified as project documentation after five years would not work. The same might be true for items that the
company might need for litigation or audit purposes.

Manual disposition means that human intervention is necessary to check expired content to decide whether the business
still needs the items or if the deletion can happen. A workflow notifies one or more expert reviewers, nominated because
they have the skills needed to make decisions about content, when the retention period expires for marked items. The
expert then decides to keep the material or to approve its removal. The review process also helps you to understand
whether people are applying labels correctly. For instance, if you see documents stamped with a label that is obviously
inappropriate, you might ask why people use the label in error and then take steps to update procedures or change
behavior.

As this SharePoint blog from 2006 reveals, manual disposition is not a new idea. What is new is that the latest update to
labels means that you can mark content for manual disposition from all the Office 365 workloads covered by data
governance instead of just SharePoint. For now, Microsoft has enabled “Disposition Review” for SharePoint and OneDrive
for Business with the intention of supporting other workloads later, Exchange, Teams, and Office 365 Groups being
obvious targets.

Before manual disposition was available, the actions available to SharePoint or Exchange when an item’s retention period
expires were to remove the item or do nothing. Now, when the retention period expires, if the content triggers a
disposition review, notifications go to the individuals selected in the label settings to tell them that content awaits their
decision. As noted above, at present, you can only trigger a disposition review for SharePoint and OneDrive for Business
content. You can apply the same labels to Exchange items but these items will not trigger a review and Exchange removes
the items when their retention period expires.

The GUI allows you to use a distribution group to nominate experts, but not an Office 365 Group or security group. If you
want to use an Office 365 Group or security group to define reviewers, you must connect to the Security and Compliance
Center with PowerShell and run the New-ComplianceTag cmdlet to create the label, passing the name of the group in the
ReviewerEmail parameter. When you specify a reviewer email address for a new label, Office 365 knows that this means
that the label triggers a disposition review.

Reviewers must have accounts with cloud mailboxes in the tenant. They must also be a member of a role group that
includes the Disposition Management and View-Only Audit Logs roles. At the time of writing, these groups are:

Organization Management.
Compliance Administrator.

If you wish, you can create a new role group and assign the necessary roles and members to that group.

Deciding Whether to Delete

When an item marked with a label that triggers the manual disposition process reaches the end of its retention period,
Office 365 halts the deletion process and notifies the people specified in the label settings that items are available for their
examination. The reviewers can then go to the Dispositions section under Data governance in the Security and
Compliance Center to view the waiting items (Figure 19-21 ). Not shown here is the choice to export details of items
awaiting disposition to a CSV file that experts can use to make decisions that are later actioned by someone else such as a
compliance administrator.

Figure 19-21: Processing an item marked for manual disposition

The options available to deal with reviewed items are:

Delete permanently : The organization no longer needs this content. When a reviewer approves an item for final
deletion, Office 365 releases the block on permanent deletion and logs the action in an ApproveForDelete audit
record. The workload-specific jobs that remove expired items from the workloads remove the items the next time they
process the host location. If the item is in a document library when its retention period expires, marking it for
deletion means that the item goes into the first-stage Recycle Bin within 7 days of the disposition decision. On the
other hand, if the item was already deleted and is in a site’s preservation hold library, it is eligible for immediate
deletion and will be removed within 7 days. Office 365 captures the actual file deletion in a Deleted file or Deleted file
from second-stage recycle bin audit record. The delay in deletion is because of the need to run background jobs to
process the disposition decisions.
Apply another label : The organization should keep this content. The other label might not have a retention action
or have a longer retention period.
 Extend retention period : Keep the original label but extend the retention period to a specific date (a one-year
extension is the default) after which the item will go through the review process again. This action overwrites the
computed retention date for the item with the retention date selected by the reviewer. Office 365 records the
extension in an ExtendRetention audit record. The audit record does not capture the new retention date.

To help them decide how to dispose of an item, the reviewer can click the link to the item to view its content. However, a
reviewer can only view content if they have the necessary permission. Office 365 does not reveal content to a reviewer if
they cannot access the location where it belongs, which means that sometimes a reviewer will have to consult with the
owner of the content to decide on its disposition. Office 365 moves items that a reviewer extends or assigns another label
to back into their original location.

To help them keep up to date, reviewers receive email notifications after items go through disposition review. Reviewers
can see details of Items that they or other reviewers previously authorized for deletion by selecting “Completed
dispositions ” in the Show drop-down box. This view only shows items that reviewers approved for disposition that are
awaiting final deletion. It does not show items where the reviewer decided to apply a different label or extend the
retention period.

Obviously, a busy tenant can generate a heavy workload of review items if disposition review is the norm rather than an
exception. For this reason, users should receive training about when they should apply labels that trigger reviews.
Reviewers also need training to understand how to deal with items awaiting their attention so that they know when they
can authorize deletion or when they need to seek further guidance from the business about how to handle items.

Event-based Retention
Labels normally use age-based retention periods and invoke retention actions based on the creation or last modified date
of content. Event-based retention takes a different approach and waits until a specific event occurs before starting the
retention countdown for items. For instance, let’s assume that you want to preserve all project documents for seven years
after a project completes. The event is the project completion, which the project manager might have to sign off. The
retention period begins as soon as the event occurs.

Because it depends on something happening rather than the passing of time, event-based retention is more complex than
date-based retention. Here is the general flow of what happens.

1. The administrator creates a new label and selects “an event” as the decision point for the retention period rather
than the usual “date created” or “date modified” as used with other labels.
2. The administrator selects an event type (which must already exist) to associate with the label. An event is something
like, the expiration of a contract or the departure of an employee or any other common occurrence in the life of a
business. Office 365 includes a set of pre-packaged event types, but you can create new event types if needed.
3. After saving the label, the administrator includes it in a label policy and publishes the policy to make it available to
end users. After an hour or so, the label is available to SharePoint and OneDrive for Business. It takes a little longer
for the label to appear in Exchange.
4. Users apply the label to content that they want to link with the event. For example, they might look for the set of
documents belonging to a contract and apply the label to those documents.
5. When they apply the label, users also give a value to a field called “Asset ID,” which is part of the standard SharePoint
Online schema. A label for an event type can be used with many different events, so a mechanism is necessary to
isolate the content belonging to a specific event. The Asset ID is used to identify individual projects, tasks, or other
entities. For instance, if the event deals with the departure of an employee, the Asset ID might hold the employee’s
number. It is critical that the Asset ID is populated correctly because this is the value used to find content associated
with an event. You can find out what items are stamped for a specific event by using SharePoint search or Office 365
content searches using the complianceassetid tag. For example, find items with complianceassetid:PK1 .
6. When an event occurs, like an employee leaving or a contract reaching its end, the administrator goes to Events
under Data Governance in the Security and Compliance Center and triggers compliance processing. They select the
event type to use, or choose an existing label configured for event-based retention used to classify items. To find the
items for the event, they input the associated Asset ID (for SharePoint and OneDrive items in the form
complianceassetid:<value> ) and/or keywords to locate Exchange items. If other keywords are needed to find the
relevant items, they can be entered at this stage (for instance, a tenant might already have assigned a different form
of asset identifiers to project documentation). Finally, they select the date the event happened and save the event.
7. Office 365 background processes now start looking for content matching the event in SharePoint, OneDrive, and
Exchange (in effect, a content search is run). As the search finds matching items, the retention action and period
specified in the label are applied to the items. Once the items are stamped, normal retention processing begins. For
instance, if the label states that items should be kept for five years, Office 365 keeps the items for five years after the
event date. It can take up to a week before the background processes find all matching content across Office 365
workloads.

Once created, you cannot update or remove an event, so you should be sure that everything is ready to find the items
relating to an event when you create it. In addition, once you associate a label with an event, you cannot change the label
to associate it with a different event. For these reasons, it is important to have a good understanding of the business
events that occur within the tenant and how people working in the business can use labels to aid the processing of content
associated with before using event-based labels. For more information about event-based retention, see the Microsoft
support article .

Removing Retention Labels

Four scenarios occur for removing retention labels from a tenant:
 A label is created, but not published . Because the label is not in use, you can edit the label in the Security and
Compliance Center and remove it with the Delete label option. Alternatively, run the Remove-ComplianceTag cmdlet.

[PS[ C:\> Remove-ComplianceTag -Identity "Bad Label"

A label is published in one or more policies but has never been assigned to items . If you try and remove the
label, you’ll see an error that the label is in use. This is technically correct because the label is in a policy even if it
has never been assigned to items. To remove the label, you first remove it from all the policies it is included in (or
remove a complete policy if the label is the only one in that policy). After removing the label from all policies, you can
remove the label as described above.

A label is in active use and applied to content . You can run a content search to find all the items that have the
label (find compliance tag equal to the label name), but this is unreasonable in a tenant of any size. Instead, you can
follow the procedure as if the label was never assigned to any items by removing it from all policies and then
removing the label. The label then goes into a pending deletion state, which means that some background processes
in the different workloads must run to remove the label from items. For SharePoint and OneDrive, the process is a
timer job and can take several hours before it completes. For Exchange, the process is the Managed Folder Assistant
and it might take up to a week before it processes a mailbox.

A label is a record . As noted above, an item assigned a record label cannot be changed. You cannot remove any
label marked as a record (even if the label has never been assigned to an item), but you can stop people using it again
by removing the label from any policies that it is in.

Removing labels is not something to be done at a whim. The complexities involved in removing labels that are applied to
content underline the need for planning and preparation for the deployment of labels as part of your data governance
strategy. Removing labels from items can result in their deletion by background processes because their retention period
has expired, so if you remove a label from content, you might need to replace that labels with different labels to ensure
that the items are kept.

Finding Items Marked with Labels in

Content Searches
You can create content searches (either standalone or part of an eDiscovery case) to find content marked with specific
labels. You do this by using the “ComplianceTag” keyword in searches. For example, this search finds any items stamped
with the “Draft” or “Approved” labels.

(ComplianceTag:Draft) OR (ComplianceTag:Approved)

If the label name contains spaces, enclose it in quote marks. See Chapter 20 for more information about content searches.

Using the Compliance PowerShell

cmdlets
A set of cmdlets is available to manage labels and retention policies. To access the cmdlets, you must connect a
PowerShell session to the Security and Compliance Center endpoint as described in Chapter 4. Some of the cmdlets are
also available when connected to Exchange Online, so it is sensible to use a command prefix to show against which
endpoint you want to run a command. Here is a basic function that you can insert into your PowerShell profile to connect
to the Security and Compliance Center endpoint.

Function ConnectCC

# Helper function to connect to the Office 365 Security and Compliance Center and use their set # of PowerShell cmdlets

$global:O365Cred = Get-Credential

$global:CCSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri

https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $O365Cred -Authentication Basic –
AllowRedirection

Import-PSSession $CCSession –AllowClobber -DisableNameChecking

We use the AllowClobber and DisableNameChecking parameters because some of the cmdlets imported when you connect
to the Security and Compliance Center might already exist in the PowerShell session, especially if you have already
connected to Exchange Online. AllowClobber lets PowerShell import cmdlets that already exist. DisableNameChecking
suppresses the warning that PowerShell otherwise signals when duplicate cmdlet names are met.

It is not the intention to have a detailed and in-depth discussion of all the cmdlets here as it would occupy too much space.
In any case, given the complexity of some of the operations involving compliance, it is usually best to create and update
policies and labels through the GUI.

Working with Retention Labels

The *-ComplianceTag cmdlets expose the properties of retention labels. In this example, we extract the set of retention
labels defined in a tenant and list the most important properties of each tag to understand if a tag is used to mark an item
as a record, has a retention action, the retention duration (in days), the retention action, and whether the tag is in use.

[PS] C:\> Get-ComplianceTag | Format-Table Name, IsRecordLabel, HasRetentionAction, RetentionDuration, RetentionAction, Mode –AutoSize

Name IsRecordLabel HasRetentionAction RetentionDuration RetentionAction Mode

---- ------------- ------------------ ----------------- --------------- ----

Confidential False True 1825 Keep Enforce

Remove after 1 week False True 7 Delete Enforce

Patent Materials False True 7300 Keep Enforce

Archive Retention False True 3650 Keep Enforce

Required for Audit True True 16425 Keep Enforce

Board Records False True Unlimited Keep Enforce

Formal Company Record True True 7300 KeepAndDelete Enforce

The New-ComplianceTag cmdlet creates a new retention label. In this example, we create a retention label to keep content
for ten years. This retention label does not mark content as a formal record.

[PS] C:\> New-ComplianceTag -Name "Patent Information" -IsRecordLabel $False -RetentionDuration 3650 - RetentionAction Keep -Comment
"Items marked with this classification are associated with patents"

-RetentionType ModificationAgeInDays

The Set-ComplianceTag cmdlet updates the properties of a retention label while the Remove-ComplianceTag cmdlet
removes a label from a tenant. For example, this command sets the retention duration for the “Patent Materials” label to
15,000 days (the maximum is 24,855).

[PS] C:\> Set-ComplianceTag -Identity "Patent Materials" -RetentionDuration 15000

Working with Retention Policies

All the policies surfaced in the Security and Compliance Center are designed with a parent/child structure. The parent is
the policy itself and the child is the set of rules that implement the policy settings. Another way of thinking about this is
that the parent for a policy defines the overall data to which the policy applies while the rules say how the policy is
applied. In fact, it is a little simpler than it seems because a 1-to-1 relationship usually exists between retention policies
and retention rules (including label policies). The links between retention policies and rules is hidden by the GUI but
needs to be understood when we start to work with these policies through PowerShell. Two cmdlet sets are used:

The *-RetentionCompliancePolicy cmdlet set manipulates retention policies. Use these cmdlets to manipulate the
workload locations to which a policy applies or to enable or disable a policy.
The *-RetentionComplianceRule cmdlet manipulates the rules for retention policies. Use these cmdlets to work with
properties such as the retention duration of a policy.

When you fetch details of retention policies with the Get-RetentionCompliancePolicy cmdlet, the set returned also includes
label policies. The difference between the two is that label policies publish labels to workloads while retention policies
impose retention requirements on workloads. The policies to publish retention labels to workloads have multiple rules –
one for each label published by the policy.

Fetching Retention Policies

For example, here is how to retrieve the basic properties of a retention policy using the Get-RetentionCompliancePolicy
cmdlet:

[PS] C:\> Get-RetentionCompliancePolicy –Identity 'GDPR Personal Data' -DistributionDetail | Format-List ExchangeLocation,
SharePointLocation, OneDriveLocation, ModernGroupLocation, TeamChatLocation, TeamChannelLocation, Workload, Enabled, Mode,
RestrictiveRetention, DistributionStatus

ExchangeLocation : { All}

SharePointLocation : { All}

OneDriveLocation : {}

ModernGroupLocation : {}

TeamChatLocation : {}

TeamChannelLocation : {}

Workload : Exchange, SharePoint , OneDriveForBusiness, Skype, ModernGroup

Enabled : True

Mode : Enforce

RestrictiveRetention: False

DistributionStatus : Success

We can interpret the output as follows:

ExchangeLocation : Lists the names of the mailboxes covered by the policy.

SharePointLocation : Lists the SharePoint Online sites covered by the policy.
ModernGroupLocation : Lists the aliases for the Office 365 Groups covered by the policy.
OneDriveLocation : Lists the OneDrive for Business locations covered by the policy.
TeamChatLocation : Lists the locations for personal Team chats covered by the policy.
TeamChannelLocation : Lists the Teams (all channels) covered by the policy.
Workload : Lists the workloads where the policy is active. “Skype” means that Skype for Business is covered, but
before the policy is effective, you also must select the accounts covered by the policy, in which case the accounts are
listed in the SkypeLocation property (not shown here).
Enabled : Tells you whether the policy is active or not. The default is $True , but if a policy has been disabled for
some reason, this value is $False .
Mode : If “Enforce”, the policy is active.
RestrictiveRetention : The default is False, meaning that the policy can be changed. If True, a preservation lock
exists on the policy, meaning that administrators can only make limited changes to the policy.
DistributionStatus : Success means that the different workloads have the information needed to enforce the policy
settings. Pending means that Office 365 is distributing a change to a policy. Usually it takes longer for SharePoint
Online to enforce a policy than it does for Exchange Online. Any other value shows that a problem has occurred in the
policy distribution. Sometimes this happens for good reason, such as a change occurring in an Office 365 datacenter.
If you see problems when distributing labels to certain locations, it might be possible to figure out where the problem
lies by examining the DistributionResults property for the policy. You won’t see the current distribution status or any
errors unless you include the DistrubutionDetail parameter (see below).

If you see “All” listed as a value for a workload, it means that the policy covers every location supported by that workload.
For example, “All” listed in ExchangeLocation means that the policy covers every Exchange mailbox in the tenant. You can
exclude specific mailboxes or sites in a workload from the policy. If this is the case, you will find a list of those locations in
the properties ExchangeLocationException , SharePointLocationException and so on.

Including Distribution Details

When you look at retention policies with the Get-RetentionCompliancePolicy cmdlet, you won’t see details of the individual
mailboxes or sites specified for locations, the up-to-date distribution status, or any error information unless you include
the DistributionDetail parameter. To see details of locations or errors, you need to expand the relevant property for the
location you want to examine. For example, here’s how to examine details of the mailboxes that a policy applies to.

[PS] C:\ Get-RetentionCompliancePolicy -Identity "Senior Leadership Team" -DistributionDetail | Select -ExpandProperty ExchangeLocation

DisplayName : Kim Akers

Name : Kim.Akers@Office365itpros.com

ImmutableIdentity : f120e18f-8305-41e3-abd4-de93d4a2a493

Type : IndividualResource

Workload : Exchange

SchemaVersion : 2

DisplayName : Brian Weakliam

Name : Brian.Weakliam@office365itpros.com

ImmutableIdentity : aae8332a-6832-4c00-b873-6ec443c36395
Type : IndividualResource

Workload : Exchange

SchemaVersion : 2

Here’s an example of looking at retention policies that have encountered problems when distributed to workloads. Often
these are transient problem caused by a recipient selected for the policy being unavailable for some reason that you can
fix by editing the policy to re-add the recipient.

[PS] C:\> $Errors = (Get-RetentionCompliancePolicy -DistributionDetail | ? {$_.DistributionStatus -eq "Error"})

ForEach ($E in $Errors) {

$Results = ($E | Select -ExpandProperty DistributionResults)

Write-Host "Policy:" $E.Name "Issue:" $Results }

Policy: Company Confidential Policy Issue: [Exchange]SMO-Academy@office365itpros.onmicrosoft.com:Recipient not found: f120e18f-830

5-41e3-abd4-de93d4a2a493

Policy: Black Matter Policy Issue: [ModernGroup]'ModernGroup' Resources:Policy deployment has been interrupted by an unexpected Office

365 datacenter issue. Please contact Microsoft support to fix the deployment issue. [ModernGroup]BlackMatterTeam@office365itpros.com

:Recipient not found: 6661b878-83b5-41bb-aad4-ba14e8879b90

Reporting Retention Policies Applied to SharePoint

As an example of using the distribution detail information for retention policies, let’s say that you want to know what
retention policies apply to SharePoint sites. This code fetches information about the retention policies including their
distribution detail, excluding retention policies for Teams, those that don’t process SharePoint, and policies used to
publish retention labels. We then examine each policy to extract the locations within the policy scope and figure out
whether the retention settings are simple or advanced (using a keyword query or sensitive data type). Some policies apply
to every SharePoint site, so the location is “All.” Others have specific SharePoint sites defined, and some policies process
everything except a set of excluded sites.

[PS] C:\> $Report = @()

$RetentionPolicies = (Get-RetentionCompliancePolicy -ExcludeTeamsPolicy -DistributionDetail | ? {$_.SharePointLocation -ne $Null})

# Now exclude all the retention policies that publish labels

$Policies = @()

ForEach ($P in $RetentionPolicies) {

$Rule = Get-RetentionComplianceRule -Policy $P.Name

If ([string]::IsNullOrWhiteSpace($Rule.RetentionDuration) -and [string]::IsNullOrWhiteSpace($Rule.ApplyComplianceTag)) {

Write-Host "Policy" $P.Name "publishes retention labels to workloads - excluded from this report" }

Else {

$Policies += $P }

# Now we have a cleansed set of retention policies that apply to SharePoint

ForEach ($P in $Policies) {

$Duration = $Null

Write-Host "Processing retention policy" $P.Name

$Rule = Get-RetentionComplianceRule -Policy $P.Name

$Settings = "Simple"

$Duration = $Rule.RetentionDuration

# Check whether a rule is for advanced settings - either a KQL query or sensitive data types

If (-not [string]::IsNullOrWhiteSpace($Rule.ContentMatchQuery) -and -not [string]::IsNullOrWhiteSpace($Rule.ContentMatchQuery)) {

$Settings = "Advanced/KQL" }

Elseif (-not [string]::IsNullOrWhiteSpace($Rule.ContentContainsSensitiveInformation) -and -not

[string]::IsNullOrWhiteSpace($Rule.ContentContainsSensitiveInformation)) {
 $Settings = "Advanced/Sensitive Data" }

# Handle retention policy using advanced settings (keyword search or sensitive data type)

If ($Rule.RetentionDuration -eq $Null -and $Rule.ApplyComplianceTag -ne $Null) {

$Duration = (Get-ComplianceTag -Identity $Rule.ApplyComplianceTag | Select -Expandproperty RetentionDuration) }

$RetentionAction = $Rule.RetentionComplianceAction

If ([string]::IsNullOrEmpty($RetentionAction)) {

$RetentionAction = "Retain" }

If ($P.SharePointLocation.Name -eq "All") {

$ReportLine = [PSCustomObject][Ordered]@{

PolicyName = $P.Name

SiteName = "All SharePoint Sites"

SiteURL = "All SharePoint Sites"

RetentionTime = $Rule.RetentionDurationDisplayHint

RetentionDuration = $Duration

RetentionAction = $RetentionAction

Settings = $Settings}

$Report += $ReportLine }

If ($P.SharePointLocationException -ne $Null) {

$Locations = ($P | Select -ExpandProperty SharePointLocationException)

ForEach ($L in $Locations) {

$Exception = "*Exclude* " + $L.DisplayName

$ReportLine = [PSCustomObject][Ordered]@{

PolicyName = $P.Name

SiteName = $Exception

SiteURL = $L.Name }

$Report += $ReportLine }

ElseIf ($P.SharePointLocation.Name -ne "All") {

$Locations = ($P | Select -ExpandProperty SharePointLocation)

ForEach ($L in $Locations) {

$ReportLine = [PSCustomObject][Ordered]@{

PolicyName = $P.Name

SiteName = $L.DisplayName

SiteURL = $L.Name

RetentionTime = $Rule.RetentionDurationDisplayHint

RetentionDuration = $Duration

RetentionAction = $RetentionAction

Settings = $Settings}

$Report += $ReportLine }

$Report | Sort SiteName| Format-Table PolicyName, SiteName, RetentionDuration, RetentionAction, Settings -AutoSize
The output is an ordered array. We can look at the data in different ways (Figure 19-22 is an example) or export it to a CSV
file to load into Excel or Power BI.

Figure 19-22: Analyzing the Retention Policies applied to SharePoint

Viewing Teams Retention Policies

Remember that a Teams retention policy can only cover Teams personal chats and channel conversations and a general
retention policy applied to other locations cannot cover Teams. If you only want to work with retention policies that affect
Teams, use the TeamsPolicyOnly parameter when fetching retention policies:

[PS] C:\> Get-RetentionCompliancePolicy -TeamsPolicyOnly

Likewise, to exclude the Teams policies, use the ExcludeTeamsPolicy parameter:

[PS] C:\> Get-RetentionCompliancePolicy -ExcludeTeamsPolicy

Setting Retention Policies

You can force Office 365 to republish a policy to the workloads by running the Set-RetentionCompliancePolicy cmdlet. For
example:

[PS] C:\> Set-RetentionCompliancePolicy –Identity 'Patent Materials' -RetryDistribution

The Set-RetentionCompliancePolicy cmdlet can also be used to add or remove workload locations to the policy. In this
example, we remove a mailbox and add a mailbox to the set of Exchange locations. The same is done to remove and add
some SharePoint sites. Note that you must give the URL for each site.

[PS] C:\> Set-RetentionCompliancePolicy –Identity 'Patent lock-down'

–AddExchangeLocation 'Frank Clonan' –RemoveExchangeLocation 'Rob Young'

–RemoveSharePointLocation 'https://office365itpros.sharepoint.com/Projects/'

–AddSharePointLocation ' https://office365itpros.sharepoint.com/Exchange Connections 2015'

Remove a Retention Policy

To remove a retention policy, run the Remove-RetentionCompliancePolicy cmdlet. Remember that you will not be able to
remove a policy if a preservation lock is in place.

Retention Policy Rules

Returning to our discussion about rules, when you create a new retention policy, Office 365 creates the underlying rule for
the policy to instruct workloads how to process content. We can see the rule created for a retention policy by using the
Get-RetentionComplianceRule cmdlet and passing the name of the policy we want to examine:

[PS] C:\> Get-RetentionComplianceRule –Policy 'Patent Materials' | Format-List ContentMatchQuery, RetentionDuration,

RetentionComplianceAction, ExpirationDateOption, Workload

ContentMatchQuery : Patent NEAR(10) claim

Patent NEAR(10) prosecution

Patent NEAR (10) application

Rete ntionDuration : 3650

RetentionComplianceAction: KeepAndDelete

ExpirationDateOption : ModificationAgeInDays

Workload : Exchange, SharePoint, OneDriveForBusiness, Skype, ModernGroup

Some of the output (Workload in this case) matches what you see when examining the policy. The interesting pieces here
are:

ContentMatchQuery : The Keyword Query Language (KQL) query used to determine whether items come under the
scope of the preservation policy. In this case, three separate tests are used.
RetentionDuration : The length of time in days that items will be held. You can also use “unlimited”, in which case
items are held indefinitely until the hold set by the policy is released or the policy is removed from the tenant. The
hold duration is calculated using the created date for email items and the date last modified for files in SharePoint
Online or OneDrive for Business sites.

Rule settings can be changed using the Set-RetentionComplianceRule cmdlet. For example, this command sets the
retention duration for a rule to 3,600 days (the maximum duration is 24,855 days):

[PS] C:\> Set-RetentionComplianceRule -id 86f67249-74b9-48bf-8fe6-9e0c58416dfb -RetentionDuration 3600

Changes made to a rule will not be effective until the policy is updated to all the workloads that it covers. This might take
a few hours.

To publish labels and make them available to workloads, we create a retention policy (called a label policy) and associate a
rule for each label with that policy. In other words, every label published by the policy is represented by a separate rule.
Labels can be found in multiple label policies and the connection between label and rule is through the label GUID. If you
run the Get-RetentionComplianceRule cmdlet to find all the rules belonging to a policy, you can find the different labels by
looking at the ApplyComplianceTag property, which holds the GUID pointing back to the label.

It is possible to create a new retention policy through PowerShell by running the New-RetentionCompliancePolicy cmdlet
to create the policy followed by the New-RetentionComplianceRule cmdlet to create the associated rule. However, it is
normally easier and more effective to use the GUI to create new policies, so this is the recommended approach.

Tracking Retention Holds for Mailboxes

When a non-org-wide retention policy applies an in-place hold to a mailbox, Exchange Online notes the fact by updating
the InPlaceHolds property of the mailbox with the GUID for the hold. Thus, you can get a quick view of what mailboxes are
on hold by checking what’s stored for InPlaceHolds with some PowerShell. For instance:

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox | ? {$_.InPlaceHolds –ne $Null} | Format-Table DisplayName, InPlaceHolds

DisplayName InPlaceHolds

----------- ------------

Tony Redmond {skp748f77b020124e6e8304e66021fb297b:3, mbx748f77b020124e6e8304e66021fb297b:3}

Kevin A. Laahs {UniH26c5d797-0fd3-496d-92ac-4f405700c91 7}

Kim Akers {mbx748f77b020124e6e8304e66021fb297b:3, skp748f77b020124e6e8304e66021fb297b:3, UniHec6...

Ben Owens {mbx748f77b020124e6e8304e66021fb297b:3, skp748f77b020124e6e8304e66021fb297b:3, UniH47f...

One reason why you might want to do check holds set on mailboxes is to find out what holds are keeping inactive
mailboxes alive.

[PS] C:\> Get-Mailbox -RecipientTypeDetails UserMailbox -InActiveMailboxOnly | Format-Table DisplayName, InPlaceHolds

DisplayName InPlaceHolds

----------- ------------

Holly Holt {}

Emma Robinson {}

Frank Clonan {}

James Gangley {mbx29550d04cffd42109bdd94cc56c65041:2}

Jodie Smith (Program Manager) {}

Rob Young {d9eb7052cc0f4200b6a1ad0d6f2171ed...}

Mary Smith (Customer Support) {}

Ed Banti {skp748f77b020124e6e8304e66021fb297b:3}

When you see an inactive mailbox shown with a null value in InPlaceHolds , you know that the hold on the mailbox comes
from an org-wide retention policy rather than a non-org-wide policy. More on this topic in a little while.

To get a full view of the holds that apply to an individual mailbox, use the Get-Mailbox cmdlet and expand InPlaceHolds :
[PS] C:\> Get-Mailbox -Identity "Ben Owens" | Select -ExpandProperty InPlaceHolds

mbx748f77b020124e6e8304e66021fb297b:3

skp748f77b020124e6e8304e66021fb297b:3

UniH47f67751-1036-4621-80d6-d25837adf813

UniHec6163be-6ed6-4b16-afe8-1b2165b9359f

UniH84dea76f-c845-4101-b066-a8b10c13c210

One of the holds listed applies to Skype for Business conversions (skp). Another is for mailbox contents (mbx). If you see a
minus sign before an “mbx” hold, it means that a retention policy explicitly excludes the mailbox.

The other holds starting with “UniH” are “unified holds” and belong to:

A hold placed by an eDiscovery case created through the Security and Compliance Center.
A hold placed by an old preservation policy now upgraded to an Office 365 retention policy.

If a hold shows up as a GUID without a prefix, it belongs to an Exchange in-place hold managed from the EAC. These
include holds created for mailboxes included in eDiscovery cases managed in the SharePoint eDiscovery Center. While you
might come across these GUIDs, they should begin to disappear over time as workload-specific holds expire and are
phased out.

We can use the GUID noted for a hold placed by an Office 365 retention policy to find which policy the hold belongs to. As
we know, tenants manage retention policies through the Security and Compliance Center. To make the policies effective, a
publication process makes the policies known to individual workloads for implementation. When you create a policy, Office
365 assigns the policy a GUID as its identity, so to find which retention policy applies, we pass the GUID noted in the
organization configuration (the first five or six characters are enough) in a command like this:

[PS] C:\> Get-RetentionCompliancePolicy | ? {$_.Guid -like "748f77b*"}

Name Workload Enabled Mode

---- -------- ------- ----

Senior Leadership Team (SLT) Retention Policy Exchange, Skype, ModernGroup True Enforce

We now know that the mailbox comes under the scope of the Senior Leadership Team (SLT) Retention Policy. This is a non-
org wide policy that applies to selected Exchange locations (mailboxes). If we look at the ExchangeLocation property, we
see a list of the mailboxes the policy applies to, or “All” if the policy applies to all mailboxes. For example, here’s a typical
entry for a mailbox.

[PS] C:\> Get-RetentionCompliancePolicy | ? {$_.Guid -like "748f77b*"} | Select -ExpandProperty ExchangeLocation

DisplayName : Tony Redmond

Name : Tony.Redmond@office365itpros.com

ImmutableIdentity : d446f6d7-5728-44f8-9eac-71adb354fc89

Type : IndividualResource

Workload : Exchange

SchemaVersion : 2

The ImmutableIdentity property reported for each mailbox on hold equates to the GUID identifying the user account in
Azure Active Directory and is the same value that you see if you run the Get-Mailbox cmdlet and examine the
ExternalDirectoryObjectID property.

Group Mailboxes and Holds

You cannot look at the hold information for group mailboxes in the same way because Get-Mailbox is unavailable for group
mailboxes and the Get-UnifiedGroup cmdlet does not report holds. Therefore, if we want to discover the retention policies
that apply to specific groups, we must specify the DistributionDetail parameter when calling Get-
RetentionCompliancePolicy to force Office 365 to return details of the locations covered by the policy. For example:

[PS] C:\> Get-RetentionCompliancePolicy -DistributionDetail | ? {$_.ModernGroupLocation -ne $Null

-and $_.ModernGroupLocation -notlike "*All" } | Format-List Name, ModernGroupLocation

Name : Clean up Office 365 Groups with Connectors

ModernGroupLocation : {office365tenantservicehealth, office365roadmapupdates, askhr}

Name : Senior Leadership Team (SLT) Retention Policy

ModernGroupLocation : {Senior Leadership Team}

This technique only works to report details of the holds placed by Office 365 retention policies. We need to process the
other types of holds differently to uncover their secrets. More on this topic in just a moment.

Org-wide Retention Policies

Org-wide retention policies applying to all mailboxes do not stamp hold information on individual mailboxes. Instead, the
Exchange Online organizational configuration for the tenant stores details of the holds belonging to retention policies
applicable to all mailboxes. This approach avoids the need to write the hold information into every mailbox and then check
for individual holds when evaluating items for deletion. Here is how to retrieve information about the org-wide retention
policies that exist in a tenant.

[PS] C:\> Get-OrganizationConfig | Select -ExpandProperty InPlaceHolds

mbx15382841af9f497c83f9efe73e51888d:1

mbx9696959111f74ecda8a40aef97edd2c2:1

mbx703105e3b8804a1093bb5cb777638ea8:1

grp703105e3b8804a1093bb5cb777638ea8:1

mbxc1e2d6f1785d4bf8a7746a26e58e5f66:1

grpc1e2d6f1785d4bf8a7746a26e58e5f66:1

mbxf6a1654abdba4712a43c354e28a4d56c:2

grpf6a1654abdba4712a43c354e28a4d56c:2

In this case, “mbx” refers to policies applied to user mailboxes and “grp” means policies applied to group mailboxes. Five
policies exist here, which apply to all user and group mailboxes.

We can use the same method to take a GUID and use it to trace back to a retention policy. This time we output the
Exchange and Office 365 Group locations covered by the policy:

[PS] C:\> Get-RetentionCompliancePolicy -DistributionDetail | ? {$_.Guid -like "703105e*"} | Format-Table Name, ExchangeLocation,
ModernGroupLocation

Name ExchangeLocation ModernGroupLocation

---- ---------------- -------------------

Office 365 for IT Pros eBook Content {All} {All}

As you can see, both locations have an “All” value, meaning that all user and all group mailboxes come under the scope of
the policy. Another way of looking at the same data is to search for retention policies that apply to all Exchange user
mailboxes and are being enforced.

[PS] C:\> Get-RetentionCompliancePolicy -DistributionDetail | ? {$_.ExchangeLocation -Like "*All*" -and $_.Mode -eq “Enforce”} | Format-
Table Name, Mode, ExchangeLocation, Guid

Name Mode ExchangeLocation Guid

---- ---- ---------------- ----

GDPR Personal Data Enforce {All} 96969591-11f7-4ecd-a8a4-0aef97edd2c2

Company Confidential Policy Enforce {All} f6a1654a-bdba-4712-a43c-354e28a4d56c

Office 365 for IT Pros eBook Content Enforce {All} 703105e3-b880-4a10-93bb-5cb777638ea8

Formal Company Records Enforce {All} c1e2d6f1-785d-4bf8-a774-6a26e58e5f66

Exchange GDPR test Enforce {All} 15382841-af9f-497c-83f9-efe73e51888d

Five policies are returned and the GUIDs reported match the “mbx” holds included in the organizational configuration for
the tenant.

Discovering Holds in Force for a Mailbox

Taking all that we know about the different forms of Office 365 holds that exist into account, we can come up with some
code to report the complete set of holds that are in force for a mailbox. This code reports accepts the name of a mailbox
and reports any organization-wide holds followed by holds where the mailbox is in scope.

[PS] C:\> $User = Read-Host "Enter User to check"

$Mbx = Get-Mailbox -Identity $User -ErrorAction SilentlyContinue

If ($Mbx -eq $Null)

{ Write-Host $User "is not a valid user"

Break

$OrgHolds = (Get-OrganizationConfig).InPlaceHolds

If ($OrgHolds.Count -gt 0) {

Write-Host "The following organization-wide mailbox holds are in force..."

ForEach ($Hold in $OrgHolds) {

$RetentionPolicy = (($Hold -Split ":")[0].Substring(3))

$HoldType = $Hold.Substring(0,3)

Switch ($HoldType)

"grp" {$Type = "Office 365 Groups"}

"mbx" {$Type = "Mailbox"

Get-RetentionCompliancePolicy -Identity $RetentionPolicy | Select Name, Workload }

}}}

$MbxHolds = $Mbx.InPlaceHolds

If ($MbxHolds.Count -gt 0) {

Write-Host ""

Write-Host "The following specific holds are in place on the mailbox..."

ForEach ($MHold in $MbxHolds) {

$RetentionPolicy = (($MHold -Split ":")[0].Substring(3))

$HoldType = $MHold.Substring(0,3)

Switch ($HoldType)

"grp" {$Type = "Office 365 Groups"}

"skp" {$Type = "Skype IM Conversations"}

"uni" {$Type = "Unified Hold"

$CaseHold = $RetentionPolicy.SubString(1)

$Text = (Get-CaseHoldPolicy -Identity $CaseHold -ErrorAction SilentlyContinue).Name

Write-Host "eDiscovery Case:" $Text }

"mbx" {$Type = "Mailbox"

Get-RetentionCompliancePolicy -Identity $RetentionPolicy | Select Name, Workload }

 }

# There might be an old Exchange in place hold...

$int = $Mhold.substring(0,1)

If ([bool]($int -as [int]) -eq $True)

{ $Text = (Get-MailboxSearch -InPlaceHoldIdentity $MHold -ErrorAction SilentlyContinue).Name

Write-Host "Exchange In-Place Hold:" $Text }

}}

If ($Mbx.LitigationHoldEnabled -eq $True) {

Write-Host "Litigation hold is enabled on the mailbox" $Mbx.DisplayName }

Exchange Retention Holds

Apart from in-place holds, Exchange Online supports retention holds. This is not a hold of the type generally referred to as
Exchange in-place or litigation holds, Office 365 retention holds, or Office 365 eDiscovery case holds, all of which stop
information being removed through the action of a user or a system process. An Exchange retention hold stops the
Managed Folder Assistant processing the retention policy for a mailbox for a period and is usually applied when a mailbox
owner cannot manage their mailbox because they are ill, on vacation, or absent for some other reason.

Exchange retention holds are still in use. Because they affect how MFA processes mailboxes, they influence the Office 365
retention policies applied to mailboxes. In other words, if a retention hold stops MFA from processing a mailbox, MFA will
not process either Exchange mailbox retention policies or Office 365 retention policies for that mailbox while the retention
hold exists. To set a retention hold, set the RetentionHoldEnabled property for a mailbox as follows:

[PS] C:\> Set-Mailbox -Identity "Kim Akers" -RetentionHoldEnabled $True

When the user can resume working with their mailbox, it is usual to give them a week or so to allow them to process new
messages awaiting their attention. You can then disable the retention hold by setting the RetentionHoldEnabled property
to $False . MFA will then restart applying retention policies to the mailbox.

Migration
Office 365 data content can have multiple policies controlling its disposition. These policies exist within a tenant but have
no application elsewhere. Therefore, if you move content from an Office 365 tenant, even to another Office 365 tenant, the
effect of retention policies and labels disappear. This problem already exists when users move from on-premises
environments to the cloud and the situation is similar in merger/acquisition scenarios when tenants join or split. The
migration technology available from Microsoft or third parties takes no account of retention policies because the focus is
on extracting information from one tenant to bring it to another. It is unlikely that this situation will change soon due to
the complexities of reconciling retention demands as content moves from one organization to another.

Some basic steps can be taken to help.

Understand how data governance functions within the source tenant (or tenants, in the case of amalgamations). Know
what retention policies exist and how those policies treat content.
Prepare for the migration by understanding where data is stored after the move.
Create a new data governance strategy for the target tenant.
Execute the strategy as data is moved from source tenants so that it is managed from day 1. In other words, have the
retention policies in place in the target tenant so that newly arriving content is preserved as soon as it is transferred.
This task might take some PowerShell scripting.

Data that is subject to holds will stay in the source tenant until its retention period expires. That is, Office 365 keeps the
data if the owning tenant is funded. In the case of tenant mergers, it is likely that funding will disappear eventually for the
source tenants and they will close. At this point, any data kept due to retention policies will be removed from Office 365.

Exchange Mailbox Lifecycle

Items held in Exchange Online mailboxes follow a lifecycle from their creation by the user (calendar, posts, tasks, contacts,
and so on) or when delivered as new messages. The major points in the lifecycle of Exchange Online items are:

Mailboxes store and process incoming and outbound email. Mailboxes can be archive-enabled to enable long-term
retention of less-frequently accessed items that need to be kept for extended periods.
The Deleted Items folder is the equivalent of the Trash or Wastebasket folder found in mailboxes in other email
systems. The Deleted Items folder holds items that the user removes from other folders. Items stay in the Deleted
Items folder until the user empties the folder or they are removed automatically because of a retention policy.
Items removed from the Deleted Items folder move into Recoverable Items , a set of sub-folders used to preserve
deleted items. The Recoverable Items folder also holds system items, such as audit entries to record actions taken
within the mailbox. The Recoverable Items structure is not part of the normal IPM_SUBTREE folder tree exposed by
 clients. It is stored in the server copy of the mailbox and never synchronized to local clients. The Recoverable Items
folder structure is often referred to as the “dumpster”. This term goes back to its first implementation as a
mechanism to allow users to recover deleted items without the need for administrator intervention.
The Managed Folder Assistant (MFA), a background assistant that processes mailboxes regularly to remove items
no longer needed from the Recoverable Items structure, remove items from other folders, and move items to into
folders in archive mailboxes. MFA’s processing is controlled by the retention policies that are applied to mailboxes
and constrained by the different holds that might exist for those mailboxes. The MFA does not process mailboxes that
are smaller than 10 MB.
Administrators create Retention policies to define how the MFA automatically manages mailbox items on behalf of
users. A mailbox can be assigned a single retention policy, consisting of a set of tags that apply to default folders
(such as the Inbox and Sent Items folders), other folders and items as dictated by the user, and every other item in
the mailbox (including the archive if available) that is not under the control of another tag.
Administrators can apply different forms of holds (retention hold, legal or litigation hold, or in-place hold) to
mailboxes to control whether users can remove information. Holds are usually invoked as the result of a legal or other
authorized action that forces a company to keep data for some purpose. When a mailbox is on hold, MFA will not
permanently remove items from the mailbox and Exchange Online copies any item that the user attempts to remove
or update. Retention holds are explained in this chapter and legal and in-place holds in Chapter 20.

Collectively, these components interact with each other to form the lifecycle of Exchange Online mailbox items. Let’s
discuss what happens when items are deleted in a little more detail to gain a better understanding of how the different
parts of the lifecycle fit together.

Cleaning Mailboxes
Over time, users remove items from various mailbox folders. Some remove messages as soon as they are read, some keep
everything and leave items to accumulate. Exchange uses a two-stage removal process. First, the item is “soft-deleted”
and moves to the Deleted Items folder, which is a default mailbox folder that acts as a convenient collection point for any
item removed from another folder. Then, if the user empties the Deleted Items, the items are “hard deleted” and moved to
the Recoverable Items\Deletions folder. You can also assign a folder retention tag to the Deleted Items folder to govern
how long items stay in the folder.

A user can force a “hard delete” for an item by using the Shift+Delete key combination in either Outlook or OWA. This
command instructs Exchange to ignore the Deleted Items folder and move the item directly to the Recoverable
Items\Deletions folder. Exchange also moves items into the Deletions folder when a user moves an item from a mailbox
folder to a PST. The reason here is that the move is a combination of a delete from the mailbox folder and a create in the
PST.

If they need to restore a deleted item, users can restore items from either Deleted Items or Deletions. Deleted Items is just
another default folder that exists in every mailbox, so to recover from Deleted Items, the user finds the item and moves it
to another folder in the mailbox. Deletions is a hidden folder and the items stored there do not appear in normal client
interfaces and a user cannot search for items in Deletions. To restore items held in the Deletions folder, use the “Recover
Deleted Items” feature available in both Outlook and OWA (Figure 19-22 ).

Figure 19-22: Recovering deleted items with OWA

When a user soft- or hard-deletes an item, Exchange writes a pointer to the original folder into the item’s Last Active
Parent Folder Identifier property (the MAPI property is LastActiveParentEntryID or LAPFID). Knowing which folder an
item originally came from allows OWA to restore the item back into that folder using the Recover Deleted Items feature,
even if the folder is renamed or moved. Exchange Online has recorded LAPFID information since the end of 2016, but it is
possible that some older items do not have a value in this property. If this is the case, OWA uses a scheme called “folder
type origin” to figure out where to recover items. Calendar items go back to the Calendar folder, Task items into Tasks,
Contacts into Contacts, and mail and any other items go into the Inbox. It can be a little strange to find recovered items in
the Inbox, especially if they are not mail items. For instance, if you recover a deleted conversation item (one recorded from
a Skype for Business Online call), it shows up in the Inbox. The same happens if you recover a Word document or Excel
spreadsheet (many people store these files in their mailbox).

Outlook clients use a different mechanism to recover items than OWA does. First, Outlook does not use the LAPFID to
restore items. Second, Outlook recovers all items into the Deleted Items folder, the idea being that users can then decide
where to move the items to as a permanent location. Microsoft might upgrade Outlook to support LAPFID in the future.
Table 19-4 summarizes where items are recovered by the two clients.

Outlook OWA

Mail item (message) Deleted Items Original folder or Inbox

Contact item Deleted Items Original folder or Contacts

Calendar item Deleted Items Original folder or Calendar

Task item Deleted Items Original folder or Tasks

Any other non-mail item Deleted Items Original folder or Inbox

Table 19-4: The recovery destination for various Exchange item types

If you remove a complete folder, you can recover the individual items within the folder, but you cannot recover the
complete folder as a single entity.

Administrator Recovery from Recoverable Items

While items are in the Recoverable Items folder, the mailbox owner can use the Recover Deleted Items feature to move
selected items back into the mailbox. Sometimes, users need help to recover items. In the past, an administrator might
sign into the user mailbox to perform the recovery on behalf of the user. The problem with this approach is that it
compromises the privacy of the mailbox. For this reason, Exchange Online has two cmdlets to help administrators recover
user data without needing to use a client to open the mailbox.:

Get-RecoverableItems executes basic searches of the Deleted Items and Recoverable Items folders to find items
without the need to sign into the mailbox.

Restore-RecoverableItems finds and copies items from Deleted Items or Recoverable Items to their original folders.

The basic idea is that you use Get-RecoverableItems to construct a search to find the desired items and then use the
search as input to Restore-RecoverableItems when you are sure that it will process the correct items. Before trying to run
these cmdlets, make sure that the account you use to sign into PowerShell holds the Exchange “Mailbox Import Export”
RBAC role. To find out who has the role already, you can run the following command. In the example, the members of the
Organization Management role group have the role as does the Administrator account.

[PS] C:\> Get-ManagementRoleAssignment -Role "Mailbox Import Export" | Format-Table RoleAssigneeName

RoleAssigneeName

----------------

Organization Management

Administrator

An example of using Get-RecoverableItems to search a mailbox for items is shown below. The search looks for any item of
type Ipm.Note (messages) moved into Recoverable Items during a certain time span. This happens when the user or the
Managed Folder Assistant moved the item into its current folder. For instance, if a mailbox has a retention policy that
moves items from Deleted Items into Recoverable Items after 120 days, the Managed Folder Assistant might have
processed the items found by the search above at least four months ago. A user can bypass Deleted Items and send an
item direct to Recoverable Items by using the SHIFT+Delete key combination. In this case, the LastModifiedTime property
(used for date filters) is the date when the user executed SHIFT+Delete.

[PS] C:\> Get-RecoverableItems -Identity Kim.Akers -SourceFolder RecoverableItems -FilterStartTime "2/16/2018 10:00:00" -FilterEndTime
"2/20/2018 17:00:00" -FilterItemType Ipm.Note

If you want to search the Deleted Items folder instead of Recoverable Items, specify “DeletedItems” (these values are
language independent). You cannot create a search for both the Deleted Items and Recoverable Items folders, so if you
want to check the two folders, you need two separate searches. You also cannot search either folder in the archive
mailbox. Likewise, you can specify different types of items to look for (like Ipm.Appointment for a calendar item or
Ipm.Contact for a contact), but you cannot combine different item types in a search. Users might not be certain when an
item was deleted, but they might be able to tell you the message subject. If so, you can search like this:

[PS] C:\> Get-RecoverableItems -Identity Marc.Vigneau -SourceFolder DeletedItems -SubjectContains "Tasks”

Be aware that a search based on SubjectContains finds any item that contains the string in its subject. In this case, it will
unearth items with subjects like “My Tasks” and “Hard and Difficult Tasks” and “Tasks 2017.”

The example of the data returned for found items is shown below.

Identity :
FoTJp+dslCpMvzflTJoPw98AAk3A87FGAAAAAO+4Ga1BbPZCivtiNBUcCOEHAITJp+dslCpMvzflTJoPw98AAAAAARQAAITJp+dslCpMvzflTJoPw98AAk1/hRYAAAk=

MailboxIdentity : Marc.Vigneau

ItemClass : IPM.Note

Subject : You have been added to a team in Microsoft Teams

EntryID :
00000000EFB819AD416CF6428AFB6234151C08E1070084C9A7E76C942A4CBF37E54C9A0FC3DF000000000114000084C9A7E76C942A4CBF37E54C9A0FC3DF00024D7F85160000

SourceFolder : Recoverable Items\Deletions

LastParentFolderID : 84C9A7E76C942A4CBF37E54C9A0FC3DF00000000010C

LastModifiedTime : 02/16/2018 11:43:21

LastParentPath : Inbox

OriginalFolderExists : True

IsValid : True

ObjectState : New

Obviously, the more precise the search, the more likely you are to find the right item. Once you are happy that your search
finds the right items, you can proceed to recovery. The Restore-RecoverableItems cmdlet takes the same search that you
use to find items and restores each item to its original location:

[PS] C:\> Restore-RecoverableItems -Identity Marc.Vigneau -SubjectContains "Team" -SourceFolder RecoverableItems

Of course, if you wanted to, you could create your own processing loop to process a batch of mailboxes and restore the
messages if any matches are in Recoverable Items. Here’s an example:

[PS] C:\> $Mbx = (Get-Mailbox -RecipientTypeDetails UserMailbox -Filter {CustomAttribute1 -eq "IT"} | Select Alias, DisplayName)

Write-Host "Recovering items for" $Mbx.Count "mailboxes..."

ForEach ($M in $Mbx) {

Write-Host "Checking mailbox" $M.DisplayName

Restore-RecoverableItems -Identity $M.Alias -SourceFolder RecoverableItems -SubjectContains "Important and Critical Message" }

Deletions, Purges, and Versions

The Recoverable Items structure uses sub-folders to organize items that it needs to keep. Some items stay until their
deleted items retention period elapses. Others stay in the structure for a lot longer because one or more holds exist on the
mailbox.

Items reach Recoverable Items\Deletions folder when:

They are hard-deleted.

They are deleted from the Deleted Items folder.
The Deleted Items folder is emptied.

Items stay in the Deletions folder for the period set in the deleted items retention period for the mailbox. In Exchange
Online, the default period is usually 14 days. You can increase the deleted items retention period to a maximum of 30 days
by running the Set-Mailbox cmdlet (and while you will probably use whole days, you can specify a retention time down to
the second). For example:

[PS] C:\> Set-Mailbox –Identity 'Sanjay Patel' –RetainDeletedItemsFor 29.23.57.03

An exception exists for calendar items, which Exchange keeps for 120 days.

Recoverable Items Only Online . Because the Recoverable Items folder is only present on the server, you can only use
Recover Deleted Items when Outlook can make a network connection to Exchange Online. In addition, if you ever need to
use the MFCMAPI utility to see the items in the various sub-folders under Recoverable Items, you must configure
MFCMAPI to open the message store online. Do this by going to the Tools menu, select Options , then set the checkbox
"Use the MDB_ONLINE flag when calling OpenMsgStore ".

When the deleted item retention period (including the SIR – see below) expires for an item, MFA removes it from the
database and the user can no longer recover the item. The exception to the rule is when a mailbox is on hold as Exchange
will then move items into the Recoverable Items\Purges folder and keep the items there for as long as the hold applies.
During this period, administrators can recover items with a content search.

A user can try to remove an item permanently by using the Recover Deleted Items feature to select the item and then
select Purge (Figure 19-22 ). What happens next depends on the SingleItemRecoveryEnabled setting for the mailbox.
Exchange uses the Single Item Recovery (SIR) feature to ensure that a deleted item can be recovered for the longest
possible time set by the deleted item retention period.

If SingleItemRecoveryEnabled is False and the mailbox is not subject to a hold, Exchange removes the item from the
database at once and it is irrecoverable.
If SingleItemRecoveryEnabled is True (the default for Office 365) and the mailbox is not subject to a hold, MFA moves
the item to the Recoverable Items\Purges folder. The item stays there until its deleted item retention period expires.
At that point, MFA removes the item from the database and the item is irrecoverable. However, as discussed below,
while this description is true, a retention policy might cause something different to happen.
If SingleItemRecoveryEnabled is True and the mailbox is subject to a legal hold, Exchange keeps the item in the
Recoverable Items\Purges folder while the hold applies.
If SingleItemRecoveryEnabled is True and the mailbox is subject to an in-place hold, part of MFA known as the Email
Lifecycle Assistant (ELC) determines whether a copy needs to be retained to satisfy the hold criteria (query and date
range). If this is the case, MFA moves the item to the Recoverable Items\DiscoveryHolds folder and kept there until
the hold lapses. ELC examines items that have exceeded the deleted items retention period in mailboxes at least once
a day.

See Chapter 20 for more information on how to apply in-place holds to mailboxes.

Nothing moves to DiscoveryHolds : If you review items in the Recoverable Items structure, you might see that nothing
ever moves into the DiscoveryHolds folder in the primary mailbox, even if those items are subject to an in-place hold. This
can happen when the mailbox is under the control of the Default MRM Policy and is archive-enabled. Here is why.

When the Deleted Items folder is emptied, or items are hard-deleted by users, they end up in the Deletions sub-folder.
Items in any folder under Recoverable Items are subject to the folder tag contained in the MRM policy, which instructs
the Managed Folder Assistant to move the items to the archive after 14 days. If the mailbox is archive-enabled, the items
are moved. If not, they stay in the primary mailbox.

ELC processes items when they reach the deleted items retention period. By default, this is 30 days. Items subject to a
hold will be moved to the DiscoveryHolds folder at this point. However, no items reach the 30-day deleted item retention
period in the primary mailbox because they have already been moved to the archive. ELC processes the archive and will
move the held items to the DiscoveryHolds folder in the archive, but no trace is ever seen of an item in the
DiscoveryHolds folder in the primary mailbox. This can be confusing at first, but it is quite logical when you consider the
age limits that control where items stay for different periods in a mailbox.

No client can permanently remove items under hold, including low-level utilities such as MFCMAPI. Exchange integrates
checks for holds into how it manages data in its databases and no one can circumvent the effect of a hold. Preventing the
unauthorized removal of data from mailboxes allows Exchange to preserve items in an immutable fashion when needed by
an organization.

Exchange monitors mailbox items that are subject to an in-place hold to detect if the user or any other process changes
the item. If this happens, a copy of the original item is moved into the Recoverable Items\Versions folder to ensure that
everything that happens to an item is fully recorded. This action is called a “copy on write”. When the hold elapses, MFA
removes the items from the Versions folder along with items held in the DiscoveryHolds and Purges folders.

The SearchDiscoveryHoldsFolder folder : This is a sub-folder of the DiscoveryHolds folder that is used by ELC when it
processes the DiscoveryHolds folder to decide if any items can be removed. If such an item is found, it is moved to
SearchDiscoveryHoldsFolder and kept there until it is eventually removed by the Managed Folder Assistant.
Recoverable Items Quota
The Recoverable Items structure has a separate 100 GB storage quota to that given to the primary mailbox. If a mailbox is
archive-enabled, a separate quota is available to the archive mailbox. Between the two mailboxes, Exchange can hold up to
200 GB of recoverable data for a user. Items stored in the Recoverable Items folder in the archive behave in the same
manner as if they were in the primary mailbox and are discoverable by searches.

Some mailboxes created prior to 2014 might still have older 30 GB recoverable items quota or you might want to increase
the quota in circumstances such as when you have a heavily trafficked mailbox that is on hold and many items the user
removes daily. You cannot change the RecoverableItemsQuota for a mailbox with the EAC or PowerShell, but you can log a
support case with Office 365 Support if a mailbox is likely to exhaust its recoverable items quota. To find out how much
data exists in Recoverable Items for a mailbox, use either of the Get-MailboxStatistics or Get-MailboxFolderStatistics
cmdlets:

[PS] C:\> Get-MailboxStatistics –Identity TRedmond | Select TotalDeletedItemSize

TotalDeletedItemSize

--------------------

641.8 MB (672,946,488 bytes)

Or, to be a little more precise:

[PS] C:\> Get-MailboxFolderStatistics –Identity TRedmond –FolderScope RecoverableItems | Format-Table Name, FolderSize, ItemsInFolder,
FolderAndSubFolderSize -AutoSize

Name FolderSize ItemsInFolder FolderAndSubfolderSize

---- ---------- ------------- ----------------------

Recoverable Items 0 B (0 bytes) 0 563.3 MB (590,650,854 bytes)

Audits 523.1 MB (548,504,424 bytes) 149324 523.1 MB (548,504,424 bytes)

Calendar Logging 18.37 MB (19,261,189 bytes) 734 18.37 MB (19,261,189 bytes)

Deletions 10.72 MB (11,236,121 bytes) 115 10.72 MB (11,236,121 bytes)

DiscoveryHolds 0 B (0 bytes) 0 0.B (0 bytes)

SearchDiscoveryHoldsFolder 0 B (0 bytes) 0 0 B (0 bytes)

Purges 5.707 MB (5,984,254 bytes) 753 5.707 MB (5,984,254 bytes)

Versions 5.402 MB (5,664,866 bytes) 53 5.402 MB (5,664,866 bytes)

Add the Archive switch to the cmdlet parameters if you want to see data for the archive mailbox.

If the mailbox exceeds the Recoverable Items quota:

The user cannot remove items from their mailbox.

The Managed Folder Assistant cannot move (soft-delete) items into the Recoverable Items.
Write-on-protect cannot take copies if users alter items subject to a litigation or in-place hold.
Exchange cannot save audit items if mailbox auditing applies to the mailbox.

You can take two actions to restore the Recoverable Items folder to normal working order. First, you can ask Microsoft to
increase the recoverable item quota for the problem mailboxes. This might take a day or so to be effective and during that
time the mailbox might experience problems such those listed above. The second solution is to run the Managed Folder
Assistant and instruct it to clean up duplicate items that might be present in Recoverable Items. This step should have an
immediate effect and restore the mailbox to good health. To run the Managed Folder Assistant in clean-up mode, use a
command like this:

[PS] C:\> Start-ManagedFolderAssistant –Identity TRedmond -HoldCleanup

Check the reported data for the Recoverable Items folder after the Managed Folder Assistant completes and hopefully you
will discover that the overall size goes down. For more information about how to track what the Managed Folder Assistant
does, see the section “Logging the Managed Folder Assistant” later.

Exchange Mailbox Retention Policies

As we know, Office 365 retention policies allow administrators to control how long many types of data is kept within Office
365 or how quickly data is removed. Exchange mailbox retention policies are supported by both Exchange Online and
Exchange on-premises servers to instruct the Managed Folder Assistant (MFA) as to how long users can keep mailbox
content (the retention period). The policy also tells MFA what to do once the retention period lapses, with the choice being
to either remove the item (recoverable or permanent) or move it to the archive mailbox. A mailbox can only be assigned
one retention policy at a time, but many retention policies can exist within a tenant, so that the needs of different user
groups are met.

A retention policy is composed of a set of retention tags. Exchange Online supports three distinct types of retention tags:

Retention policy tags (sometimes called “folder tags”) the default folders found in every mailbox use these tags.
You only assign retention policy tags to the default folders that you want MFA to process. The set of default folders
are:

Inbox, Sent Items, Deleted Items, Calendar, Contacts, Tasks, RSS Feeds, Recoverable Items, Clutter, Archive,
Conversation History, Journal, Junk E-Mail, Notes, Outbox, Sync Issues

Microsoft does not support the creation of retention policy tags for non-default folders. It
does not make sense to create a folder tag for the Outbox because messages only stay here
while they wait to be transmitted by the server.

Personal tags allow users to exercise control over specific items or non-default folders. A policy can have many
personal tags, but it is good practice to avoid user confusion by limiting the tags to between 7 and 10 in total.
Default tags tell MFA how to process any item that is not covered by a retention policy tag or a personal tag. A
policy can have up to three default tags, each of which serves a different purpose:

The delete tag: Instructs MFA to move items into the Recoverable Items folder or to permanently remove them from
the mailbox.
The archive tag: Instructs MFA to move items into the archive mailbox. This tag is only effective if the mailbox is
archive-enabled.
The voicemail tag: Gives specific instruction as how to handle voicemail messages. The reason for this tag is that
many companies have different retention requirements for voicemail than they do for regular messages.

When label policies publish labels for use with Exchange Online mailboxes, the labels show up in clients and behave much
like personal tags.

The meaning of retention : People who approach Exchange retention policies for the first time can be forgiven for
misunderstanding what “retention” means in this context. Retention means “keep what you need”. There are two ways of
achieving the goal. You can either make sure that you preserve what is needed or you can remove information that is not
needed. Tenants often use Exchange retention policies to remove information after the items reach a certain age. You can
also use retention policies to preserve items by moving them from primary mailboxes to archive mailboxes where the
items can be kept for a further period. Policies can combine delete actions and archive actions to achieve the retention
goals of the organization. Unlike Office 365 retention policies, Exchange retention policies do not place holds on content.

Managing Hybrid Mailbox Retention Policies

The Hybrid Configuration Wizard (HCW) can transfer retention policies and tags from an on-premises Exchange
organization to Exchange Online. The transfer process does not overwrite policies and tags that already exist within
Exchange Online. The transfer is on a one-time basis and the HCW does not synchronize the policies and tags created
within Exchange Online thereafter. The two platforms run separate and distinct retention regimes, so manual intervention
is needed to keep the two sides aligned if you make changes afterwards on either platform.

A major difference between the two platforms is that Exchange Online assigns the “Default MRM Policy” (more on this
soon) to all newly-created mailboxes, including those transferred from on-premises servers, while Exchange on-premises
leaves it to the administrator to decide whether to assign retention policies to new mailboxes.

Benefits of a Mailbox Retention Policy

The need for user coaching and training to communicate the value of a good retention policy and the benefits it brings to
users is obvious. The benefits include:

Automatic clean-up of mailbox contents : An argument can be advanced that the automatic removal of
information from a mailbox is bad but given the volume of email that users must cope with nowadays, having old
items removed from a mailbox after a reasonable period seems like goodness. Given the very large mailboxes that are
now in use, users can argue that it is a waste of time to go through their mailbox to clean it up on a regular basis.
Automatic clean-up is an example of how intelligent technology helps to save time for users.
Ability to control retention : Users can control how items are kept in mailboxes by applying personal tags to
folders and individual items. Including personal tags that prevent items ever being removed or archived from folders
is popular with users, even if they never actually use the tags.
Ability to achieve the desired effect of corporate retention policies : Retention policies can help users to keep
information that is necessary to follow regulatory or legal guidelines. On the other hand, retention policies can also
be employed to remove information from mailboxes that would otherwise be embarrassing or unhelpful under the
harsh spotlight of legal discovery. Personal tags allow users to clearly mark items needed for audit or other regulatory
purposes and default tags can be deployed to remove non-essential items that the company considers should not be
retained.

Of course, operational needs differ enormously from company to company and a general-purpose retention framework
such as that in Exchange Online can only deliver value if work is done to understand the business requirements and then
adapt the available functionality in the most effective way to meet those requirements. Gaps might be found, but it is
always surprising quite how much can be done with off-the-shelf software.
The Default Retention Policy
To make it easier for administrators to implement a messaging records management strategy and keep mailboxes under
control, Exchange Online includes a default retention policy called the “Default MRM Policy.” This is the same policy used
with Exchange 2013 and Exchange 2016. However, while in an on-premises deployment, an administrator must take an
action to apply an MRM policy to mailboxes, Exchange Online automatically applies the Default MRM Policy to every
mailbox when it is created, including the mailboxes that are moved to Exchange Online from an on-premises server.

The set of tags included in the Exchange Online version of the Default MRM Policy are a good example of the diversity and
use of tags usually found in a well-designed retention policy. Table 19-5 lists the retention tags that are included in the
Default MRM Policy.

Tag name Type Meaning

1 Month Delete Personal Items move to Recoverable Items after 30 days

6 Month Delete Personal Items move to Recoverable Items after 180 days

1 Week Delete Personal Items move to Recoverable Items after 7 days

1 Year Delete Personal Items move to Recoverable Items after 365 days

5 Year Delete Personal Items move to Recoverable Items after 1825 days

Never Delete Personal Prevents MFA deleting items. Note: MFA can still archive items if dictated by an
archive tag

Personal 1 year move to Personal Moves an item into the archive after 365 days
archive

Personal 5 year move to Personal Moves an item into the archive after 1825 days
archive

Personal never move to Personal Prevents MFA moving items into the archive mailbox
archive

Default 2 year move to Default Items not under the control of another tag move to the archive (if available) after 730
archive days

(*) Deleted Items Folder Items in the Deleted Items folder move to Recoverable Items after 30 days. Microsoft
stopped processing this retention tag for the Default MRM Policy in early 2015

Junk Email Folder Items in the Junk Mail folder move to Recoverable Items after 30 days

Recoverable Items 14 Folder Moves items in the Recoverable Items (plus sub-folders) to the archive after 14 days
days move to archive

Table 19-5: Retention tags in the Default MRM Policy

The design of the default retention policy is quite interesting. A total of thirteen tags exist, but only three of the default
folders are covered (Deleted Items, Junk Email, and Recoverable Items). Many retention policies implemented by
companies focus on removing old content from the Inbox and Sent Items folders and include retention tags for this
purpose, possibly moving items after 60 days or so. No default delete tag is included in the policy, so MFA leaves items in
their folders unless they are explicitly tagged for deletion.

Although no default delete tag is present, the Default MRM policy does contain a default archive tag. As we know, a
default tag tells MFA how to process any item that is not stamped with a more specific tag. In this case, Items that do not
inherit a tag from their current folder or have not been assigned a tag by the user are moved to a folder of the same name
in the archive mailbox after 2 years, or 730 days. If the mailbox is not archive-enabled, MFA ignores the default archive
tag.

Because newly-migrated mailboxes are stamped with the Default MRM Policy, the potential exists that the retention tags
that govern how the MFA processes mailbox items will change when the mailbox becomes active in Exchange Online.
Consider the situation where a company runs Exchange 2016 on-premises servers and changes the Default MRM Policy to
limit the period that items can be kept in common folders such as the Inbox and Sent Items. Everything works in a
satisfactory manner for on-premises mailboxes. A decision is then made to introduce Office 365 and a hybrid connection is
configured to enable interoperability between the on-premises and cloud components. When mailboxes move to the cloud,
Exchange Online assigns the Default MRM Policy to them. No tags are present in this policy to cover the Inbox and Sent
Items folder with the result that a different retention model runs for the newly-moved mailboxes. The potential exists for
no one to notice the issue for months or years afterwards, which might or might not create a compliance issue.

For this reason, experienced consultants often recommend that customers do not use the Default MRM Policy and create
and apply a new specially-tailored retention policy to all mailboxes. The new policy can be imported into Exchange Online
and stamped onto mailboxes as they are moved to the cloud to ensure that the same retention regime is applied on both
sides of the on-premises/cloud divide. Some added scripting is necessary to ensure that the correct policy is assigned to
mailboxes, but this is not difficult to do. We will get to discussing how to assign retention policies to mailboxes later.

Figure 19-23 shows a set of retention policies as viewed through the compliance management section of the EAC. A
custom retention policy called "Management retention policy" is highlighted and the tags in the policy are shown in the
details pane. This retention policy has a mixture of default retention tags available in every Office 365 tenant and some
custom tags created to meet specific business needs.

Figure 19-23: Details of the retention tags contained in a mailbox retention policy

Discovering How Many Items Move into The Archive

Knowing that a default archive tag is included in the Default MRM Policy, it might strike you that a lot of items already
have been moved into archives for mailboxes in your tenant. The easiest way to find out is to run the Get-
MailboxFolderStatistics cmdlet to check out the archiving activity for a mailbox. The code below retrieves information for
the copies of the Inbox and Sent Items folder in the archive. Apart from noting the number of items in each folder, we also
report the creation date of the newest and oldest items were added to the folder, which should give some insight into how
retention works. For instance, the newest items added to the archive are dated from early June 2016. Given a “2-year
default move to archive” tag in the policy, this is what you would expect to see if you looked in June 2018. The creation
date of the oldest items in the folders depend on the default delete tag applied to the mailbox. In this case, because the
oldest item dates from March 2011, we know that the tag allows items to be kept for at least seven years.

[PS] C:\> Get-MailboxFolderStatistics –Identity "Kim Akers" –Archive -IncludeOldestAndNewestItems | Where-Object {$_.Name –Like "Inbox" –
or $_.Name –Like "Sent Items"} | Format-Table Name, ItemsinFolder, FolderSize, NewestItemReceivedDate, OldestItemReceivedDate –AutoSize


Name ItemsInFolder FolderSize NewestItemReceivedDate OldestItemReceivedDate

---- ------------- ---------- ---------------------- ----------------------

Inbox 15642 1.845 GB (1,981,302,727 bytes) 05/06/2016 23:38:01 25/04/2012 00:40:18

Sent Items 12946 1.685 GB (1,809,115,575 bytes) 05/06/2016 09:11:00 12/04/2011 16:52:41

You can also use the Get-MailboxStatistics cmdlet with the Archive switch to list the properties of an archive mailbox. For
example:

[PS] C:\> Get-MailboxStatistics -Identity "Kim Akers" -Archive | Format-List

Changing the Default MRM Policy for a Tenant

The Default MRM policy is stamped onto every new user, shared, room, or resource mailbox when it is created. The policy
is not applied to the mailboxes used by Office 365 Groups. Exchange Online finds the correct policy to use by consulting
the mailbox plan that is assigned to the mailbox. The configuration settings for each tenant lists the mailbox plans that are
available and each plan has an associated mailbox policy. For instance, the tenant in this example has three plans and each
plan has the Default MRM Policy as the default retention policy.

[PS] C:\> Get-MailboxPlan | Format-Table Name, RetentionPolicy

Name RetentionPolicy

---- ---------------

ExchangeOnline-12c139bc-eafa-4a43-b4d2-e285f83e075d Default MRM Policy

ExchangeOnlineDeskless-bc1e76cc-4c0b-491c-a518-3a0a43cbf78e Default MRM Policy

ExchangeOnlineEnterprise-8fc1c029-5e32-485e-9810-179fb4701447 Default MRM Policy

In an on-premises Exchange organization you can change the default retention policy by setting the IsDefault flag to $True
:

[PS] C:\> Set-RetentionPolicy –Identity 'New Retention Policy' –IsDefault $True

However, this approach does not work in Exchange Online. The command to reset the IsDefault flag completes but
Exchange Online continues to use whatever retention policy is assigned in the mailbox plan. If you want to ensure that
new mailboxes get a different retention policy, you can update the retention policy defined in the mailbox plans used
within the tenant. For example, to assign a different retention policy to the plan used for users holding Office 365 E3 and
E5 licenses, we run this command:

[PS] C:\> Set-MailboxPlan -Identity ExchangeOnlineEnterprise-8fc1c029-5e32-485e-9810-179fb4701447 -RetentionPolicy "Management Retention

Policy"

You can then check that the correct retention policy is assigned to the plan by running the Get-MailboxPlan cmdlet. The
final check is to create a new user account and assign them an E3 or E5 license. When Exchange Online creates the user’s
mailbox, it should assign the “Management Retention Policy”. We check this with:

[PS] C:\> Get-Mailbox -Identity NewUser | Select RetentionPolicy

To make sure that the intended retention policy stays assigned to mailboxes, it is sensible to check from time to time. For
example, this PowerShell code looks for user mailboxes that do not have the right policy assigned (or no policy assigned)
and then assigns the correct policy:

[PS] C:\> Get-Mailbox –RecipientTypeDetails UserMailbox -Filter {RetentionPolicy –ne "Preferred MRM Policy" –or RetentionPolicy –eq $Null}
| Set-Mailbox –RetentionPolicy "Preferred MRM Policy"

Retention Actions and Retention Periods

The two most important settings in a retention tag are the retention period and the retention action. The period is the
length of time in days that must elapse before MFA will enforce the defined action. Figure 19-24 shows the properties of a
retention tag created to control how long items stay in the Inbox. These properties are:

Tag name . Folder and default tags are never shown to users, so their names do not matter as much as the personal
tags that appear in client user interfaces (see discussion later). However, in all cases, the tag name should show its
purpose and in the case of personal tags, they should tell users the length of the retention period. “1 Week Delete” is
a reasonable example of a personal tag name that combines both pieces of information. Some companies prefer the
simplicity of names like “Required for Audit” to communicate the use of a personal tag without bothering users with
details of what this means in terms of time.
Retention action . This is the instruction given to MFA to execute when the retention period expires. Exchange
retention policies support three actions:

Delete and Allow Recovery : MFA moves the item into the Recoverable Items\Deletions folder.
 Permanently Delete : MFA removes the item from the database unless it comes under the control of a hold. This
action can be dangerous unless you are certain that you want to remove items. Given the storage quota available to
mailboxes, it is safer to use the Delete and Allow Recovery action as users can then recover items if necessary.
Move to Archive : MFA moves the item into a folder of the same name in the archive. This action only applies when an
archive mailbox is available. MFA ignores the tag if an archive otherwise. The Move to Archive action is only available
for default tags and personal tags.

The retention period . This sets the age of an item before MFA will process the retention action in the tag. The
period can be “Never” or between 1 and 24,855 days. Microsoft has published an interesting description explaining
how MFA calculates retention dates for items. If a tag uses Never as a retention period, it means that MFA will never
execute the retention action because the item will never expire. Tags that have a retention period of Never also have
their RetentionEnabled property set to False.
Comment . Free-form text used to explain the purpose of a retention tag when administrators work with the tags
through EAC. The comment is also seen by users when they use OWA options to add personal tags to the set of tags
made available to them through policy.

Folder tags are always associated with one of the default folders while default tags apply to items across the entire
mailbox. You must define the tag type when you create a new retention tag and you cannot change a tag to another type
thereafter. If you make a mistake, you will have to remove the bad tag and recreate it as the right type.

Figure 19-24: Properties of a retention tag

You can also create retention tags with PowerShell. To begin, here is an example of a personal retention tag that applies to
all types of items. The effect of the tag is to move items to the archive after 180 days. Remember that the MoveToArchive
retention action can only be specified for default and personal tags and is not available for folder tags that apply to one of
the default folders. Items that are processed by this retention action are moved to a folder of the same name in the archive
mailbox.

[PS] C:\> New-RetentionPolicyTag –Name "Project Contoso Documents" –AgeLimitForRetention 180

–Comment "Archive documents required for Project Contoso" –MessageClass * -RetentionEnabled $True

–Type Personal –RetentionAction MoveToArchive

The first retention tag created in the next example is for the Inbox folder and removes items that are more than 30 days
old. The items can be recovered if necessary. The second is to remove items from the Deleted Items folder after they are
120 days old. Again, the mailbox owner can recover items if necessary.

[PS] C:\> New-RetentionPolicyTag –Name "Inbox 30 day expiry" –AgeLimitForRetention 30

–Comment "Move items out of Inbox after 30 days" –RetentionEnabled $True

–RetentionAction DeleteAndAllowRecovery –Type Inbox



[PS] C:\> New-RetentionPolicyTag –Name "Deleted Items 120" –AgeLimitForRetention 120

–Comment "Move items out of the Deleted Items folder after 120 days" –RetentionEnabled $True

–RetentionAction DeleteAndAllowRecovery –Type DeletedItems

Finally, here is how to create two types of default retention tag. The first removes any item in a mailbox that is more than
seven years old. The second specifically processes voicemail messages (as defined in the message class) and will remove
these items after 7 days.

[PS] C:\> New-RetentionPolicyTag –Name "Delete Items after 7 years" –AgeLimitForRetention 2555

–Comment "Clean up mailboxes by removing items older than 7 years" –RetentionEnabled $True

–RetentionAction PermanentlyDelete –Type All

[PS] C:\> New-RetentionPolicyTag –Name "Voicemail Removal" –AgeLimitForRetention 7

–Comment "Clean up voicemail after 7 days" –RetentionEnabled $True

–RetentionAction PermanentlyDelete –Type All –MessageClass Voicemail

Although it is always best to come up with a logical name that makes sense to end users, you can give a retention tag any
name you like and can set translated values for display through different language interfaces. This must be done through
PowerShell as the EAC does not allow you to manipulate language settings. Unfortunately, OWA does not support language
translations, but Outlook does. In this example, we configure French, Dutch, and Spanish values for the “Keep for Audit”
tag:

[PS] C:\> Set-RetentionPolicyTag –Identity "Keep for Audit"

–LocalizedRetentionPolicyTagName "fr-FR: Conserver pour verification", "nl-NL: Houden voor controle", "es-ES: Mantenga la auditoria"

Remember that a retention tag is not unique to a policy. Multiple policies can use the same retention tag, so it is best to
create tags with the intention in mind that they can be used in many policies. Users can also add personal retention tags
to the set available to their mailbox through OWA Options.

The Recoverable Items Folder Tag

The Recoverable Items folder tag is a special case. EAC does not display this tag when listing available retention tags in
EAC or in client user interfaces, but you can see details of the tag by running the Get-RetentionPolicyTag cmdlet. Another
special case is that only the “move to archive” action is available for the tag. This is logical because it does not make much
sense to use the “delete and allow recovery” action as the items already exist in the Recoverable Items structure. A case
that the “permanently delete” action should be supported fails because Exchange Online uses a different method (the
deleted items retention period) to remove deleted items from mailboxes. The effect of the Recoverable Items folder tag in
the Default MRM policy is:

If a mailbox has an archive, MFA moves deleted items into the archive after 14 days. The items are stored in
corresponding folders in the Recoverable Items structure in the archive mailbox. The items remain in the archive
indefinitely because the retention policy does not contain a default delete tag to instruct that items should be
permanently removed after a set period. In addition, the normal deleted items retention period does not apply to
archive mailboxes.
If the mailbox is not archive-enabled, MFA ignores the instruction contained in the tag.

Why Items Stay in The Deleted Items Folder

The Default MRM Policy in on-premises Exchange versions has a retention policy tag to instruct the Managed Folder
Assistant to remove items from the Deleted Items folder after they are 30 days old. This also used to be the case for
Exchange Online. Now the MFA for Exchange Online ignores that tag when a mailbox is stamped with the Default MRM
Policy. The tag is processed if it is included in any other retention policy, including if you rename the standard policy.

Microsoft does not use database backups inside Office 365. Instead, the protection delivered by multiple database copies
spread across multiple datacenters and other native data protection features ensure that information is protected for as
long as it is needed. Evidence from support calls proved that many users removed items in error and later found that they
could no longer recover those items because they had been expunged from the database. At this point the items are
irrecoverable because backups are not available. It is unreasonable for users to understand the complexities involved in a
backup-free regime and this led to many calls to Microsoft Support and many instances of unhappy customers who could
not get their data back. This became a prime factor in Microsoft’s decision to stop removing items from the Deleted Items
folder.

Other reasons include:

Users often triage Inbox items by deleting them. In other words, if a message is unimportant or does not have to be
dealt with at once, removing the item from the Inbox allows the user to concentrate on items that need action. Later,
the user can go to Deleted Items and review items there to see if anything needs attention. This is a primitive form of
 the Focused Inbox feature that exists in almost all mail systems. Problems occur when items disappear from the
Deleted Items and cannot be recovered.
Many small companies use Office 365. Small companies tend to be less sophisticated in terms of IT policies than large
enterprises and have fewer IT resources to create and manage policies. However, a support call from a small
company costs the same energy, time, and expense to process and resolve.
Although Exchange Online includes a wide variety of compliance tools, it takes time and knowledge to master these
tools. Given the choice of activities, an administrator would probably take care of day to day tasks rather than
plunging into the details of how to build and deploy a well-crafted retention policy.
The added storage needed to keep items is not a concern because Office 365 mailboxes are assigned large quotas.

Compliance features are designed to support the regulatory and legal requirements of large enterprises. In this instance,
Microsoft seems to have done the right thing by building retention policies into Exchange Online and assigning a default
policy to user mailboxes. The only retention tags in the default policy that could be construed to interfere with user
management of personal data are the folder tag that remove items from the Deleted items folder and the default archive
tag that moves items from the primary mailbox to the archive (if enabled) after two years. Looking at the arguments
advanced above, you can see how a case can be made to stop cleaning out the Deleted Items folder.

By default, users can accumulate deleted items for as long as they like. If you decide that this is not sensible and believe
that the Deleted Items folder needs to be managed, you can:

Rename the Default MRM Policy to use any other name. MFA will recognize the new name and understand that it
should process the Deleted Items tag as before.
Create a new retention policy that has the standard Deleted Items tag and apply that policy to all mailboxes. You
might want to increase the retention period (to between 60 and 120 days) to allow users more time to recover items.
If someone has not worked out that they have lost something important after an extended period, it is possible that
the information might not be so important after all.
Include a default delete tag in the Default MRM Policy that removes items in all folders after a certain period. This
approach is not as precise as applying a folder tag, but the Deleted Items folder will come under the control of the
default delete tag if no other more specific tag is in place.

If users empty the Deleted Items folder, normal processing will continue, and items will stay in Recoverable Items until the
deleted items retention period expires. Given human nature, you can expect that users will not empty the folder (unless
someone tells them how to do it) and so will be able to recover deleted items for as long as they need.

The change in policy means that items will continue to accumulate in the Deleted Items folder over time. Apart from
increasing the size of the OST synchronized to workstations, this should not be a concern because a single folder in an
Exchange Online mailbox can hold up to one million items and adequate quota is available to mailboxes to keep the data
for many years. Most users will accumulate two or three gigabytes of deleted information annually, so this should not be a
problem in an era of 100 GB mailboxes. And if an archive mailbox is available, the default archive tag in the policy will
mean that items are moved to the Deleted Items folder in the archive after two years.

Some ramifications from keeping Deleted Items around : Although the notion that you can keep a deleted item for
as long as you want might seem like a promising idea, many large companies consider this to be perfectly horrible
because of the impact on their compliance strategy. Items that stay in mailboxes stay discoverable and items that are
discoverable can be found by investigators. Companies do not want to break any regulations but equally they do not want
to keep anything around that might be embarrassing or useful in eDiscovery situations. If you are in this situation, you
need to create a new retention tag to govern Deleted Items and include that tag in the retention policies that are applied
to user mailboxes. The steps needed to create a new retention tag for the Deleted Items folder are explained in the next
section.

A further issue is created for those running hybrid environments as MFA behaves differently on-premises to the way that
it processes online items. For example, if you move a mailbox from Exchange Online back to an on-premises server, MFA
will begin to process items in the Deleted Items folder according to whatever on-premises retention policy is applied to
the mailbox. If the on-premises retention policy applies a retention action to these items, many items might be removed
from the mailbox. The other issue when moving mailboxes around between cloud and on-premises environments is that
the dramatically increased size of the Deleted Items folder means that moving a mailbox from Exchange Online will take
much longer than before. It is always best to ensure that information is treated the same on both platforms, so you should
make sure that the same retention policies that apply the same retention actions for the same periods are used for all
mailboxes.

Creating a Mailbox Retention Policy

A retention policy is created by linking together a set of retention tags. Any retention tag defined within the tenant can be
added to a new retention policy, subject to the following constraints:

Only a single folder tag can be added for each default folder. For example, a policy cannot have two folder tags to
instruct MFA how to process the Inbox.
Only a single default tag can be defined for each of the supported conditions:

Delete (either permanently or delete and allow recovery).

Move to Archive.
Voicemail. The default tag to cover voicemail items must be defined with a message type of “voicemail.”
 Some personal tags to allow users to preserve information according to their business needs. Due to the limited space
available to display tags in client user interfaces, you should restrict this set to between 7 and 10 tags.

If you assign a tag to a folder, the tag is inherited by any sub-folders that might exist or will be created in the future and
the tag will apply to all items in those folders. You can override the influence of inheritance by assigning a personal tag to
an item or folder.

A cause of possible conflict : A retention policy can have default tags for both deletion and archival. Tags applied to
items in the primary mailbox stay active when items move into the archive. When both a delete tag and an archive tag are
stamped on items, care must be taken that the retention periods cause actions to be taken in the correct sequence. For
example, if the delete tag specifies that an item is to be permanently removed after 365 days and the archive tags causes
items to be moved into the archive after 730 days, logic dictates that nothing will end up in the archive because items will
be permanently removed long before they can be old enough to be archived. Logic works if the retention periods for the
two tags are reversed because in that case, MFA will move items to the archive after 365 days and then remove them
after a further year.

It is sensible to write down the business justification to create a new retention policy and to chart out the set of tags to be
included in the policy before you create anything. You should also be able to define the set of mailboxes to which the policy
will be applied. If you cannot make a cogent case to create a new retention policy, some doubt exists whether the new
policy is needed. A profusion of policies can create confusion and uncertainty in those who are asked to administer a
tenant after you have happily moved to your next job.

To create a new retention policy, go to the compliance management section of the EAC, select retention policies, and click
[+] . Figure 19-25 shows the screen used to collect the information needed to create a new policy. As you can see, all we
have is the name of the new policy (invisible to users), and the set of tags that form the policy.

Figure 19-25: Creating a new retention policy with EAC

You can also create a new retention policy with PowerShell. The command is very simple because all it must do is to
associate the policy with a set of links to the tags.

[PS] C:\> New-RetentionPolicy –Name "Business Analysts"

–RetentionPolicyTagLinks "Keep for Audit", "6 Month Delete", "1 Year Delete", "Inbox 30 day expiry", "Never Delete"

If you forget to include a tag or need to remove a tag from a policy, you can add or remove it with the Set-RetentionPolicy
cmdlet. When reviewing the example shown below, you might ask why the complete set of tags is passed to the cmdlet
instead of trying to add or remove a single item from the list. The answer is that experience shows that this is the safest
way to update a set of tags in a policy. Too many instances have occurred where administrators tried to update a policy
and ended up with just a single tag, which then causes MFA to do a lot of work to remove tags from mailbox items. If you
do not like this approach, use the EAC instead. (In fact, if you look at the command log, you will see that the EAC always
writes out the complete set of tags when it updates a retention policy).

[PS] C:\> Set-RetentionPolicy –Identity "Business Analysts"

–RetentionPolicyTagLinks "Keep for Audit", "6 Month Delete", "1 Year Delete", "Inbox 30 day expiry", "Never Delete", "Junk Email"

The Remove-RetentionPolicy cmdlet can be used to remove a retention policy from a tenant. To ensure that automatic
maintenance continues smoothly for mailboxes, make sure that an alternate policy is assigned to mailboxes before you
remove a policy.

[PS] C:\> Remove-RetentionPolicy –Identity "A bad retention policy"

After you remove the policy, you might use the Get-RetentionPolicy cmdlet to view the set of retention policies that are
defined in the tenant. The RetentionPolicyTagLinks property shows the set of tags that are linked or associated with the
policy and link the policy with its tags.

[PS] C:\> Get-RetentionPolicy | Format-Table Name, RetentionPolicyTagLinks -AutoSize

Name RetentionPolicyTagLinks

---- -----------------------

ArbitrationMailbox {AutoGroup, ModeratedRecipients, Never Delete, AsyncOperationNotification}

New MRM Policy {Keep for Audit, Keep 10,000 days, Recoverable Items 14 days move to archive, Junk Email...}

Business Analysts {Inbox 30 day expiry, 1 Year Delete, 6 Month Delete, Keep for Audit...}

Default MRM Policy {5 Year Delete, 1 Year Delete, 6 Month Delete, Personal 5 year move to archive...}

You might wonder why a retention policy called “ArbitrationMailbox ” is present. Do not remove this policy. It is used by
Exchange Online to look after the contents of arbitration mailboxes, which are used for different background processes
such as message moderation and OAB generation. Although irreversible harm will not be caused by removing this policy, it
will stop MFA processing the arbitration mailboxes and they will consequently swell up with unremoved content.

System tags : The set of tags defined for the ArbitrationMailbox policy have some special tags like ModeratedRecipients
, AutoGroup , and AsyncOperationNotification that are not available for use in normal retention policies. You can view
these tags by running the Get-RetentionPolicyTag cmdlet and passing the IncludeSystemTags parameter. Do not remove
these tags!

Assigning retention policies to mailboxes

A retention policy is assigned to a mailbox with the Set-Mailbox cmdlet or through the EAC. Using a GUI is an effective
way to apply a policy to one mailbox but the navigation from mailbox to mailbox becomes tiresome if more than a few
mailboxes must be processed. To assign a retention policy to a mailbox with EAC, select the mailbox, edit its properties, go
to Mailbox Features , select the retention policy, and then Save .

You do not have to assign a retention policy to a mailbox. To do this, select "No policy" and then Save . You can also select
a group of mailboxes and assign the same retention policy to the selected mailboxes in one operation. The Advanced
Search facility can be used to select a group of mailboxes based on their department, city, country, or region, or one of the
customized attributes. You can then bulk-apply a retention policy to those mailboxes.

Even though the EAC include search facilities to find mailboxes to which you want to apply a retention policy, PowerShell
is often the fastest and most efficient way to do the job, especially the need arises to update retention policies for many
mailboxes. For example, this command searches for a set of mailboxes based on the Office location and then assigns the
same policy to each mailbox in the set:

[PS] C:\> Get-Mailbox –Filter {Office –eq "NYC"} | Set-Mailbox

–RetentionPolicy "Retention Policy for NYC Users"

Preventing Retention Policies from Running

Exchange Online supports the use of retention holds to suspend the processing of the retention policy assigned to a
mailbox. The retention hold is designed for situations such as an extended vacation or illness when a user might not be
able to access their mailbox for an extended period and might therefore return to work to find that the Managed Folder
Assistant had removed many items from the mailbox.

You can only set a retention hold through PowerShell. This example sets a retention hold on Rob Young's mailbox between
21:00 on 3 March and 09:00 on 1 April. A retention comment gives some information to other administrators who might
wonder why the hold is in place.

[PS] C:\> Set-Mailbox –Identity 'Rob Young' –RetentionHoldEnabled $True

–StartDateforRetentionHold '03/03/2015 21:00' –EndDateforRetentionHold '01/04/2015 09:00'

–RetentionComment 'Rob Young on vacation until 1 April'

A retention hold can exist alongside a litigation or in-place hold. The retention hold only stops the processing performed
by the Managed Folder Assistant; the other holds stop items being removed from mailboxes until the holds elapse.
Exchange Online automatically places any recovered inactive mailbox on retention hold for 30 days after the recovery. See
Chapter 6 for more information on how to recover inactive mailboxes.

How Clients Use Mailbox Retention Policies

The tags contained in a mailbox retention policy (including Office 365 labels published through label policies) become
available to a client after MFA has processed a mailbox to apply the policy. Part of the processing is the creation of a
hidden “folder associated item” (FAI) of message class IPM.Information.MRM in the Inbox to hold details of the retention
policy, including its tags. Although you cannot see the FAI unless you use a utility like MFCMAPI to examine its contents,
the XML data contained in the PR_ROAMING_XMLSTREAM property of the item tells clients that a retention policy in
place for the mailbox and gives the necessary information to reveal retention tags through the interface. If Office 365
labels are in use, their details are combined into the information held about retention policies in a mailbox and they are
shown to users in the same way as personal tags. The MFA updates the hidden item each time it processes the mailbox to
add or remove tags or to stamp the mailbox with a different retention policy.

The next time a client opens a mailbox, the retention policy tags are available for display. Not all tags are shown for every
folder and item as the client limits the set to the tags that the user can apply. Figure 19-26 shows how OWA displays
retention tags. In this case, we are in the Inbox folder, which is a default mailbox folder. In some respects, this is a bad
example because the user is forced to select from a large set of tags. The set includes the parent policy (whatever tag
applies to the Inbox), personal tags, and Office 365 labels. Each tag displays its retention period, making it easy for the
user to know when the retention action will happen. The current state of the selected item is “Use parent folder policy” (a
tick appears beside this entry), which means that if the user does not select another tag, MFA applies whatever policy
exists for the folder. As the item is in the Inbox, this might be a default folder tag for the Inbox (if one exists in the policy).
If not, whatever default tag is in the policy is applied.

Unfortunately, experience proves that most users have no idea whatsoever what the meaning of personal tags are and
what happens when personal tags are applied to folders or items. They have no concept of the difference between an
archive policy and a retention policy. Indeed, most users have never explored how to use “Assign Policy” to tag an item. In
addition, if they use a mobile client they will never see this information because the client does not have the necessary
user interface to expose policy information.

Figure 19-26: OWA displays retention tags

You can apply personal tags to folders in an archive mailbox. However, when this happens, the MFA checks to see if a
personal tag is stamped on the same folder in the primary mailbox and if a difference exists, the tag used for the primary
mailbox is replicated to the archive folder. This is done deliberately to ensure that a difference does not exist in processing
behavior for the two folders.

Users are sometimes confused to discover that Outlook uses a slightly different approach. However, the same principles
apply. Select an item and use Assign Policy to see the available tags. In the case shown in Figure 19-27 , a different
mailbox is open, and a different set of retention tags are available. The tags are divided into archive and retention (which
is how Outlook refers to what OWA calls labels). Again, the retention period is clearly visible. In this case, because a tick
does not appear beside any tag, we know that this folder does not have an explicit tag assigned to it. Note that if Outlook’s
user interface cannot display the full set of tags available to the user, they see an entry for More retention policies ,
which they can select to see all the tags.

If you select a tag and apply it to an item, Outlook checks the retention period and compares it to the age of the item. If
the item is older than the retention period, Outlook warns the user that applying the tag will cause the item to expire. In
other words, the next time that the MFA processes the mailbox, it will apply the retention action to this item. The user can
then decide to go ahead and apply the tag or not. OWA does not check items in the same way and it is possible for a user
to assign a tag using OWA that will cause an item to expire without warning.

Figure 19-27: How Outlook 2016 displays retention tags

Set Folder Policy allows users to view what policy is assigned to a folder and set the retention and archive policy for the
folder (Figure 19-28 ). You cannot assign a policy to a default folder as these folders are governed by folder tags that can
only be set through a retention policy assigned to the mailbox. However, you can set the archive policy, even for default
folders. Policies can also be viewed and set by selecting a folder and viewing its properties. Another choice offered by
Outlook is to View Items Expiring Soon , which conducts a search of the mailbox to find items whose retention period
will expire in the next month.

Figure 19-28: Viewing the policies assigned to a folder



Revealing the actual retention dates : Although you might have read and understood the information published online
to explain how the Managed Folder Assistant calculates the retention period stamped onto items , there is nothing quite
like checking to satisfy yourself that the right period is being used. You can confirm how long an item will be kept by
examining its properties with MFCMAPI. Look for the PR_RETENTION_DATE property and you will find the date and time
that the item will expire. Once expired, the Managed Folder Assistant will invoke the retention action and the item will be
remove or archived.

Optional Personal Tags

Exchange does not limit users to the personal tags included in the retention policy assigned to their mailbox. In fact, if the
MyRetentionPolicies option is set in their user role assignment policy, they can use any personal tag defined within the
tenant. The set of personal tags is revealed through the "Retention Policies" section of OWA Options. Here the user can
see the set of tags that are already available to them through the retention policy that is assigned to mailbox. They cannot
remove any tag that is inherited from the assigned policy, but they can see an explanation as to what a tag does and they
can click [+] to add personal tags from the set of tags available in the tenant that are not already available to the user by
being included in the policy currently assigned to their mailbox (Figure 19-29 ). When a user adds a personal tag and
saves the new set, Exchange Online updates the MRM information held in the hidden FAI in the Inbox folder. If MFA has
processed the mailbox to update its retention policy data, the tag then becomes available to the user the next time they
restart Outlook or OWA.

Figure 19-29: How to use OWA options select a personal tag to use with items

Blocking the ability for users to select personal tags . The retention requirements for your company might not
accommodate the ability of users to select personal tags to apply to items because you want to restrict users to whatever
tags exist in the retention policies that are applied to their mailboxes. It is easy to prevent users from being able to add
personal tags by editing the default role assignment policy, the role-based access control policy that controls the options
available to users. To block users from selecting personal tags:

1. Go to the Permissions section of the EAC

2. Select User Roles
3. Edit the Default Role Assignment Policy and scroll down to the MyRetentionPolicies section
4. Uncheck MyRetentionPolicies and save the policy

The restriction will be enforced the next time the user logs on to OWA.

An administrator can also add a tag to a set available to a user by running the Set-RetentionPolicyTag cmdlet. In this
example, two personal tags are added to the mailbox of Kim Akers and the Get-RetentionPolicyTag cmdlet is then used to
check that the tags are in the set available to the mailbox.

[PS] C:\> Set-RetentionPolicyTag –Mailbox "Kim Akers" –OptionalInMailbox "Keep 10,000 days", "Keep for Audit"


[PS] C:\> Get-RetentionPolicyTag –Mailbox "Kim Akers" | Format-List

Name

----

5 Year Delete

1 Year Delete

6 Month Delete

Personal 5 year move to archive

Personal 1 year move to archive

Personal never move to archive

1 Week Delete

1 Month Delete

Never Delete

Keep 10,000 days

Keep for Audit

Default 2 year move to archive

To remove optional personal tags from a mailbox, run the Set-RetentionPolicyTag cmdlet and null the list. This is also a
good way to force a refresh of the MRM configuration data for a mailbox.

[PS] C:\> Set-RetentionPolicyTag –Mailbox "Kim Akers" –OptionalInMailbox $Null

How MFA Processes Retention Policies

A mailbox retention policy does not become effective as soon as you make changes because MFA must process the mailbox
to apply the policy first. When MFA processes a mailbox to apply a retention policy, it goes through the contents of the
mailbox to “stamp” folders and items with policy details. MFA updates several MAPI properties in the mailbox with
information to help MFA process items per policy, including:

A GUID for the retention policy. A policy name can be changed but its GUID (the retention identifier) never changes.
Retention period. The number of days that an item can stay in the folder before it is removed.
Retention expiry. The calculated date when MFA will remove an item.
Retention flags, including whether the tag is inherited from the parent folder or has been explicitly set by the user.
Archive period. The number of days that an item can stay in its folder before it is archived.
Archive date. The calculated date when MFA will move an item to the archive.

In addition to Exchange mailbox retention policies, MFA also processes the instructions of Office 365 labels and Office 365
retention policies if a mailbox or items in a mailbox come within their scope. This includes the removal of Teams
compliance records if these items come within the scope of an Office 365 retention policy.

Retention Processing
Although date calculation differs slightly for different item types, to simplify the discussion, we can say that a retention
period begins at the point when an item is first created in a mailbox (for messages, this is when the item is delivered to the
mailbox).

If you remove a default or retention policy tag from a policy, MFA will remove the tag from all items to which it previously
applied and will restamp these items with whatever tag now applies. This might be a default tag or no tag. Clearly MFA
must do a lot of work to do to update the items in a mailbox, so it is not recommended to remove tags without some
consideration of the possible impact. It is always less disruptive to change the set of tags contained in a policy than it is to
remove a policy and start again.

Any change made to the default tags or the retention policy tags in a policy will force the MFA to assess every item in a
mailbox to which the updated tags are applied and potentially update the properties of the item to reflect a new retention
expiry date. For example, changing the retention period for a tag invokes similar processing in that MFA must find every
item stamped with the tag and then update its retention period and expiry date. The same is true if you change the
retention action defined in a tag.

In addition, for Outlook desktop clients, if MFA updates an item because of a change to its tag, Outlook must synchronize
that update from the server to the cached copy in the OST. Although the amount of data involved to synchronize the
updated property for an item is small (less than 1 KB per item), the number of items in user mailboxes and the number of
mailboxes affected by the change might combine to create an unexpected load on the network. The load is temporary and
will pass once synchronization is complete.
An exception exists for personal tags. These tags stay in place on whatever items to which they have been applied and
MFA will continue to process the tags as before. The idea is that the user explicitly placed these tags on items or folders
and Exchange Online should continue to follow their wishes. However, personal tags that are removed from the retention
policy will be invisible to the user and cannot therefore be applied to newly-created items unless the user chooses to add
them back as optional tags (how this is done is explained in “Optional personal tags” earlier in this chapter).

Removing a tag forces its removal from every retention policy known to Exchange Online (for the tenant) and creates a
great deal of server processing to remove the now-deleted tag from items. MFA cannot process a non-existent tag so as it
processes mailboxes, it finds items that were previously stamped with the defunct tag and restamps the items with
whatever tag is now applicable (usually a default tag).

You can disable a tag by setting its retention period to Never . MFA will leave these tags in place but will ignore the items
stamped with the tag because it knows that they will never expire. Another way of forcing MFA to ignore previously-
stamped items is to set the RetentionEnabled property to False (this is what happens when you use the EAC to set the
retention period for a tag to "Never"). For example:

[PS] C:\> Set-RetentionPolicyTag –Identity "Keep for Audit" –RetentionEnabled $False

If you remove a retention policy, Exchange Online removes the policy from its configuration and removes it from any
mailbox to which it applied. This means that MFA will no longer process these mailboxes. You can find out if any mailboxes
have not been assigned a retention policy and apply a policy to these mailboxes by running the command:

[PS] C:\> Get-Mailbox –Filter {RetentionPolicy –eq $Null} |

Set-Mailbox –RetentionPolicy "New Default MRM Policy"

Renaming a retention policy does not force any processing on the part of MFA. Exchange Online uses the policy GUID to
associate the policy with mailboxes and the GUID is unchanged if a policy is renamed. The name of the policy is simply a
convenience to allow the administrator to know the intent and purpose of the policy, so the policy can be renamed as often
as you wish. For instance, here is how you might rename the Default MRM Policy.

[PS] C:\> Set-RetentionPolicy –Identity 'Default MRM Policy'

–Name 'Office 365 Exchange Book Retention Policy'

Incorporating Office 365 Retention Policies and Labels

The introduction of Office 365 retention policies and labels gives MFA added work to do as it must:

Make retention labels available to clients so that users can apply labels to items in the same way as personal tags.
Process mailbox contents per the settings of Office 365 retention policies applied to mailboxes. A mailbox can have
both an Exchange mailbox policy and an Office 365 retention policy assigned. If the settings in the two policies clash,
the MFA uses the rules of retention explained earlier in this chapter to resolve the differences.

Administrators do not have to do anything to force MFA to execute the added tasks. It all happens automatically.

Logging the Managed Folder Assistant

You can use the Export-MailboxDiagnosticLogs cmdlet to find out the last time that the Managed Folder Assistant
processed a mailbox and what happened during the run. The information about Managed Folder Assistant activity is
contained in the “Elc” (Electronic life cycle) properties in the report. For example, the ElcLastRunSuccessTimeStamp tells
you the date and time that the Managed Folder Assistant last successfully processed the selected mailbox while
ElcLastRunDeletedFromDumpsterItemCount holds the total number of items removed from the Recoverable Items folder
(the famous “dumpster”). In this truncated version of the output, we can see the last successful run of the assistant was at
01:03 on 12 April 2017 and that 554 items were removed (from all folders under the mailbox root), 16 items were
archived, and 556 items were moved from the Recoverable Items structure to the archive.

[PS] C:\> $Log = Export-MailboxDiagnosticLogs -Identity TRedmond -ExtendedProperties

[PS] C:\> $xml = [xml]($Log.MailboxLog)

[PS] C:\> $xml.Properties.MailboxTable.Property | ? {$_.Name -like "ELC*"}

Name Value

---- -----

ElcLastRunTotalProcessingTime 97103

ElcLastRunSubAssistantProcessingTime 61121

ElcLastRunUpdatedFolderCount 9

ElcLastRunTaggedFolderCount 0

ElcLastRunUpdatedItemCount 881

ElcLastRunTaggedWithArchiveItemCount 0
ElcLastRunTaggedWithExpiryItemCount 881

ElcLastRunDeletedFromRootItemCount 554

ElcLastRunDeletedFromDumpsterItemCount 0

ElcLastRunArchivedFromRootItemCount 16

ElcLastRunArchivedFromDumpsterItemCount 556

ELCLastSuccessTimestamp 12/04/2017 01:13:48

MFA Workcycle
The Managed Folder Assistant runs on a workcycle basis where a goal is set and the server hosting the mailbox figures out
how to meet the goal, taking system load and available resources into account. In an on-premises deployment, the
Managed Folder Assistant workcycle aims to process every mailbox at least once daily. Because greater attention is paid to
the consumption of resources by background processing in Office 365, the workcycle used for the Managed Folder
Assistant aims to process every mailbox at least once weekly. The larger quotas given to online mailboxes is one reason
why it is safe to extend the workcycle. However, although the formal workcycle goal is weekly, experience (and
observation of mailbox statistics as described above) proves that mailboxes are usually processed four or five times
weekly. If system resources are available, they are released to background processes like the Managed Folder Assistant
and this accounts for any discrepancy between formal workcycle goal and what happens in practice.

Knowing how to check whether MFA has processed a mailbox, we can write some code to check all user mailboxes. MFA
does not process mailboxes with less than 10 MB of content. It’s easy to amend this code to select a different group of
mailboxes for processing, such as all those belonging to a department. In addition, the script only reports two of the ELC
properties and you could add others as needed, such as the count of items moved to an archive mailbox.

[PS] C:\> $Mbx = Get-Mailbox -RecipientTypeDetails UserMailbox -ResultSize Unlimited

$Report = @()

ForEach ($M in $Mbx) {

$LastProcessed = $Null

Write-Host "Processing" $M.DisplayName

$Log = Export-MailboxDiagnosticLogs -Identity $M.Alias -ExtendedProperties

$xml = [xml]($Log.MailboxLog)

$LastProcessed = ($xml.Properties.MailboxTable.Property | ? {$_.Name -like "*ELCLastSuccessTimestamp*"}).Value

$ItemsDeleted = $xml.Properties.MailboxTable.Property | ? {$_.Name -like "*ElcLastRunDeletedFromRootItemCount*"}

If ($LastProcessed -eq $Null) {

$LastProcessed = "Not processed"}

$ReportLine = [PSCustomObject][Ordered]@{

User = $M.DisplayName

LastProcessed = $LastProcessed

ItemsDeleted = $ItemsDeleted.Value}

$Report += $ReportLine

$Report | Select User, LastProcessed, ItemsDeleted

Forcing MFA to Process a Mailbox

If you need the Managed Folder Assistant to process a mailbox urgently, for instance to publish the retention policy
information to a mailbox, you can run the Start-ManagedFolderAssistant cmdlet to request Exchange Online to process the
retention settings for the mailbox.

[PS] C:\> Start-ManagedFolderAssistant –Identity "Ben Owens"

If all goes well, MFA refreshes the retention settings for the mailbox and new tags and labels are available to users.
Sometimes kicking the MFA convinces it to do the right thing and process a mailbox, sometimes it does not. The problem
is that Exchange Online throttles background processing and does not allow mailbox assistants like MFA to run on
demand. Microsoft wants to smoothen server load to reduce the risk that background processing interferes with the ability
to be responsive to clients, which is why a seven-day workcycle exists for mailbox assistants. Telling MFA to process a
mailbox will have an effect if MFA considers that it is reasonable to go ahead because it has not processed the mailbox
recently. Running Start-ManagedFolderAssistant several times to convince MFA to start processing is a fruitless exercise.
The Start-ManagedFolderAssistant cmdlet also supports an –InactiveMailbox switch to force immediate processing of
inactive mailboxes (see Chapter 6). In the past, Microsoft has swapped between MFA processing inactive mailbox and not.
The current state is that MFA processes inactive mailboxes and applies Exchange mailbox retention policies and Office
365 retention policies against their content. One difference exists in the processing done for active and inactive mailboxes
in that MFA does not process archive tags when it deals with inactive mailboxes.

Stopping MFA Processing a Mailbox

Times occur when you might want to stop the Managed Folder Assistant processing one or more mailboxes. For example,
when you want to remove items from a mailbox when it is on hold because that information should not be in the mailbox as
when someone forwards a message with confidential information to mailboxes that should not receive them. To remove the
items from a mailbox that is on hold, you must release the holds temporarily, remove the items, and then replace the holds.
During this time, MFA might process the mailbox and remove other items. To stop this happening, you can disable MFA
temporarily by running the Set-Mailbox cmdlet to set the ElcProcessingDisabled flag.

[PS] C:\> Set-Mailbox -Identity "Kim Akers" -ElcProcessingDisabled $True

When the holds are in place again, you can reverse the process and release MFA by switching the value to $False . You
cannot disable MFA processing if a preservation lock applies to the mailbox.

Moving from Exchange Retention

Policies
Exchange mailbox retention policies have been in use since Exchange 2010. Microsoft’s long-term direction is to convince
tenants to move away from workload-specific processing, like that done by mailbox policies, and adopt the Office 365
equivalent, whenever that makes sense for a tenant. If you are a new tenant starting off with retention, the right decision
is to use Office 365 retention policies. However, if you have been using mailbox retention policies for many years and have
a hybrid tenant, that decision is not quite so clear cut.

To make retention policies available to other Office 365 workloads, Microsoft evolved and expanded the core principles
behind Exchange retention policies. In doing so, they have dropped some Exchange-specific features, like the ability to
move items to archive mailboxes. Losing the ability to archive items automatically is regrettable but this action is only
valid for Exchange and does not apply for other applications. A more fundamental problem for some is the loss of
granularity in that an Exchange retention policy can include tags for default folders, personal tags, and a set of default
tags to control deletion, archival, and voicemail. By comparison, an Office 365 retention policy applies the equivalent of a
single default tag (to remove or keep content) to mailboxes.

Office 365 creates in-place holds for all its retention policies. The holds apply for all locations covered by a policy and last
until the policy’s retention period expires, which means that users cannot remove content from Exchange, SharePoint,
OneDrive, or Office 365 Groups if that content comes within the scope of the policy. The integration of in-place holds into
Office 365 retention policies is an advantage over Exchange mailbox retention policies. Although Office 365 retention
policies do not allow you to assign tags to specific folders, you can create multiple retention policies and apply them to
specific content within mailboxes, if the content can be found using keyword queries or because it holds some sensitive
data., like social security numbers or other “personally identifiable information” (PII). The same sensitive data types used
in Data Loss Prevention policies are supported for retention. As outlined in Table 19-6 , some significant differences exist
between the two types of retention policies.

Exchange Retention Policies Office 365 Retention Policies

Applies Exchange mailboxes (including shared mailboxes). Exchange mailboxes

to
Office 365 Groups

SharePoint document libraries

OneDrive for Business sites

Skype for Business IM

Public folders

Teams

** Microsoft says that Yammer and Planner will be

supported soon.

Assigned Assigned to selected mailboxes (the default policy is Policies can be assigned to all or selected mailboxes,
to assigned to all newly-created Exchange Online SharePoint sites, OneDrive for Business sites, Skype for
mailboxes, including those migrated from on- Business conversations, Office 365 Groups, and public
 premises servers). folders. Locations can also be excluded from policies.

Composed Each retention policy consists of a set of folder tags Policies function like default tags in that the policy
of for specific system folders (like the Inbox), personal applies to all items in a location that are not otherwise
tags, and default tags. Three default tags can exist in tagged (for instance, with an Exchange personal tag or
a policy (for deletion, archive, and voicemail). an Office 365 label).

Actions Move to Deleted Items Keep and then remove content

Permanently Delete Keep and do nothing

Move to Archive Remove old content

Enforced Managed Folder Assistant. Managed Folder Assistant (for Exchange and Office 365
by group mailboxes); other background processes like
SharePoint timer jobs service the other locations.

In-place Not used. A separate hold must be created to keep In-place holds are automatically created for all retention
holds content. policies.

Table 19-6: Comparing Exchange and Office 365 Retention Policies

As obvious from Table 19-6 , significant differences in functionality exist between the two types of retention policies, so the
answer is unclear at this point. The most attractive feature of Office 365 retention policies is that they apply to many
locations rather than being limited to Exchange. Another good point in their favor is that you can combine Exchange
retention policies with Office 365 retention policies and labels, even if that might be a double-edged sword in terms of the
resulting complexity in retention processing. Being able to apply a single consistent policy across Office 365 is a huge
advantage, but that advantage might be lessened because of the loss of some of the granular processing available in
Exchange retention policies. The decision to move to Office 365 retention policies will therefore need considerable thought
and preparation on the part of tenants.

Every tenant is different and although it might be easy for a cloud-only tenant with relatively simple retention needs to go
ahead and embrace Office 365 retention policies, the situation is probably very different for large and complex tenants
that already have a well-defined retention strategy in place. Things become even more complicated for hybrid tenants,
who often want to use the same processes on-premises and in the cloud.

Experience and time will allow us to develop better answers. In the meantime, new tenants should start with Office 365
retention policies and labels while older tenants test, compare, and contemplate what is their best course of action.
Microsoft says that they plan to keep the older workload-specific functionality available within Office 365 to allow
organizations to make the transition. That is wise because the nature of retention is that items can be kept for a long time
and no one wants to be forced into changing strategy in such a way that it might affect terabytes of kept content.

On to eDiscovery
Now we have figured out how to manage information for data governance purposes, we should discuss how to find out
what users keep. eDiscovery is a very interesting area and there is lots we can do, including the creation of content
searches that can find information across the breadth of Office 365. Let’s do some searching.
Chapter 20: eDiscovery and Content
Searches
Tony Redmond

As we know, the Security and Compliance Center is the administrative interface for cross-service compliance and
protection operations, including the methods Microsoft creates to help tenants to find information needed for
investigations. The functionality exposed here replaces the workload-specific search capabilities (like the SharePoint
eDiscovery Center) and is a one-stop shop for eDiscovery within Office 365. Although it might not seem to be, forcing
tenants to move away from workload-specific searches to embrace service-wide processing is a good thing.
Functionality is more comprehensive and better integrated, searches are faster and capable of dealing with higher
volumes of data, and Microsoft is expanding eDiscovery coverage to new parts of Office 365.

In this chapter, we cover:

Content searches : How to search through Office 365 locations (sites, mailboxes, teams, etc.) to find
information.
eDiscovery cases : How to run full eDiscovery operations spanning searches, holds, and exports.
Advanced eDiscovery : How the Equivio technology can process huge volumes of information retrieved in
eDiscovery cases.
Supervision Policies : How to check a sample of email to be sure that people follow company policies.

eDiscovery is a specialized subject and not everyone is interested in how to run searches, create holds on different
resources, or export results. However, given the massive increase in data stored by companies and the litigious nature
of the world, it is likely that many Office 365 tenants will meet the need to run some form of search in any given year,
even just to find a document or message that a user thinks they have lost. With that thought in mind, we begin by
looking at content searches.

A Modern User Interface : In early April 2018, Microsoft introduced a new “modern” user interface for content
searches and eDiscovery cases that uses the same design language as other functionality in the Security and
Compliance Center. We describe and illustrate the modern user interface in this chapter. If you prefer using the older
interface, you can switch over and use it. However, Microsoft will deprecate the old interface in mid-2018 and at that
point, you will no longer be able to switch.

Content Searches
The design goal for eDiscovery technology is to meet the needs of investigation professionals who need to pursue a
compliance query from start to finish, potentially to the point where a company produces evidence in court to prove
potential wrongdoing or other legal points. Searching for evidence is where investigations begin, and that is why
content searches are so important.

Before beginning the process to create a content search, you should know:

What Office 365 locations to scan : You can search user mailboxes (including shared and inactive mailboxes),
group mailboxes, public folders, SharePoint Online sites, OneDrive for Business sites. Any information held in
mailboxes, including compliance records for Teams channel and private conversations, is available. You cannot
search Planner task metadata or Yammer information.
The keyword query to use to find information : Content searches depend on the content indexes created and
updated by the Search Foundation. As users create information inside a supported Office 365 location, the
Search Foundation automatically processes the information and adds it to its content indexes. Queries can be all-
encompassing (“find everything”), basic (“search for this term”), or very complex.

Content searches use the Office 365 server fabric, which means that a tenant can run multiple concurrent searches.
No limitation exists on the number of mailboxes or sites that you can include in a search as Microsoft designed this
generation of search technology to be able to cater for the largest Office 365 tenant. Content searches can find,
preview and export information. If you need to put an in-place hold on information to keep email and documents until
investigators no longer need that information, you create an eDiscovery case. The holds processed by eDiscovery
cases depend on content searches to find the information to hold. Content searches can also be part of an eDiscovery
case, where investigators can use a set of searches to interrogate various locations using different queries to retrieve
the information that they need. Table 20-1 lists some scenarios where content searches are useful.

Scenario Search action

Assess the risk to the business
from data shared by users with
external people from SharePoint
Online or OneDrive for Business
sites
Include the ViewableByExternalUsers keyword in the search and set it to True. In
order to exclude the aspx files used by SharePoint Online, the full query is
ViewableByExternalUsers:true AND ContentType:document NOT
FileExtension:aspx.

Find items that hold more than Include the SensitiveType keyword in the search and specify the sensitive data type
a certain number of sensitive to look for and the number of instances of that type that must be in a document for
data items (as defined for Data the search to retrieve it. For instance, SensitiveType:"Credit Card Number|5.." finds
Loss Prevention – see chapter documents that have more than five credit card numbers. See this page for more
21) information.

Investigate whether anyone Include the Recipients keyword and specify their SMTP email address in a search.
sent email holding confidential For example, Recipients:”TRedmond@Yandex.com”
information to a specific
address outside the
organization

Investigate whether mailboxes Include the Sender and Subject keywords and with values to search. For example
have received a specific Sender:”SomeGuy@FortuneForYou.com” AND Subject:”I have transferred
phishing message. $100million to you”

Check that a message that has Include the Body keyword in the query. For example, Body:”We are about to be
some specific text in its body is taken over by MegaCorp”
not circulating within the
organization

Table 20-1: Content search scenarios

You can combine keywords together to build a search query that covers multiple conditions. SharePoint Online and
Exchange Online have different abilities to use keywords. Some of the keywords are specific to documents and some
to mailbox items. For example, you cannot include the ViewableByExternalUsers keyword in a search that scans
Exchange Online mailboxes because this kind of sharing concept does not exist for Exchange. In addition, although
both SharePoint Online and Exchange Online support sensitive data types (like credit card numbers) for the purposes
of Data Loss Prevention, only searches of SharePoint content support these keywords. This page gives guidance about
the keywords that you can use for Exchange and SharePoint locations.

A content search can scan tens of thousands of mailboxes and return multiple terabytes of results. Tto ensure that
searches execute in a reasonable time, it is important that you limit the number of mailboxes where the search finds
relevant information by either restricting the number of source mailboxes or by refining the search query so that it
excludes irrelevant items from the search results. As the foundation for successful searches is based on precision, the
latter approach is always the best one to pursue.

As a very large multi-national company, Microsoft follows many legal and regulatory requirements around the world.
Its litigation team uses Office 365 eDiscovery to manage investigations and has written a white paper to explain how
it handles eDiscovery cases . Microsoft believes that its use of Office 365 compliance technology contributes to an
annual cost saving of some $4.5 million. Your mileage might vary.

Content Search Scalability

The added speed and scalability available to Office 365 content searches comes through the way that the searches use
the Office 365 infrastructure. A limit exists for the on-premises searches performed by either Exchange or SharePoint
based on the load that a single server can manage. For instance, when you launch an Exchange eDiscovery search,
one server manages synchronous connections with all the mailbox servers that host mailboxes involved in the search.
The same implementation applies for the on-premises and cloud versions of Exchange.

Office 365 content searches exploit the massive power available in the Office 365 server fabric to process whatever
quantity of information is involved in a search. Office 365 splits the workload of content searches across multiple
servers and asynchronous messages pass between the servers doing the work to keep them updated of progress. This
implementation reduces the potential for failure and parallelizes the workload to scale up to deal with far higher
volumes of data. In fact, Microsoft has known searches to cover over 700,000 mailboxes in a single operation.
Microsoft gathers statistics about the time needed to complete content searches and cites the guidelines shown in
Table 20-2 .

Number of mailboxes Average search time

100 30 seconds

1,000 45 seconds

10,000 4 minutes

25,000 10 minutes

50,000 20 minutes

100,000 25 minutes

Table 20-2: Average time for content searches (source: Microsoft)

Office 365 needs some time to create and start a search, so the time cited by Microsoft is indicative rather than
precise. Once the search starts, it rapidly processes target locations to find items based on the keywords and
conditions in the search criteria.

Content searches also include retry logic to handle the situation where a required mailbox or site is offline for some
reason. Usually a retry is enough to complete a search. Although multiple ways exist to search mailboxes available
inside Office 365 (including the Search-Mailbox cmdlet), constraints from their on-premises roots exist for all methods
except content searches. That is where the difference lies. See this page to understand more about the limits that
apply to content searches.

Creating a Content Search

You must be a member of the Compliance eDiscovery Manager or Organization Management role groups to be able to
create and execute a content search. Remember, these roles are separate to the administrative and default user
assignment roles used by Exchange Online to control access to functionality. You assign users to the necessary roles
through the Permissions section of the Security and Compliance Center. After you assign the necessary role to a
user, they must sign out and back into Office 365 to ensure that Office 365 respects the new permissions. Sometimes
it takes a little while before the new permissions are active. If this happens, just wait for a few minutes, and then
retry. In addition, an account used to conduct compliance searches should have a functional Exchange Online mailbox.
Although no strict licensing requirement exists for a mailbox, the preview function to review search results does not
work if the account used for searching has no mailbox.

Open the Security and Compliance Center and go to the Search & Investigation section and select Content Search
. You then see a list of the existing searches existing within the tenant (Figure 20-1 ). You can select an existing search
and continue working with it to amend its search criteria and run new queries or create a new search.
Figure 20-1: Viewing a list of content searches

The down arrow beside the New search button reveals three options to create a new search:

Guided search : Use a wizard to step through the different stages of creating search criteria. The steps to
create a guided search are described in the section about eDiscovery searches later in this chapter.
New search : Use the standard interface to create a new search. This is the default if you click New search .
Search by ID list : This creates a very focused search based on folder identifiers in Exchange mailboxes (which
is why it used to be called a targeted search). See this page for instructions.

In this example, we click New search to begin a new search. Office 365 opens the new search screen. At this point,
the search is a blank canvas and we need to add the criteria to allow Office 365 to find whatever information we need.
You create the necessary criteria by specifying keywords, conditions, and locations for the search.

Search Keywords
Searches use the Keyword Query Language (KQL) to compose queries to run against the content indexes. The
keywords are free-text expressions, meaning that you can use individual words or complete multi-word phrases. If you
include phrases, you must enclose them in double quotation marks. You can leave the keywords box empty if you want
to search for everything in the locations that you choose.

Keywords support prefix matching, which means that you can include the wildcard operator (*) with the beginning of
a word (for instance, “Office*”). You can combine free-text expressions with KQL operators such as OR, AND, NOT and
NEAR. If you include multiple free-text expressions in a query, KQL combines the expressions in the search using the
AND operator. If you enter an operator in lowercase (like “and”), Office 365 will offer to uppercase the operators
when it checks the query. You should always accept this offer as uppercasing the operators makes their purpose
obvious.

The Show keyword list checkbox allows you to input a keyword in each row of a list. The reason why you would want
to do this instead of typing all the keywords into the keyword box is that Office 365 will then generate statistics for
each keyword. You can then review the statistics to understand which keyword is most effective in terms of search
results.

Keywords with non-English characters : By default, content searches are language neutral. However, if you find
that a search does not return the expected results, the cause might be that some of the keywords use non-English
characters (such as Chinese). You can force a search to be language-sensitive by clicking the Query-language-
country/region icon on the top of the search query box and then select a language country culture code value for
the search (for example, Chinese – Hong Kong SAR).
Search Conditions
You can narrow the focus of a search by including property restrictions or conditions. We met some examples of
property restrictions in Table 20-1 , such as looking for documents shared outside the tenant. To make it easy for you
to include property restrictions in content searches, the Add conditions button exposes a dialog listing the most
common restrictions (Figure 20-2 ).

Because Office 365 indexes different properties for email and documents, the conditions are in three groups:

Common : You can use these restrictions to search against all Office 365 content. Date (creation date) and
compliance tag (an Office 365 classification label) are examples of a common condition.
Emails : You can only use these restrictions when searching against mailboxes. Sender is an example of a
condition unique to email.
Documents : You can only use these restrictions when searching against SharePoint Online and OneDrive for
Business sites. Last modified (date) is an example of a condition unique to files stored in these sites.

If you add a condition that is unsupported by a location included in a search, Office 365 cannot run the query and will
signal an error. Check the conditions that you want to include and then click Add to return to the query.

When you return, you will see that Office 365 has added the conditions to the search criteria. However, you still must
complete the conditions. For instance, if you add a compliance tag condition to the search, you must say what
compliance tag (classification label) or tags you want to look for. If you add a date to a search, you must input the date
range for the search and say whether you want to look for items between two dates, before a date, or after a date.

Figure 20-2: Defining conditions for a content search

Search Locations
The last step in composing search criteria is to define where Office 365 should look for items matching the specified
keywords and conditions. You have a choice to search in:

All locations : All Exchange Online mailboxes, all SharePoint Online and OneDrive for Business sites, and all
Skype for Business Online IM conversations.
Specific locations : You can target searches against selected mailboxes and sites. The locations shown in Figure
20-3 split into mailboxes, sites, and public folders. You can select individual mailboxes and sites, but if you want
to search public folders, you can only search all public folders. The MyAnalytics data listed under mailboxes
refers to the insights about how people use email and calendars. See this page for more information.
Click the Choose users, groups, or teams link to add the Exchange mailboxes you want to include in a search and
then input some characters to allow Office 365 to find matching mailboxes. As you can see from Figure 20-4 , the list
includes user mailboxes, shared mailboxes, group mailboxes (for conversations and meetings in Office 365 Groups and
meetings and compliance records for Teams), and inactive mailboxes (which have a dot “ “. prefix in front of mailbox
name).

Figure 20-3: Selecting locations for a content search

You can add distribution groups to the mailbox search (but not dynamic distribution groups). Office 365 expands the
membership of distribution groups and adds the individual mailboxes from each group to the set of target mailboxes.

Choose the mailboxes to include in the search and continue the process until you add all the target mailboxes to the
search.
Figure 20-4: Selecting mailboxes to add to a content search

To add SharePoint Online and OneDrive for Business sites to a search, you must know the URL for the target sites.
You can find the URL by opening the site in a browser and copying the URL, or you can run a PowerShell command to
find the site URL. The Get-SPOSite cmdlet returns all the sites in a tenant, including their URLs, but if you want to
search a site belonging to an Office 365 Group or Team, you can find the URL with the command:

[PS] C:\> Get-UnifiedGroup -Identity MyGroup | Select SharePointSiteURL

SharePointSiteUrl

-----------------

https://mytenant.sharepoint.com/sites/mygroup

Cut and paste the URL into the Edit locations screen (Figure 20-5 ) and click Choose to add it to the list of target site.
The URL for a OneDrive for Business site uses a different format. For example, here is the URL for the OneDrive site
belonging to Ben Owens:

https://mytenant-my.sharepoint.com/personal/Ben_Owens_mytenant_com/

Repeat this process to add all the sites you want to include in the search, and then Done to return to the list of search
locations. If the set of target mailboxes is correct, click Done to return to the search screen.
Figure 20-5: Selecting SharePoint and OneDrive sites to add to a search

At this point, all the necessary criteria are in place to run a search as we know the search query to use and the
locations to run the query against. Before you can run the query, you must save the search and give it a:

Name : A unique name for the search (or just "Test" if you run out of ideas). Only eDiscovery administrators see
this name, so it is sensible to input something that is meaningful to anyone who will access the Security and
Compliance Center. The name must be no more than 50 characters. Most organizations will follow a naming
convention for searches. For example, you could include the name of the department or group who requested the
search, or use a name assigned by the legal department.
Description: Free text field to record information about the search. For example, you might include more
information about the reasons for the search, who created the search, and its expected retention.

Click Save to save the search. The norm is for Office 365 to run the query, but before this happens, Office 365 checks
the query for any syntax errors and to suggest improvements if necessary. Once the query is acceptable, Office 365
runs the query against the target locations.

Hybrid SharePoint searches : If a hybrid connection is in place, it is possible to use hybrid SharePoint searches to
find content stored within on-premises SharePoint farms. However, when you run content searches, Office 365 filters
the results from the on-premises sources because no method exists to allow Office 365 to apply holds to on-premises
SharePoint data or to export data from an on-premises SharePoint location.

OneDrive Permissions
OneDrive for Business sites store the personal files of users. As such, tenant administrators cannot search these sites
without permission. If you want to include OneDrive for Business sites in a search, you need to assign permission to
allow whoever runs the content search access to the content stored in the site. The procedure to assign the necessary
permissions is explained in this Microsoft support article .

If you want to add the account used to run content searches as a site collection administrator for a single OneDrive
for Business site, the process is:

1. Connect to SharePoint Online with PowerShell.

2. Run the Set-SPOUser cmdlet to make the account running the search a site collection administrator. For
example:
[PS] C:\> Set-SPOUser -LoginName eDiscoveryUser@office365itpros.com -IsSiteCollectionAdmin $True

-Site https://mytenant-my.sharepoint.com/personal/kim_akers_office365itpros_com/

Previewing Search Results

When you run a query, Office 365 does not conduct a complete in-depth search. Instead, it consults content indexes to
generate statistics about the results that it estimates the search will find if you execute a full search. This is known as
a preview search. The actual number of items that a full search will find might differ from the preview, but that is
usually of minor importance. In any case, it is better to get quick results back from a preview rather than having to
wait for a full search to complete, the idea being that you get the chance to assess the effectiveness of the query and
can tweak its parameters to better focus on the desired results. The process of tweaking search criteria might go
through multiple iterations before you are happy with the search results and can then go ahead to export the full
search results.

When the estimate completes, Office 365 selects some samples from the top locations (those that hold most matches)
and displays the items (Figure 20-6 ). Typically, the individual items listed in the preview screen include email
messages, Teams compliance records, contacts, and Skype for Business IM conversations. If an item has an
attachment, it also shows up is viewable if it is an Office document. Naturally, the fewer the number of results, the
less time it will take for the search to show the preview results. A hundred or so preview items is usually a reasonable
amount to assess whether a search query is effective.

The preview screen supports options to see individual results (the sample items) and to preview automatically. These
are the default choices. If you select search statistics (Figure 20-7 ), you can see three different views:

Summary : Displays the locations scanned (SharePoint and Exchange), the number of locations with hits
(matching results), the number of matching items found, and the size of those items.
Queries : The location type (SharePoint or Exchange), the condition (search query used), the number of locations
with hits, the number of matching items found, and the size of those items. If you use a keyword list, you can see
how effective each keyword is in terms of locating items (a result labelled as Primary means the complete search
query; Keyword means that the results from a specific keyword). You can also see whether any unindexed items
matched the query. Unindexed items are often graphic files like a bitmap or JPEG file.
Top locations : The sites and mailboxes where most matching items were found, including the number of those
items and their size.

A search might find many more results than you possibly want to go through and review. Office 365 limits the number
of items selected for preview . This is acceptable because the intention behind the preview screen is to help
investigators understand the effectiveness of the search query in terms of finding the required data. The preview
screen is not a tool to browse through the complete set of items uncovered by a search. Table 20-3 lists the limits for
the preview screen in terms of its ability to display items found by searches.
Figure 20-6: Viewing preview results

Figure 20-7: Search statistics

Limit Number

Maximum number of items per user mailbox displayed 100

Maximum number of items from all mailboxes displayed (the newest items are shown) 1,000

Maximum number of items found in SharePoint Online and OneDrive for Business sites displayed 200

Maximum number of SharePoint Online and OneDrive for Business sites that can be previewed for search 200
results

Maximum number of items per public folder mailbox displayed 100

Maximum number of items from all public folder mailboxes displayed 200

Maximum number of public folder mailboxes that can be previewed 500

Table 20-3: Display limits for search preview

Exporting Search Results

Eventually you will be happy that a search finds the right information. At this point, you might want to export the files
and messages found by the search to perform a more detailed examination of individual items, give the data to
external investigators, or to make it available to someone who does not have the permission to run searches in the
Security and Compliance Center. The export function allows you to extract search results from the source locations,
copy them to a secure holding point in Azure, and then copy the data from Azure to PSTs (for messages), individual
files, or ZIP files. You can also export a report of the search results, meaning that instead of exporting the actual data,
Office 365 creates and exports reports listing the files and messages that it would export for a search.

Selecting Export Options

To begin, make sure that the search is current as you will not be able to export results if they are more than seven
days old. It has always best to refresh search results before beginning an export as this ensures that the exported data
will be completely up to date. To start the export job, select Export results from More in the menu bar. Figure 20-8
illustrates the screen used to gather information for the export process. While Office 365 always exports SharePoint
and OneDrive files as individuals file, you can export items extracted from Exchange mailboxes to:

A single PST for all Exchange content , which is convenient for investigators and usually the best option
when dealing with small amounts of information. Exchange Online does not include the New-
MailboxExportRequest cmdlet available in the on-premises version to export content from a mailbox to a PST.
However, you can mimic the functionality by creating a content search for all content in a single mailbox and
export the results (the entire mailbox) to a PST.
To multiple PSTs (one per mailbox), which allows the people who must process the results to split the workload
across multiple individuals. If an export processes more than 10 GB of data, the job will automatically split into
multiple PSTs, each of which is up to 10 GB (it is possible to change this limit with a registry setting ).
To individual MSG files . This choice exists because some third-party investigation tools do not support import
from PST. In addition, if you want to export messages encrypted with rights management, you must export them
as individual files. The export process can decrypt email messages protected with IRM. Any protected
attachments stay encrypted because the decryption process does not extend to documents (which is the reason
why protected files found in SharePoint sites are still encrypted and their contents are not indexed). An exception
exists for messages protected with Office 365 Message Encryption. Copies of received messages extracted from
user mailboxes remain encrypted while copies of sent messages are unencrypted because these messages have
not passed through the transport system, where Office 365 Message Encryption processes messages. The export
job writes a copy of each message uncovered by the search to the target destination in the file system. The
administrator can then move the copied files from there as needed. The export job organizes the individual MSG
files into a folder structure beginning with the mailbox that the items belong to and then:

Recoverable items: Items extracted from folders in the Recoverable Items structure are here. For example, if an
item purged by the user but kept due to a hold placed on the mailbox, it is in the Recoverable Items\Purges folder
in that user’s mailbox.
Top of Information Store: The export job places items extracted from folders visible in the user’s mailbox here.
The export job creates a separate file system folder for each folder in the mailbox where the search found a
matching item. Given the complex folder structure that exists within some mailboxes, it is possible that the 260-
character maximum path to a Windows folder will be reached. When this happens, the export job truncates the
folder names to stay within the limit.

To a Zipped file : This option includes both Exchange and SharePoint content. Exchange items are in separate
MSG files while the export includes individual files SharePoint and OneDrive. Exporting to a ZIP file avoids the
issue that sometimes arises when the file path to a SharePoint item exceeds 260 characters. Like PST files, if an
export exceeds 10 GB, Office 365 splits the export into multiple ZIP files (you can use the same registry setting
mentioned above to control PST sizes to change the maximum size for a ZIP). Note that files in a ZIP file only
 have the modified date for files, not the created dates. However, the created date is stored in the XML manifest
for the export.

If the search includes SharePoint or OneDrive for Business sites, you will have the choice to include versions for
documents found by the search. Versions are only available for documents if versioning is enabled for document
libraries.

Figure 20-8: Defining export settings for a search

You can export up to 2 TB of data for a single search. If a search finds more than this amount of data, you will have to
split the search to get under the 2 TB limit. One way of doing this is to create several searches, each of which uses the
same keyword query but different date ranges. Reflecting that it is a multitenant environment, Office 365 also limits a
tenant to exporting 2 TB of data in a single day. A tenant can have up to ten export jobs running concurrently, but a
single user can only run three of those exports.

Speeding Up Exports : When you download search results, the export tool starts some concurrent operations to
fetch information from Azure. By default, the number of concurrent download operations is 8 times the number of
cores in the computer you use. To speed things up and reduce the time needed to download mail items to a PST and
documents to the target destination, you can increase the number of concurrent operations by making a change in
the system registry . The maximum you can set is 512. Be aware that increasing concurrent downloads might cause a
problem if insufficient network capacity exists between your PC and Office 365, so it is wise to increase the value
gradually to discover the sweet spot where downloads are rapid and do not overwhelm your PC or network.

Handling Unindexed Items

Unindexed items in SharePoint or OneDrive for Business sites can be of interest to investigators and you should
include these items in exports if you want to be sure that an investigation gets the opportunity to consider every
possible. After all, it is possible that a human might make sense of a file included in an export where Office 365
cannot. Three options are available:

All items, excluding unindexed items . This is the default and means that Office 365 only exports items
meeting the search criteria.
All items, including unindexed items : The export includes unindexed items, but only if the search also finds
items matching the search criteria in a site.
All unindexed items : The export includes all unindexed items from all sites in the search, even if the search
does not find matching items.

The usual process is to exclude unindexed items from exports and then decide whether a deeper examination is
necessary incorporating these items.

Downloading Exported Results

After selecting the options for the export job, click Export . Office 365 starts a background process to extract the
information from source locations and copy it to the holding area in Azure. To follow the progress of the export job,
click the Export tab in the menu bar to see a list of all the export jobs processed for searches. Select the export job
for your content search (it has the same name as the content search with a suffix of “_Export”). Office 365 displays the
status for the export job. It will take some time to export data for a content search that uncovers tens of thousands of
items spanning gigabytes of data. In this case (Figure 20-9 ), the number of search results is small, and we can see
that the export is complete.

Before going ahead to download the exported results from Azure, we must copy the export key. The export key is a
shared access signature to grant access to the secure storage area holding the export results in Azure and is of the
form:

?sv=2014-02-14&sr=c&si=eDiscoveryBlobPolicy9%7C0&sig=PpqtiOKpBMzZPtA1ksuY8iciP6jsYYI2VHePjDXY45w%3D

The export key becomes the credentials to authorize the download the exported data using a program called the
eDiscovery Export Tool. Essentially, the report key is a token that Azure Data Services recognizes when offered by a
process that wishes to access some data belonging to another entity. You cannot use the key to access data held in
Exchange Online or SharePoint Online with a browser or another client. After copying the export key to the clipboard,
you can paste it into Notepad or other editor to make sure that it is always at hand. However, if you are ready to go
ahead and download the results, click Download Results . It doesn’t matter if Office 365 has not finished preparing
data for export yet because the export tool checks with Office 365 when it downloads information and will pause if
necessary to let Office 365 complete the export.

The original version of the Security and Compliance Center supported the use of Chrome browsers to download the
eDiscovery Export tool. Chrome used the ClickOnce add-in for this purpose, but the latest versions of Chrome do not
support the add-on. For this reason, you must use either Edge or Internet Explorer to download and run the export
tool (you can use Chrome for all other search functions).

The download tool starts automatically after the download completes. To go ahead, enter (paste) the export key and
the target location for the tool to copy the exported data (Figure 20-10 ). The export tool then authenticates itself with
Azure and downloads the exported data to the selected destination, including metadata for SharePoint and OneDrive
documents.

If necessary, you can use Restart Export to recommence the export process. This action removes any earlier search
results stored in Azure and then recopies the search results from the search locations to the holding location in Azure.
Figure 20-9: Downloading export results

Figure 20-10: The eDiscovery Export Tool starts

The export job creates several sub-folders in the destination to hold the export data. The folder named after the date
and time of the export includes an Electronic Discovery Reference Model manifest listing all the exported items. The
Exchange folder holds the PST files used to export the data. Figure 20-11 shows the content exported from a
SharePoint Online document library. In this case, the export job copied the files to a ZIP file. Once exported to disk, it
is easy to give the information found by a search to an external company, such as specialized legal investigators. The
external company can then apply their own tools to analyze the search results. Alternatively, as discussed later, you
can license the Advanced eDiscovery capabilities available for Office 365 to analyze the search results.
Figure 20-11: SharePoint documents exported from a search

Export for third-party review : The eDiscovery industry spans many companies who specialize in the analysis of
information recovered from IT systems like Office 365. In February 2016, as part of their work to improve the
capabilities of “Advanced eDiscovery” within Office 365, Microsoft announced an initiative to support the export of
Advanced eDiscovery information in a form that third-party applications can interpret and analyze. The export data
exists in a location in Azure Data Services controlled by the third-party. Contact Microsoft to get an up-to-date list of
certified partners if you are interested in this capability.

Export Search Report

Being able to preview the results of a content search gives investigators an insight into the kind of information
uncovered by a search. However, it would be unreasonable to try to preview every item found by a search, especially
when large numbers are involved, in which case an investigator might want to make a broader check of the data
uncovered by a search before going ahead to export the data. To export the reports for a search, select the select and
then Export results from the More menu. This downloads the same reports (the XML manifest, summary log, and
results log) holding details of the search results included in the full set of downloaded search results. When you
choose to export the report, an export job generates the reports and stores them in Azure. You receive a export key to
access to the data and can use same download tool to export the reports to the nominated destination. After checking
that the correct results are available, you can then continue to perform a full export.

Reopening a Search
You can go back and open a search at any time. Select Content search under Search & Investigation in the
Security and Compliance Center to list the available searches and select the one you want to work with, and then
Open query . If the search results are less than 7 days old, Office 365 displays the preview items. If the search is
older, Office 365 runs an estimate search to refresh its results and displays preview items. To change the search
criteria, cancel the search and then modify the keywords, conditions, or locations as needed. Then click Save & run
to execute the search based on the new criteria.

Compliance Boundaries
When you have the permission to create and run content searches, you can look for anything across the set of
supported locations in a tenant, including the ability to pry into sensitive mailboxes or hunt for interesting documents
stored in document libraries that you would not otherwise be able to see. Compliance Security Filters allow tenants to
impose control over the data visible to investigators by establishing boundaries for searches. Large companies often
divide administrative and other responsibilities along geographic or divisional lines. When the time comes to conduct
content searches, they might not want those who run the search to be able to include search locations outside their
business, country, or region. This makes a lot of sense: someone running a content search to respond to a discovery
action in France does not necessarily need to look at German mailboxes. Apart from respecting user privacy,
Compliance Security Filters also mean that content searches return less data that investigators must then review.

A Compliance Security Filter creates a restrictive view of mailboxes or SharePoint and OneDrive sites within a tenant.
When users that conduct searches come within the scope of a filter, they cannot see any data returned by searches
except that given by the restrictive view. Therefore, we can set things up that U.S.-based eDiscovery administrators
only can see results from U.S. based mailboxes or that only certain eDiscovery administrators are able to search
particularly sensitive SharePoint sites.

Before starting to plan your filter strategy, you should read the Microsoft support article about Compliance Security
Filters and this blog post . The key point to note is that you can only create and manage filters through PowerShell. To
guide you in creating a filter, answer the following questions:

Who will the filter apply to? You can specify individual users or use the name of a Security and Compliance
 Center role group, including a role group created specifically for this purpose. You cannot use a distribution
group, Office 365 Group, or security group to define a set of users.
What can the users do ? You can restrict users to individual compliance actions (Export, Preview, Purge,
Search) or “All”. You cannot specify two or three actions. In most cases, you will want to use Search or All.
What can the users see? You can combine mailbox and SharePoint locations into a single filter that works
across multiple workloads. In both cases, you can have filters that look for specific objects (mailboxes or sites) or
content (based on KQL queries).

With these points in mind, here is a simple filter that applies to a single named user, allows that user to perform all
compliance actions, and restricts them to searching mailboxes with a specific value in their CustomAttribute6
property. Here is the command to create the Compliance Security Filter.

[PS] C:\> New-ComplianceSecurityFilter -FilterName VikFraudSearch -Users "Marc Vigneau" -Filters "Mailbox_CustomAttribute6 -eq 'POI'"
-Action All

To add mailboxes to the set that searches can find, we update the property as follows:

[PS] C:\> Set-Mailbox -Identity TRedmond -CustomAttribute6 POI

After updating the mailboxes, you can test the filter to check the set returned with Get-Recipient :

[PS] C:\> Get-Recipient -RecipientType UserMailbox -RecipientPreviewFilter {CustomAttribute6 -eq 'POI'}

Now that we know that the filter is valid and returns a set of mailboxes, we can test it with a search. Log into the
Security and Compliance Center using the account of one of the users restricted by the filter. The user must have the
permission to run searches. Create a new search for all mailboxes with a query that you know will find some
information. Launch the search and wait for it to complete. The results should reflect the filter in terms of the number
of mailboxes and the amount of information found.

Now log into the Security and Compliance Center as another administrator who the filter does not restrict and run the
same search again. This time the results should be very different. Figure 20-12 shows an example of a content search
run by restricted (left) and unrestricted (right) eDiscovery administrators. The restricted search only scans 5
mailboxes while the unrestricted search looks through 227. The number of found items is also different, as you’d
expect.

Figure 20-12: The results of applying a search filter

Microsoft suggests many other examples of Compliance Security Filters in its documentation, including how to filter
mailboxes based on the ISO 3166-1 code (for example, 124 is the three-digit code for Canada while 372 is the code for
Ireland). One example of obvious interest is a filter that restricts access to confidential or sensitive mailboxes. The
code to create such a filter first finds the distinguished name of a distribution group called the “Senior Leadership
Team”. The members of the group are the mailboxes that we want to restrict. We include the distinguished name in
the filter to stop anyone who runs a search against these mailboxes from being able to preview items found by the
search.

[PS] C:\> $DG = (Get-DistributionGroup -Identity SLTDL).DistinguishedName

[PS] C:\> New-ComplianceSecurityFilter -F ilterName NoSLTPreview -Users A ll -Filters "Mailbox_MemberOfGroup -ne '$($DG)'" -Action
Preview

The Get-ComplianceSecurityFilter cmdlet reveals the details of the filter:

[PS} C:\> Get-ComplianceSecurityFilter -FilterName NoSltPreview

FilterName : NOSLTPREVIEW

Description :

Action : Preview

Users : {all}

Filters : {Mailbox_MemberOfGroup -ne 'CN =DL Senior Leadership Team,OU=tenant.onmicrosoft.com,OU=Microsoft Exchange Hosted
Organizations,DC=EURPR04A002,DC=prod,DC=outlook,DC=com'}

This kind of filter effectively stops casual browsing of sensitive mailboxes by people who should know better. It does
not stop searches finding items, nor does it stop eDiscovery administrators being able to export items found by the
searches. The downside of a filter is that it applies to all content searches, including those executed by people that
might legitimately have reason to preview content found in the sensitive mailboxes. For this reason, it is best to define
the users to whom the filter applies whenever possible.

Users cannot preview mailbox items found by a content search in the designated mailboxes, but they can preview
documents and other items (like PDFs) found in sites covered by searches. To ensure full confidentiality for the Senior
Leadership Team, you need to define a site filter to protect these locations. This filter restricts access to documents
found in the document library. In this context, “All” means members of the eDiscovery manager role group rather than
all users.

[PS] C:\> New-ComplianceSecurityFilter -FilterName NoSLTPreviewDocs -Users "All" -Filters " Site_Site -ne
'https://tenantname.sharepoint.com/sites/ SLTGroup'" -Action Preview

You can create a filter with clauses for multiple workloads. Here is a filter with two clauses: one for U.S.-based
mailboxes and the other for a specific SharePoint site:

[PS] C:\> New-ComplianceSecurityFilter -FilterName CountryFilter -Users annb@contoso.com -Filters "Mailbox_CountryCode -eq '840'",
"Site_Site -eq 'https://tenant.sharepoint.com/sites/Confdocs'/'"

-Action All

The Effect of Filters on Content Searches

In its documentation, Microsoft explains that:

“The permissions filter is added to the search query when a Content Search is run. The permissions filter is essentially
joined to the search query by the AND Boolean operator.”

And:

“In a Content Search query, multiple permissions filters are combined by OR Boolean operators. So results will be
returned if any of the filters are true. In a Content Search, all filters (combined by OR operators) are then combined
with the search query by the AND operator.”

If you use multiple filters, the filters are joined with the query (AND) and then combined (OR). This allows the search
to find all the content per the query and then apply each filter to arrive at a combined set of results. The exact results
that any action (search, preview, or export) produces depends on the actions specified in each filter. The ability to
combine filters with content searches creates a great deal of flexibility in what you can do to control searches, even if
it might take some time and effort to arrive at the filters needed to generate the desired result.

Note that you cannot exclude specific public folders using a search filter. The filters only work for user and group
mailboxes and SharePoint and OneDrive sites.

Audited searches . Office 365 records all the search actions performed with content searches or eDiscovery cases
in the Office 365 audit log under the “eDiscovery activities” category. Audited actions include creating, editing, or
starting a content search, viewing items through preview search, and exporting search results. Individual workloads
like Exchange do not record their eDiscovery searches and holds in the audit log. You can search the audit log to find
eDiscovery activities using the Audit log search in the Security and Compliance Center (see Chapter 21), or through
third-party products like Quadrotech Radar for Security and Audit.

Office 365 eDiscovery

Content searches are useful in a variety of situations. Sometimes it is to find information that a user has “lost,” and
sometimes it is to find information for more serious reasons, such as looking for evidence of corporate or personal
malfeasance needed to prove a point for internal or external purposes. If the information exists in an indexed Office
365 location, content searches will find it.

Good as content searches are at finding information, searching is only part of the eDiscovery lifecycle. And while the
results of an individual search might be critical to proving or disproving a point, many eDiscovery projects involve
teams of investigators using multiple searches looking for different sets of information. Within Office 365, eDiscovery
cases deliver the structure needed by investigators to organize their work, including searches, holds, and exports. To
access eDiscovery cases, go to the eDiscovery section under the Search and Investigation section in the Security
and Compliance Center, which reveals the set of eDiscovery cases available to the logged-in user (Figure 20-13 ). In
this case, most of the cases are “Active,” meaning that they are being worked by investigators. Closed cases are still
available and can be opened if necessary.

Figure 20-13: Listing eDiscovery cases

eDiscovery Case Components

If you open an eDiscovery case, you see that its major parts are:

In-place holds to ensure that users cannot remove information . Holds ensure that Office 365 keeps
information needed for an eDiscovery case. A hold covers a set of locations (Exchange mailboxes, SharePoint and
OneDrive for Business sites, and Exchange public folders) and criteria to specify the items in those locations that
come under the scope of the hold. A hold can be as broad as to include all items in all available locations within a
tenant or as specific as to hold just one or two items found with a highly-specific phrase in a selected mailbox.

All items that fall under the scope of the hold stay in the specified sites and mailboxes
(including those used by Office 365 Groups). Users can try to remove items that come
under the scope of the hold, but if this happens, Office 365 keeps a copy of the item until
the hold expires. In-place holds on Exchange mailboxes include both the primary and
archive mailboxes. Standard content searches do not include the ability to place holds on
sources. Note that a user account must have at least an Exchange E3 license before you
can place it on hold. The license must stay assigned to the account for the duration of
the case. An eDiscovery case can include multiple holds, each of which has its own
target set of locations and hold criteria.

Searches used to gather information of interest to the case . The searches in an eDiscovery case behave
very much like content searches. An eDiscovery case might only have one search, but it might equally deploy
 multiple searches, each of which focuses in on different material. Each search covers a different aspect of the
case and might look for content based on different queries, various locations, or even content created in different
languages. The intention is that multiple searches allow investigators great flexibility in how they approach
looking for information because they can use multiple searches to interrogate information. The searches created
for eDiscovery cases do not appear along with other content searches because they belong to an eDiscovery case.
Both types of searches use the same technology to find information using the content indexes populated from
Exchange Online and SharePoint Online data.

One difference between regular content searches and those executed within an
eDiscovery case is that in a case, you can specify the special “Locations on hold ” target
for a search, meaning that you want to search the held content in all the locations
included in a case. For example, assume two holds exist for the case. The first covers two
mailboxes and two sites and the second spans a further twelve mailboxes. In this
instance, “Locations on hold ” means search the held content in the fourteen mailboxes
(including their archive mailboxes if these exist) and two sites. If the scope of any of the
holds belonging to a case changes (such as adding new locations), the scope of
“Locations on hold ” changes to match the holds when you refresh the search results.
Searching against on-hold content is a current workflow scenario in eDiscovery
situations where investigators place locations such as mailboxes or sites on hold before
starting to search those locations. The names assigned to searches belonging to
eDiscovery cases must be unique and not clash with standalone content searches.

Exports of information gathered by the searches . An eDiscovery case might use several searches to find
different sets of information. You can export results from a single search using the same steps as for exporting
results for a normal content search, or you can combine the results of multiple searches belonging to a case to a
single set of data to give to external investigators.

When you combine results from multiple searches, Office 365 combines the queries
using the OR operator to form an overall query and runs the query against the locations
to generate the search results. In other words, the search results created for export
come from a single search instead of assembling together the sets generated for the
individual content searches. Office 365 deduplicates the results of all searches so that
the export includes only a single copy of an item found in a location. Because of the way
that a combined search brings multiple search queries into one, it is possible that the
overall keyword limit for a query (500) might be met. If so, you will see an error and the
search will end. To achieve the desired result, you must combine fewer searches
together or simplify the queries.

Microsoft refers to the eDiscovery functionality in Office 365 E3 as Core eDiscovery. Advanced eDiscovery, discussed
later in this chapter, is a feature of the Office 365 E5 plan that can also be licensed through an add-on.

Case Members
Each case has one or more members, each of whom must be a member of the eDiscovery Manager role group. The
members are the only people who can access the results of the searches associated with a case. The eDiscovery
Manager role group divides into two sub-groups:

eDiscovery Managers deal with specific cases. They only have access to the content belonging to those cases.
Office 365 only reveals the presence of cases owned by a specific eDiscovery Manager when they access the
Security and Compliance Center. They cannot see the cases belonging to other eDiscovery Managers.
eDiscovery Administrators have oversight over all eDiscovery cases and can view and edit any case within the
tenant regardless of who is the eDiscovery Manager for the case. To access a case, an eDiscovery administrator
adds themselves as a member of the case.

It is logical that a very small set of users within a tenant should be eDiscovery Administrators.

Case Management
When you access the eDiscovery section of the portal, Office 365 shows you all the cases that your account manages
or has access to. You can then Open an existing case, Create a case to start a new case, or click the name of an
existing case to update the properties of that case, including the members working on the case and its current status.
Figure 20-14 shows that the selected case has two members. You can use the same form to manage the status of the
case by closing or removing the case. Another aspect of eDiscovery cases is that investigations are often long projects.
In this case, we have a recent case that began in April 2018, but it is certainly not unknown for cases to last five years
or more.

Figure 20-14: Members of an eDiscovery Case

Creating a New eDiscovery Case

From the eDiscovery section of the portal, click Create a case to begin. You can then input the name of the new case
and some details to describe the need for its creation. Figure 20-15 shows a typical situation. The name of the case
clearly shows its intent while the description tells anyone reviewing the case what its purpose is and who authorized
the investigation. Click Save to continue and be returned to the portal.
Figure 20-15: Creating a new eDiscovery case

Creating a new case creates the eDiscovery case container for searches, holds, and other activities. Nothing much
exists in the case at this point, so we should select the case and click Open to begin adding these elements to the
case. The case screen then displays to allow us to access the different components.

eDiscovery Case Holds

The next step is to create one or more holds to preserve content needed for the case. You can use multiple holds for
an eDiscovery case but in most cases one hold that applies to Exchange mailboxes and SharePoint and OneDrive for
Business sites is enough. Click Hold to access that section of the case and then Create (+) The steps to create a hold
are:

1. Name the hold. Assign a unique name for the hold along with an optional description. Ideally, the description
should tell an eDiscovery manager the intention behind the hold and link the hold to the eDiscovery case. For
instance, if the case name is “Investigation 2018-003,” then you might call the holds in the case “Investigation
2018-003 #1,” “Investigation 2018-003 #2,” and so on following whatever naming convention makes sense.
2. Define the locations the hold will cover. The locations divide into:

1. Exchange email: User, shared, and group mailboxes. Group mailboxes include group conversations and calendars
for Office 365 Groups and the calendar and compliance records for Teams. If you select a distribution group,
Exchange expands the membership and adds valid mailboxes to the hold.
2. SharePoint Online and OneDrive for Business sites.
3. Exchange public folders. You can include or exclude all public folders, but you cannot select specific public
folders.

3. Define the search criteria to find items to hold. The criteria are like those used for content searches and include
keywords and conditions.

In Figure 20-16 , we see the final point in creating a hold, including the query used to find items. A hold does not have
to include a query, and if it does not, it means that you want to apply a hold to every item in the selected locations. You
do this when you want to preserve complete mailboxes or sites because you are unsure of the material that you want
to hold.
Figure 20-16: Creating a hold for an eDiscovery case

When you create the hold, Office 365 publishes the hold details to the workloads for the selected locations. It can take
some time to synchronize across all workloads to make the hold effective everywhere, but it should certainly be in
place within a few hours.

To view information about a hold, select it to reveal a details pane. Here you find the current hold statistics provided
by the relevant workloads to show how many items come under the scope of the hold, the date and time of the last
modification for the hold, the user (case member) who changed the hold, and its status. When Office 365 is publishing
a new or updated hold to workloads, the status is “On (Pending) .” Once the hold is effective in all workloads, the
status changes to “On (Success) .” If you change a location or query for a hold, the status reverts to “On (Pending) ”
while Office 365 publishes the changes to the workloads.

The hold status is controlled by a slider. If you move the slider to Off and save the update, Office 365 instructs the
workloads to release the hold. Users can then permanently remove items that previously came under the scope of the
hold, so it is unwise to release a hold unless you are positive that the eDiscovery case no longer needs the items.

eDiscovery Case Searches

After applying a hold to make sure that users can no longer remove any content that we might need for our
eDiscovery case, we move on to searching. The simplest eDiscovery case has just one search, but more complex cases
might use multiple searches, each of which takes a different approach to looking for material of interest. The idea is
that an investigator can use different searches to home in on the information they need. Apart from the ability to use
different queries to focus on different material in each search, the searches in a case might target different locations.
One search might focus on a set of mailboxes and look for a specific item. A second search might look for some
documents in a SharePoint Online site and a third might concentrate on a single user and retrieve a much wider range
of content from their mailbox. Like other searches, you can reiterate several times to refine the results retrieved by
search criteria until you find the desired content.

Click Search to go to the search page and then New search to create a new search. As with regular content
searches, you can opt for a guided search, a search using an ID list, or a regular search. The steps to create a regular
content search are described earlier in this chapter, so we describe a guided search here. Three steps occur:

1. Name the search . Give the search a unique name, and as with holds, it is best if the name links the search to
the eDiscovery case.
2. Define the target locations to search . You have three options:

All locations: This choice creates a search across all content locations (all mailboxes, all sites, all public folders)
in the tenant. Because of the number of search resulted generated, this is a catch-all option that should be used
with care. Office 365 Advanced eDiscovery is an effective way to process large numbers of search results.
 Locations on hold: Search all the locations placed on hold by the eDiscovery case. If the case includes multiple
holds, the search covers all the locations from all holds. The search also combines queries specified in the holds
to make sure that it includes all relevant items.
Specific locations: Select the mailboxes and sites for which the hold will apply.
Create the query : As with content searches, you enter the keywords and conditions to construct a query to find
information in the selected locations. If you leave the keyword field blank, it means that you want the search to
return every item found in the selected locations.

Figure 20-17: Viewing the preview results for a search in an eDiscovery case

Click Finish to create the search and return to the search interface. Office 365 launches the search and displays
preview results (Figure 20-17 ). As you can see, searches inside an eDiscovery case use the same user interface as
regular content searches.

eDiscovery Case Exports

After investigators run searches to find the information they need, they can then export the search results. This
process works like exports for content searches, with the notable exception that you can select to export the results
for multiple searches in the same operation. The same general approach is used.

Access the search whose results you want to export.

Click the More button and select Export Results .
Set the export characteristics:

Decide what items found by the search to include in the export (all items, all items except those that are in an
unrecognized format, encrypted, or indexed, or just the items in an unrecognized format, encrypted, or
unindexed).
Decide what PST structure to use for mailbox items (one PST per mailbox, one PST for everything, one PST with
all messages in a single folder) or to use individual MSG files.
Decide whether to deduplicate the search output.
Decide whether to include versions (if available) for SharePoint and OneDrive for Business documents.

A background job named “Search Name_Export” then exports the search data to a secure Azure location.
When the search data is available, you download it to a workstation using a secure key generated by Office 365
as credentials to access the data.
Investigators use the downloaded PSTs, MSG files, and documents to review the information and assess its
content.

Bulk Actions
Because eDiscovery cases can span multiple searches, it makes sense to be able to be able to perform bulk operations
against searches. If you select multiple searches, Office 365 opens a Bulk actions panel offering these choices

Delete the selected searches.

Edit search locations (but only if the locations are not on hold).
Edit the keyword searches applied against the search locations.
 Examine search statistics for the combined searches (Figure 20-18 ) with the intention of helping investigators
understand how effective the searches are in finding information so that they can tweak settings to improve the
searches. Like the statistics for an individual search, you can see top locations, a summary, or results from
individual queries and keywords.
Export the results for all searches in one operation.
Export the reports for all searches in one operation.

Figure 20-18: Viewing results for multiple searches

Closing eDiscovery Cases

Eventually, when the investigation for an eDiscovery case winds down, the case manager can close the case by
selecting the Close case option (see Figure 20-14 ) When you close a case, Office 365 sends commands to the
underlying workloads to release all the holds for the case. It can take some time to process the hold release
commands by Exchange Online and SharePoint Online, which manage the underlying data sources. After the
workloads respond to confirm that they have removed the holds, Office 365 sets the case status is set to closed and
records the time and date of closure and the user who invoked the closure in the case properties. Later, if the case
records are not needed, you can remove the case from Office 365 after first removing any holds belonging to the case.
These holds are inactive, but they exist in case someone wants to reopen the case and reestablish the holds, so they
must be removed before you can remove the case.

Warning ! Closing a case could result in immediate data loss as background maintenance processes are then able to
remove the items that are no longer on hold. Closing a case is not something to do on a whim. If you make a mistake
and need to reactivate a case, the case manager can reopen the case. However, reopening a case does not reestablish
the holds that were previously in place and they will have to be recreated by going to the Holds section of the case,
selecting each hold, and then taking the Turn It On option in the action pane. The gap in time between removing
the original holds and reapplying new holds creates the potential that data will be removed from the sources during
this period. The exact amount of data that might be lost is unpredictable because it depends on whether the
background processing to remove data from sources has run and removed data.

GDPR Data Subject Requests

Article 15 of the GDPR grants a data subject (a person) the right to have a data controller (an Office 365 tenant)
provide them with a copy of their personal data. To help tenants respond to these requests, the GDPR dashboard (a
preview feature) in the Security and Compliance Center supports the creation and management of special eDiscovery
cases, called Data Subject Request (DSR) cases.

A DSR case is preconfigured to search for the personal data belonging to a defined individual. If the person does not
have a mailbox in the tenant, you input their SMTP address. Office 365 then creates an eDiscovery case to search the
person’s mailbox (if it exists), all public folders, and all SharePoint Online and OneDrive for Business sites for any
information relating to the data subject. Like any other eDiscovery case, you can create extra searches using different
criteria to gather more or different information that you think is relevant, or you can tweak the settings of the
preconfigured search to change the keywords, conditions, or locations. However, you cannot create an in-place hold
for a DSR like you can for a regular eDiscovery case.

As an example of how you might change the search criteria, the preconfigured search locations might not include all
the mailboxes that the data subject might appear in – group, shared, and user. You should check the locations to
ensure that everything relevant is found.

Things to remember about DSRs include:

The search locations are not dynamic. If new mailboxes or sites are added to the tenant after you create the DSR,
the search will not look in those locations.
A DSR can only search cloud locations. If you run a hybrid organization, you must run separate searches to find
relevant information in the on-premises locations. Exchange 2010 and later and SharePoint 2010 and later both
include the ability to run eDiscovery searches that can be tailored as DSRs.
GDPR says that DSRs must be responded to within a month of the request. It is therefore important to keep an
eye on the progress of DSRs and highlight instances when DSRs are delayed.
DSRs do not address the need to remove information about a data subject (the right to be forgotten defined in
article 17). However, the export reports generated by a DSR will tell you where data matches are found and allow
you to then check the individual items to decide whether items should be removed. Remember, not all data found
for a data subject needs to be removed as it is legally permissible to keep data under certain circumstances, such
as to comply with a legal obligation.

When you are happy that the searches return the information necessary to satisfy the data subject request, you can
export the information as normal and prepare it to give to the data subject. A further check is always needed to
exclude any information in the exported data that is unrelated to the data subject. For instance, if the name of the
data subject is common (like John Smith), it is likely that some of the matches returned by the searches do not refer to
the data subject. It is also possible that some of the information is commercially sensitive and needs review by the
business and potentially by legal advisors before it can be released.

Once the request is satisfied and the results have been released to the data subject, you can close the DSR. This does
not remove the case from Office 365 as the case remains in the system and can be reactivated if required. If you want
to remove a DSR case from Office 365, you can do so by running the Remove-ComplianceCase cmdlet in PowerShell.
For example:

[PS] C:\> Remove-ComplianceCase -Identity "James Smith DSR Case"

According to Microsoft’s Office 365 Data Subject Guide , 90% of an organization’s data stored in Office 365 is in
Word, Excel, PowerPoint, OneNote, and Outlook and is indexed and searchable. Some Office 365 data cannot be found
by content searches, so the information you can find through a DSR case is not necessarily all the personal
information belonging to a data subject existing in a tenant. For instance, tasks assigned to them in Planner,
conversations in Yammer, and Sway files are not covered, so extra effort is needed to review and retrieve this
information. The documents available in the Service Trust Portal covering Windows, Azure Active Directory, and
Dynamics 365 are helpful to ensure full coverage for data subject requests across all aspects of your Office 365
environment.

Protected Documents and Searches

Office 365 does not index the content of documents protected with a Azure Information Protection rights management
template. This means that a content search cannot find these documents based on their content and must rely instead
on metadata such as the document title, comments, or tags. Protected documents found in this way can be exported
like any other item found by a search, but they continue to be encrypted and cannot therefore be examined by an
investigator unless an arrangement is made to decrypt the files.

Chapter 24 contains some advice about how an Azure Information Management super-user can run PowerShell code
to find and decrypt protected files exported for a search. This is especially important for GDPR data subject requests
(and possibly for data erasure requests) because someone must check the content of documents to understand
whether they relate to a data subject.

Auditing of Search Activities

Administrators and eDiscovery managers access and view user content through content searches. This access is
necessary as otherwise you’d never be able to perform a compliance search and export its results. Because the access
is privileged, Office 365 captures audit records when content searches and eDiscovery cases are created, when
searches run, and when search results are previewed and exported. Having audit information to hand enables
organizations to ensure that they meet their requirements to protect confidential user information under regulations
such as GDPR.

You can review the audit records for search activities through the audit log search feature in the Security and
Compliance Center. This is a reasonable approach when you know the time and date when an activity occurred, and
you only need to review a small number of activities. If you want to process a lot of audit records, perhaps if you need
to verify that administrators are not abusing their access to user information, it is better to use PowerShell to
interrogate the audit log and extract the records you might be interested in analyzing. This example (see Chapter 21
for a more extension discussion of using audit log data) extracts records captured when the results of content
searches are exported, viewed, or previewed and outputs a CSV file. The CSV file can be opened and reviewed in
Excel to imported into Power BI.

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 31-May-2018 -EndDate 29-Aug-2018 -Operations "SearchExportDownloaded",
"SearchViewed", "ViewedSearchPreviewed" -ResultSize 1000)

If ($Records.Count -eq 0) {

Write-Host "No audit records for content search activities found." }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime

User = $AuditData.UserId

Action = $AuditData.Operation

Status = $AuditData.Status

Exchange = $AuditData.ExchangeLocations

SharePoint = $AuditData.SharePointLocations

Query = $AuditData.Query }

$Report += $ReportLine

}}

$Report | Export-Csv c:\temp\SearchAuditRecords.csv -NoTypeInformation

Office 365 captures audit records for many other content search and eDiscovery activities and it is easy to include
other activities in the PowerShell code shown above to export those records.

Advanced eDiscovery
Advanced eDiscovery is a feature included in the Office 365 E5 plan available as an add-in for the E3 plan. Although
the standard approach to eDiscovery (hold, search, and export) is suitable for investigations spanning up to 10,000
items, human examination of thousands of items retrieved by a search is a time-consuming and expensive exercise.
Advanced eDiscovery applies algorithms to refine very large sets of search data retrieved to make it easier for
investigators to find what they are looking for. Consider a situation where a search uncovers a hundred thousand
items. The choices are then to:

1. Refine the criteria for content searches to return the most precise set of discovered information and then review
all the found items to decide which are useful and which are not. This is an acceptable tactic for relatively
targeted searches where the desired material can be accurately described in terms of search criteria. For
instance, looking for evidence that a specific phrase was discussed in email sent between four known individuals.
2. Ingest the output of content searches, use analysis to parse the set of discovered files and home in on the
material needs close examination, followed by a review of sample items by expert investigators to find items that
are of relevance and those that are not. The items deemed to be of relevance are then fed through machine
learning algorithms to construct filters (think of them as very complex search queries) that are used to
interrogate the complete set of files to locate the desired information.

The first choice described above is what happens with the standard eDiscovery features available in Office 365. The
second is what Office 365 Advanced eDiscovery does using “Zoom” technology.

The cost of large-scale eDiscovery actions can be staggering. It is expensive enough to retrieve all the items necessary
to satisfy a discovery order handed down by a judge. It can be extraordinarily expensive to individually process each
piece of information that is provided as a response to a discovery order. The number of individual messages or
documents can easily mount into the low millions and, when very large companies are involved, quickly grow into tens
or hundreds of millions of items. The problem then becomes how to locate the proverbial needle in the haystack.

In the early days of eDiscovery, it was common to print off copies of email and documents for lawyers to review. This
process was tedious, paper-bound, and expensive. Lawyers are paid by the hour and lots of IT effort was necessary to
generate the material. However, the technique worked reasonably well then because a small volume of email or
electronic documents was involved, and the major focus was on paper documents and communications such as faxes
and telexes. Today, the situation has changed because paper is no longer the focus for business documentation and
the volume of items stored and available to be discovered has grown massively. If a discovery order turns up 50,000
items, you don’t want to incur the cost of having a lawyer or a legal associate checking each item.

Equivio enjoyed a well-deserved reputation for being able to analyze very large sets of documents and messages to
filter out the unimportant or irrelevant items to reduce the set to a manageable size. The collection input to Advanced
eDiscovery comes from content searches or eDiscovery cases. Only the results from a single search can be input to
Advanced eDiscovery; you can’t combine searches in a case and use those results as the input.

Essentially, the items found by these searches are exported as a container and imported into Advanced eDiscovery to
become available for analysis, with the goal to find items that can be excluded from further processing. In the analysis
shown in Figure 20-19 , the conclusion was that 36.3% of the email can be dropped because of “near duplicates” (ND)
or “email threading” (ET). In other words, their content is available in other items in the collection. Reducing the first
set of results in this manner makes the remaining processing more efficient and effective.

Extracts or samples can then be reviewed by skilled people who can recognize the kind of information that is needed.
When a skilled investigator examines a message or document, they do so by looking at the item in context in a way
that automated tools cannot. Even text that is marked as removed can be considered because the very fact of its
deletion might be relevant to an investigation. The review will gradually accumulate learning about the characteristics
of items relevant to the investigation by finding items of interest and culling or removing items that are not. Machine
learning techniques then analyze the characteristics of the items selected as relevant by the reviewers to build an
automatic classifier based on the accumulated knowledge that can be used to find the truly interesting and useful
information that exists within the entire collection.

Multiple reviewers can be involved in looking at the sample sets. When this happens, the potential always exists that
reviewers will differ in their understanding of what information is relevant to a discovery operation and what is not.
The machine learning algorithms are sensitive to this kind of disagreement between human reviewers (or
inconsistency in the results generated by a single reviewer) and highlight the problem if it occurs. The analysis of
different samples continues until Advanced eDiscovery believes that the classifier can locate the right information. At
this point, the analysis is said to be stable and the classifier can be applied in batch mode against all the items in the
collection.

Figure 20-19: The analysis of a container of Advanced eDiscovery results

When stability is reached, the investigators running the search exercise can decide just how exact they want to be in
terms of relevance. The closer to 100% investigators wish to search, the more items need to be reviewed and the
higher the cost. Sometimes this is what is needed, but it can also be OK to go ahead with a less precise search that
will turn up fewer items. Advanced eDiscovery helps in the decision process by including an adjustable slider to show
how many items fall into the category at a certain level of accuracy and what this means in terms of cost. Figure 20-20
(called the “Decide” screen because this is the point when those performing the analysis must decide how to proceed)
shows an example of a case where if 84.6% relevance is selected, the estimated cost to process the exported items is
$135.7K. Increasing the level of relevance further from this point (by adjusting the slider in the left-hand control) will
add to the number of items that come into scope. Apart from increasing the total volume of items to be reviewed, it’s
likely that a certain percentage of the set will prove to be irrelevant. These factors therefore increase the cost to
perhaps $700K if full coverage is needed.
Figure 20-20: Making the decision about how much data to export

After a decision is made as the scope of the relevant data, the items can be exported as individual files or as a secure
blob in Azure Blob Storage. Specialized tools can then be used to retrieve the data and perform the actual item-by-
item review and analysis.

Advanced eDiscovery is currently capable of ingesting data sets spanning up to a million items. The on-premises
version of Equivio is capable of handling far larger amounts (more than hundreds of millions) and the current limit is
set to tune and refine the cloud-based service. Microsoft expects that they will be able to lift the limit soon and at that
point, Advanced eDiscovery will be able to process as much data as is needed.

From a cost perspective, Advanced eDiscovery is available to any user with an E5 license or as an add-on, which costs
$8/month per user. Every user included in a search must be licensed, so some care must be taken to ensure that only
those users who are involved in eDiscovery actions that need this form of advanced analysis are licensed. The cost of
the licenses might be high but the cost of a botched or ineffectual response to a discovery action can be much higher.
That, and avoiding the potential cost involved in reviewing very large quantities of documents unearthed by a search
are the usual justification to deploy Advanced eDiscovery.

Advanced eDiscovery Licensing : While those who generate the input set for an Advanced eDiscovery search only
need Office 365 E3 licenses, every user included in the scope of an Advanced eDiscovery analysis needs an Office 365
E5 license. This might sound like Advanced eDiscovery is an expensive proposition, but the purpose of the content
search is to refine the set of content for Advanced eDiscovery to analyze. Refining the input set should mean that the
number of users covered by the set is reduced too, so the number of E5 licenses you need is probably less than you
think.

Using PowerShell with Content

Searches
Many Security and Compliance Center operations are quite complex, and it is usually best to execute eDiscovery and
search operations through the portal. The information about running searches with PowerShell offered here gives
some insight into what happens in the background so that you have a starting point for further investigation, if
necessary. Remember that to run these cmdlets, you must connect a PowerShell session to the Security and
Compliance Center endpoint. In addition, the account used must have the required Security and Compliance
permissions to perform whatever action you wish to take. If the account does not have the right permissions, not all
cmdlet parameters are available. For example, if the account does not hold the eDiscovery Manager role, you cannot
add the export action to a search.

Creating and Running a New Content Search

To create a new content search, run the New-ComplianceSearch cmdlet, followed by the Start-ComplianceSearch
cmdlet to start the search with the specified query to find items. For example, let’s assume that we are interested in
finding out whether any items exist in user mailboxes that have a specific phrase in their subject delivered in a certain
period. Perhaps these items belong to a potential phishing attack. Here is the command to create a simple content
search to look for items based on some subject text delivered in a certain date range.

[PS] C:\> New-ComplianceSearch -Name "Look for Phishing Items" -Description "A search to locate suspicious phishing items" -
PublicFolderLocation All -ExchangeLocation All

-ContentMatchQuery '(Subject: "Phishing") AND (Received:06/01/2016..04/05/2018)'

After creating the search, we use the Start-ComplianceSearch cmdlet to execute the query. If the search results are
older than seven days, you should run Start-ComplianceSearch to refresh the results.

[PS] C:\> Start-ComplianceSearch -Identity ‘Look for Phishing Items’

The Get-ComplianceSearch cmdlet fetches information about the search. For instance, this command checks the
status of the search:

[PS] C:\> (Get-ComplianceSearch -Identity 'Look for Phishing Items').Status

The search status is “InProgress” while Office 365 evaluates the search parameters and then performs the search.
The search status is set to “Completed” when search results are ready for review. The number of items returned by
the search is always interesting, so we can see this data with:

[PS] C:\> (Get-ComplianceSearch -Identity 'Look for Phishing Items').Items

Although content searches are much faster than their on-premises counterparts, a search across many locations can
take some time to complete. Inside a script, we might want to have a loop to check the search status and then
continue to some other processing once the search completes. Here is a simple loop that checks search status every
five seconds.

[PS] C:\> $Search = "Contoso Review of Patent Information for Project Alpha"

Start-ComplianceSearch -Identity $Search

do

Write-Host "Searching..."

Start-Sleep -s 5

$Test = (Get-ComplianceSearch -Identity $Search).Status

While ($Test -ne 'Completed')

Write-Host "All done. Items found:" (Get-ComplianceSearch -Identity $Search).Items

The full output of the Get-ComplianceSearch cmdlet has more detail than the search status and the number of found
items. For example, the information returned about this content search shows that two of the mailboxes scanned
returned zero items (edited output):

[PS] C:\> Get-ComplianceSearch –Identity 'Search for Phishing Items' |

Format-List

StatusMailRecipients : {}

LogLevel : Suppressed

IncludeUnindexedItems : True

ContentMatchQuery : Phishing(c:c)(date=2016-05-31T23:00:00.000Z..2018-04-05T10:53:37.887Z)

SearchType : EstimateSearch

Items : 2387

Size : 807344548

UnindexedItems : 2081

UnindexedSize : 1505932498

SuccessResults : {Location: James.Ryan@Office365itpros.com, Item count: 863, Total size: 419303454,

 Location: Kim.Akers@Office365itpros.com, Item count: 270, Total size: 84669642,

Location: Vasil.Michev@office365itpros.com, Item count: 237, Total size: 57118032,

Errors :

ErrorTags : {}

NumFailedSources : 0

Name : Look for Phishing Items

CreatedTime : 18/10/2016 16:32:12

LastModifiedTime : 05/04/2018 10:54:09

JobStartTime : 05/04/2018 10:54:11

JobEndTime : 05/04/2018 11:02:27

Description : A search to locate suspicious phishing items

CreatedBy : Administrator (Redmond and Associates)

RunBy : Tony Redmond

Status : Completed

ExchangeLocation : {All}

PublicFolderLocation : {All}

SharePointLocation :

OneDriveLocation :

ExchangeLocationExclusion :

PublicFolderLocationExclusion :

SharePointLocationExclusion :

OneDriveLocationExclusion :

Figure 20-21: Viewing an item found by a content search

Alternatively, you can examine the SuccessResults property to see the search results for each location:

[PS] C:\> Get-ComplianceSearch -Identity 'Contoso Review of Patent Information for Project Alpha'| Select -ExpandProperty
SuccessResults

Location: Kim.Akers@office365itpros.com, Item count: 34, Total size: 24165076

Our “Look for Phishing Items” search returned 2,387 items. The easiest way to confirm the items returned by a search
is through the GUI. Apart from anything else, this allows you to examine the items carefully to ensure that they are
items of interest and that the search is working as expected. Figure 20-21 shows that the search returned the items
we expected to find (the date range is right and the “Phishing” phrase is in the message subject). Now that we know
that the right information is available, we can decide what to do with it.

Searching SharePoint and OneDrive for Business

The only locations searched by the example query are user mailboxes and public folders. We can also add OneDrive
for Business and SharePoint sites to the set of search locations, but only if the target resources support the query. In
this case, if we try to add non-Exchange locations to the search by running the Set-ComplianceSearch cmdlet, we will
see a warning message because the search query is based on email-specific properties (Subject and Received date)
that are unsupported by OneDrive for Business and SharePoint.

To prove the point, here is how to create a content search that specifically targets SharePoint and OneDrive for
Business sites. In this case, we search for any item held in any site (the SharePointLocation parameter is set to “All”)
that includes some credit card information (the search uses the DLP sensitive data type to find these files – see
Chapter 22 for more information). We can further refine the search query by only looking for items created on or after
1 January 2015:

[PS] C:\> New-ComplianceSearch -Name "SPO and OD4B search for credit card data"

-Description "A search of SharePoint and OneDrive to locate files containing credit card data"

-SharePointLocation "All" -ContentMatchQuery '(SensitiveType:"Credit Card Number:2..") AND (created>=01/01/2015)'

Once again, after creating a new content search, we start it with the Start-ComplianceSearch cmdlet to retrieve some
results.

The example query searches all sites. If we try to refine the set of sites that we want to search, we can do so by
specifying the URL for each site as shown below. In this instance, we also add a couple of Exchange Online mailboxes
to the search locations. This will cause Office 365 to flag an error because once again the search properties we are
trying to use are not available for all locations.

[PS] C:\> Set-ComplianceSearch -Identity "SPO and OD4B search for credit card data" -SharePointLocation
"https://office365itpros.sharepoint.com/sites/O365ITPro/", "https://office365itpros.sharepoint.com/Projects/" -ExchangeLocation "Tony
Redmond", “Paul Cunningham"

PowerShell allows you to go ahead with the content search even though Office 365 flags the problem with search
properties. It is a mistake to do this as the net effect is usually to find everything in the search location that does not
support the properties used in the search query. In this example, because we use search properties specific to
SharePoint and OneDrive for Business but have included some Exchange mailboxes, the search returns every item in
those mailboxes, which is probably not what you intended.

See this blog post and this support article for tips about how to format KQL queries to interrogate SharePoint and
OneDrive for Business sources.

Content Search Actions

Behind the scenes, many of the options you take after a search returns some information use the New-
ComplianceSearchAction cmdlet to add a new action to the search. For example, when you export data from a search,
Office 365 runs the cmdlet to add an export action to the search. The following parameters are the most commonly
used:

SearchName : The name of the search to add an action.

NotifyEmail : The email address(es) of users to receive notification when an action is complete.
NotifyEmailCC : The email address(es) of CC recipients for notification messages.
Preview : Add an action to preview items found by the search.
Export : Add an action to export items found by the search to a PST or folder.
IncludeSharePointDocumentVersions : Input $True to include all versions of discovered SharePoint documents or
$False to include only the most recent version.
RetryOnError : Retry the search if an error occurs. Do not specify this parameter when using a content search
action to remove items from mailboxes as it will stop the action working.
Purge : Add an action to remove items found by the content search. Purging items is only supported by Exchange
Online.
PurgeType : Specify how the content search should remove items. Items can be soft-deleted by passing the
SoftDelete value. Soft-deleted messages are moved into the Deletions sub-folder of Recoverable Items and can be
recovered by the user for the deleted items retention period (default 14 days). In the future, you’ll be able to
specify HardDelete to force hard deletion. When items are hard deleted, Office 365 removes the item
 permanently and makes them irrecoverable. Even when a hard delete is specified, items that come under the
scope of a litigation or in-place hold are kept in the mailbox until the hold expires. However, the mailbox owner
can no longer access or recover the items.

You cannot add a new action to a search if the search is in progress or the search results are stale. For export
operations, a search is stale if it is more than seven days old. For destroy or purge operations, a search stays usable
until it is more than ten days old.

Look before you leap : No one wants to remove items from user mailboxes in error. Before committing a search to
remove items, it is best to check that the search completes successfully and that the right items are found. Use the
Preview function to verify that the items you expect to find are present and that no unanticipated items have turned
up. You might need to refine the search query several times before the search does exactly what you want it to do and
is ready to go ahead and remove the items.

Using a Content Search to Find and Remove Mailbox Items

The content search that we launched to find some suspicious items in user mailboxes turned up just 3 items. Although
a small number, we might want to go ahead and take some action against those items, such as removing them.
Administrators often use this action to eliminate:

Phishing messages before users have the chance to read them and activate the harmful links.
Messages that hold malicious attachments (viruses or other code).
Messages sent in error and hold information that the organization would like to withdraw (insofar as is possible).
You cannot retrieve messages that users send outside the organization or move to a PST.

In these circumstances, the normal approach is to create a new search covering the mailboxes that hold the problem
messages. You then run the search to ensure that it finds the correct items and to make sure that the search is
“fresh.” Because you are going to remove data from user mailboxes, it is essential that the search is as precise as
possible. Think of keyhole surgery rather than a massive incision. To help focus the search, you should capture the
essential characteristics of the targeted message and use these as the basis of the search. For instance:

An exact word or phrase that occurs in the message subject. For example: “Great Opportunity to Purchase” You
can use this phrase for the Subject property in the search query.
The sent date and time for the message, or even a date range. Use this date as the Received property in the
search query.
Include the SMTP address of the sender in the Participants property of the search query.

Combining several properties together will generate a more precise search than when you rely on a specific property,
such as the message subject. You can use the search preview facility to examine the details of the items found by the
search and verify that the search criteria are correct and working as expected.

You cannot assign a purge action to a search covering data held in SharePoint Online or OneDrive for Business.
Purges are only supported against Exchange mailboxes.

Using the Purge Action

When you are happy that the search finds correct results, you add the Purge action to the search. Note that you can
only do this if you are a member of the Security and Compliance Center Organization Management role group or the
account has the Search and Purge permission. For example, this command invokes the search called “Look for
Phishing Items” and instructs it to soft-delete any items matching the search criteria. Before Office 365 purges
anything, you must confirm that the action should go ahead.

[PS] C:\> New-ComplianceSearchAction -SearchName "Look for Phishing Items" -Purge -PurgeType SoftDelete

We can check the status of the purge operation by running the Get-ComplianceSearchAction cmdlet. The purge
reports its progression in percentage terms from zero to 100. The name of the action is formed by combining the
search name and a “_Purge” suffix. Given that the name of the search referenced above is “Look for Phishing Items, ”
the name of the action is “Look for Phishing Items_Purge. ” The following command retrieves the progress of the
purge action.

[PS] C:\> Get-ComplianceSearchAction –Identity “Look for Phishing Items_Purge” | Format-Table SearchName, JobStartTime, JobProgress,
Status –AutoSize

SearchName JobStartTime JobProgress Status

---------- ------------ ----------- ------

Look for Phishing Items_Purge 18/08/2015 19:04:38 100 Completed

To see how many items were purged, look at the Results property of the purge action.
The removal of items from a user mailbox through a search generates a SearchResultsPurged audit record in the
Office 365 audit log. The cmdlet does not report the locations from which it purged items (you can get this
information from the content search).

[PS] C:\> (Get-ComplianceSearchAction zinkfest_purge).Results

Purge Type: SoftDelete; Item count: 4; Total size 104080; Details: {Location: ; Item count: 2; Total size: 20443; Failed count: 0; ,

Location: ; Item count: 1; Total size: 54554; Failed count: 0; ,

Location: ; Item count: 1; Total size: 29083; Failed count: 0; }

Because the purge action is a soft-delete, Office 365 does not remove items from mailboxes. Items purged from user
mailboxes are in the Recoverable Items folder while those purged from group mailboxes go into the Deletions sub-
folder under Removeable Items. Exchange permanently removes purged items when the deleted item retention period
for the mailbox expires.

If you want to purge the messages permanently, you should use the Search-Mailbox cmdlet and pass the
DeleteContent parameter. The Search-Mailbox cmdlet and content searches both support queries expressed in KQL
syntax, so you can use the same queries with both types of searches. See Chapter 6 for information about the Search-
Mailbox cmdlet.

Of course, software being software, it is wise not to depend on a success status when a risk exists that a malicious
email might wreak havoc on mailboxes. After running the command to remove items, you should log into a mailbox
that you know held a problem item to check that the items are gone. The purge action only works for mailboxes.

Limit any possible damage : Removing data from user mailboxes is an operation that is fraught with error. The
potential of making a mistake and removing something that you should not is always there. To reduce the potential
for harm, purging based on a content search will only remove a maximum of 10 items from a mailbox. The purge
function is an “incident response” feature to remove small amounts of problematic data from user mailboxes.
Limiting removal to ten items should not be a problem unless malware floods mailboxes through multiple attacks
over a short period. Purges should use a precise, focused search to find data. The ideal situation is to find and
remove a single item. In addition, by limiting deletion to 10 items per mailbox, search and destroy operations
complete much faster (Microsoft reckons on being able to process 100,000 mailboxes in 30 minutes). Using soft
deletes rather than permanent removals allows users to recover items if the search removes them in error. If you
need to purge more than 10 items from mailbox, you must run a purge action multiple times, removing the purge
action and recreating it each time.

Exporting Search Data

When a content search is tuned to find the right data, the next step is usually to export the data to allow expert
examination and review. For email, the export can be to PSTs or as individual messages while exports from SharePoint
or OneDrive are as individual documents, including earlier versions if necessary.

You can use PowerShell to automate the export from Office 365 to Azure step of an export operation, but as
PowerShell cannot pass the security key to the Azure content, you cannot automate the final step to export from Azure
to PST or files. The example below shows how to start the export process for search results. In this case, we want to
export messages in PSTs (the format is “ExStream”) with messages for each mailbox stored in a separate PST. A loop
tracks the progress of the job. When complete, the data is ready in Azure to be downloaded.

[PS] C:\> New-ComplianceSearchAction -SearchName $Search -EnableDedupe $True -Export

-Format FxStream -ArchiveFormat PerUserPST -Scope BothIndexedAndUnIndexedItems

$ExportJob = $Search+"_Export"

Write-Host "Export started at" (Get-Date)

do

Start-Sleep -s 3

$ExportStatus = (Get-ComplianceSearchAction -Identity $ExportJob -Details)

Write-Host "Current status:" $ExportStatus.Status "% progress:" $ExportStatus.JobProgress

While ($ExportStatus.Status -ne 'Completed')

Write-Host ""

Write-Host "Export ended at" (Get-ComplianceSearchAction -Identity $ExportJob).JobEndTime

To download, select the search in the Security and Compliance Center, followed by Download exported results .
Office 365 reveals the export key to input into the download tool to create the PSTs and documents.

If you want to restart an export, run the Remove-ComplianceSearchAction cmdlet to remove the exported data from
Azure. Remember to append “_Export” to the end of the content search name to create the name of the export action.

[PS] C:\> Remove-ComplianceSearchAction -Identity "Office 365 Search #2_Export"

Using PowerShell to Manage

eDiscovery Cases
You can use PowerShell to create and manage eDiscovery cases. To create a case, use the New-ComplianceCase
cmdlet.

[PS] C:\> New-ComplianceCase -Name "Stock Trading Case Reference 2017-001"

The Get-ComplianceCase cmdlet returns details of an eDiscovery case. However, because an eDiscovery case is a
wrapper around content searches and holds, it does not return much information as no members or sources have yet
been added.

[PS] C:\> Get-ComplianceCase -Identity "Stock Trading Case Reference 2017-001"

RunspaceId : 883d2fb0-ef1b-484f-918d-bedf1930ef11

TenantId : b662313f-14fc-43a2-9a7a-d2e27f4f3478

Identity : 2fd9411c-3fd4-4293-a6f1-8fca699f51ed

Name : Stock Trading Case Reference 2017-001

Description :

CaseType : eDiscovery

Status : Active

ClosingStatus : Unknown

CreatedDateTime : 11/01/2017 18:13:32

LastModifiedDateTime : 11/01/2017 18:13:32

ClosedDateTime :

LastModifiedBy : Tony Redmond

ClosedBy :

ObjectState : New

We can now add some members to the case with the Update-ComplianceCaseMember cmdlet, specifying the user
principal name or display name for each member in a comma-separated string. For example:

[PS] C:\> Update-ComplianceCaseMember -Case "Stock Trading Case Reference 2017-001" -Member "Tony Redmond",
"Kim.Akers@Office365ITPros.com"

The Update-ComplianceCaseMember cmdlet replaces the existing member list for a case. If the user who runs the
cmdlet does not specify their name, the cmdlet includes it automatically. If you want to add a specific user to an
existing member list, use the Add-ComplianceCaseMember cmdlet.

Before the case is effective, we must set up the hold for the case. The hold includes a rule and a policy. We create the
rule with the New-CaseHoldRule cmdlet. The simplest form of rule has no query, in which case all content in the
search sources is covered. For instance:

[PS] C:\> New-CaseHoldRule -Policy "Stock Trading Case Reference 2017-001 -Name "Stock Trading Case Reference 2017-001

It’s more usual to include a query, defined in KQL syntax. This example sets up a condition to look for any item where
the words “stock” and “trading” occur near each other.
[PS] C:\> New-CaseHoldRule -Policy "Stock Trading Case Reference 2017-001"

-Name "Stock Trading Case Reference 2017-001" -ContentMatchQuery "Stock NEAR(10) Trading"

We now use the New-CaseHoldPolicy cmdlet to add the hold policy for the case. The hold policy defines the locations
that the case covers. Sources are

Exchange mailboxes
Public folders
SharePoint and OneDrive sites.

You can include either individual mailboxes or distribution groups. The membership of groups is expanded into
individual mailboxes and added to the policy. You can also include public folders by passing the “All” value to the
PublicFolderLocation parameter. SharePoint sites are specified with the site URL. For example:

[PS] C:\> New-CaseHoldPolicy -Case "Stock Trading Case Reference 2017-001" -Name "Stock Trading Case Reference 2017-001" -
ExchangeLocation "Nancy Anderson", "Sanjay Patel” -SharePointLocation https://office365itpros-
my.sharepoint.com/personal/ben_owens_office365itpros_com

-PublicFolderLocation All -Enabled $True -Comment “Search sources added programmatically"

To check that the hold exists, we can run this command:

[PS] C:\> Get-CaseHoldPolicy "Stock Trading Case Reference 2017-001"

Or, to see all holds in place for all eDiscovery cases:

[PS] C:\> Get-ComplianceCase -Identity | % {Get-CaseHoldPolicy -Case $_.Name}

After you create an eDiscovery case, you probably want to create one or more content searches and link the searches
to the case. We discussed the creation of content searches earlier in this chapter. This example is like those reviewed
then with the exception that we use the Case parameter to associate the search with the case.

[PS] C:\> New-ComplianceSearch -Name " Stock Trading Case Reference 2017-001 – Search 1" -Description "Search associated with Case
Reference 2017-001" -ContentMatchQuery "Stock NEAR(10) Trading" -Case "Stock Trading Case Reference 2017-001" -ExchangeLocation
"Nancy Anderson", "Sanjay Patel”

Once the search is available, you can start it in the normal manner by calling the Start-ComplianceSearch cmdlet.

The queries and search locations used in the content searches for eDiscovery cases do not have to match the hold rule
or policy. This is because a case might span multiple searches, each of which looks for different information in
different sources. The combination of all those results constitute the information for case investigators to review.

Removing Cases
The Security and Compliance Center UI does not offer the choice to remove an eDiscovery case. However, you can
remove a case by running the Remove-ComplianceCase cmdlet after removing any holds associated with the case. For
example, let’s assume that we want to remove the Contoso Alpha Case July 2015 case. To remove the holds, we issue
the command:

[PS] C:\> Get-CaseHoldPolicy -Case "Contoso Alpha Case July 2015" | Remove-CaseHoldPolicy

Office 365 publishes the command to remove the holds to the workloads. It takes some time for the workloads to
process the removals. When this process finishes, and no holds show up when you run the Get-CaseHoldPolicy cmdlet
for the case, you can run the Remove-ComplianceCase cmdlet:

[PS] C:\> Remove-ComplianceCase -Identity "Contoso Alpha Case July 2015"

GDPR DSRs
As noted earlier in this chapter, GDPR data subject requests are special forms of eDiscovery cases that are not listed
along with regular cases. However, you can see these cases in PowerShell by specifying DSR as the case type (regular
cases have a case type of “eDiscovery”). For example:

[PS] C:\> Get-ComplianceCase -CaseType DSR

Name Status CreatedDateTime

---- ------ ---------------

James Ryan DSR Active 30/04/2018 21:08:19

Tony DSR Active 01/05/2018 15:10:56

The same is done to access an individual DSR:

[PS] C:\> Get-ComplianceCase -CaseType DSR -Identity "James Ryan DSR" |Format-List

Identity : 16a90128-1e5b-4506-a6ca-160d0992fbaf

Name : James Ryan DSR

Description : James Ryan DSR

CaseType : DSR

Status : Active

ClosingStatus : Unknown

CreatedDateTime : 30/04/2018 21:08:19

LastModifiedDateTime : 30/04/2018 21:08:19

ClosedDateTime :

LastAccessTime : 01/01/0001 00:00:00

LastModifiedBy : Tony Redmond

The other cmdlets used with eDiscovery cases work as documented with DSRs.

One specific aspect of GDPR DSRs is that the regulations say that they should be processed within 30 days. We can
check the remaining time for active cases with some code like that shown below, which highlights any case that has
seven days or less left to respond:

[PS] C:\> $Cases = (Get-ComplianceCase -CaseType DSR | ? {$_.Status -eq "Active"})

ForEach ($Case in $Cases) {

$Days = (New-TimeSpan -Start $Case.CreatedDateTime -End (Get-Date)).Days

If ($Days -gt 23) {

$DaysLeft = (30 - $Days)

Write-Host "Warning!" $Case.Name "has only" $DaysLeft "days remaining to complete and respond..." }}

Adding eDiscovery Managers

Although it is usually best to manage the membership of sensitive role groups through the portal, you can add or
remove users from the eDiscovery role groups through PowerShell. To do this, connect to the Security and
Compliance Center endpoint and run the Add-RoleGroupMember (to add an eDiscovery Manager) or Add-
eDiscoveryCaseAdmin cmdlet (to add an eDiscovery administrator). For example, these cmdlets add a user to the
eDiscovery Managers role group and then check that the user is in the group:

[PS] C:\> Add-RoleGroupMember -Identity eDiscoveryManager -Member "James Abrahams"

[PS] C:\> Get-RoleGroupMember eDiscoveryManager

Only an eDiscovery administrator can add another user to the eDiscovery Administrators role group. If you hold this
status, you can run the cmdlets to add a user and check that the add worked as follows:

[PS] C:\> Add-eDiscoveryCaseAdmin -User "James Abrahams"

[PS] C:\> Get-eDiscoveryCaseAdmin

Reporting Holds for eDiscovery Cases

This is a modified version of a Microsoft script to report the holds that exist for eDiscovery cases. It is included here
as an example of how to navigate through eDiscovery cases to unpick and report on components.

[PS] C:\> $ClosedCases = 0

$ActiveHolds = 0

$Report = @()

Write-Host "Finding eDiscovery Cases"

$Cases = Get-ComplianceCase -ErrorAction SilentlyContinue

Write-Host $Cases.Count " cases found. Now extracting information."

ForEach ($Case in $Cases) {

Write-Host "Processing eDiscovery Case:" $Case.Name

$CaseMembers = (Get-ComplianceCaseMember -Case $Case.Name | Select Name, WindowsLiveId)

$Names = $Null

$First = $True

# Figure out display name for case managers - nicer than their email address

ForEach ($M in $CaseMembers) {

If ($First) {

$Names = $M.Name

$First = $False }

Else {

$Names = $Names, $M.Name -Join ", " } }

If ($Case.Status -eq "Closed") {

$ReportLine = [PSCustomObject][Ordered]@{

Case = $Case.Name

Status = $Case.Status

Created = $Case.CreatedDateTime

ClosedBy = $Case.ClosedBy

Closed = $Case.ClosedDateTime

Members = $Names }

$Report += $ReportLine

$ClosedCases++ }

ElseIf ($Case.Status = "Open") {

$HoldPolicies = (Get-CaseHoldPolicy -Case $Case.Name | % {Get-CaseHoldPolicy $_.Name -Case $_.CaseId -DistributionDetail})

ForEach ($Hold in $HoldPolicies) {

$HoldRule = Get-CaseHoldRule -Policy $Hold.Name

$ActiveHolds++

$i = 0 # Section of code to highlight inactive mailboxes that are under hold

$Mbxes = $Null

$CountMbx = 0

ForEach ($H in $Hold.ExchangeLocation) {

$Len = $Hold.ExchangeLocation[$i].DisplayName | Measure-Object -Character | Select -Expandproperty Characters

If ($Hold.ExchangeLocation[$i].DisplayName.Substring(0,1) -eq ".") {

$Mbx = ($Hold.ExchangeLocation[$i].DisplayName.Substring(1,$Len - 1)) + " (Inactive); "

$Mbxes = $Mbxes + $Mbx }

Else {

$Mbxes = $Mbxes + ($Hold.ExchangeLocation[$i].DisplayName) + "; " }

 $CountMbx++

$i++ }

# Write out the report line

$ReportLine = [PSCustomObject][Ordered]@{

Case = $Case.Name

Status = $Case.Status

Created = $Case.CreatedDateTime

ClosedBy = $Case.ClosedBy

Closed = $Case.ClosedDateTime

Members = $Names

Hold = $Hold.Name

HoldEnabled = $Hold.Enabled

HoldCreatedby = $Hold.CreatedBy

HoldModifiedby = $Hold.LastModifiedBy

Mailboxes = $Mbxes

MailboxCount = $CountMbx

SPOSites = ($Hold.SharePointLocation.Name) -Join ","

Query = $HoldRule.ContentMatchQuery

HoldCreated = $HoldRule.WhenCreatedUTC

HoldModifued = $HoldRule.WhenChangedUTC }

$Report += $ReportLine

}}

Write-Host "EDiscovery Cases found: " $Cases.Count

Write-Host "Active Cases: " ($Cases.Count - $ClosedCases)

Write-Host "Closed Cases: " $ClosedCases

Write-Host "Active Holds: " $ActiveHolds

$Report | Sort Status, Case | Format-Table Case, Status, Created, HoldCreated

The expected output is something like this, with cases organized by status and the holds described for active cases.
The result of the analysis is held in the $Report variable, so it is very possible to create different reports from the data
without changing the script.

EDiscovery Cases found: 9

Active Cases: 7

Closed Cases: 2

Active Holds: 11

Case Status Created HoldCreated

---- ------ ------- -----------

Board Minutes Closed 28/04/2017 19:14:20

Contoso Office 365 Investigation Closed 28/04/2017 19:14:20

Contoso Alpha Case July 2015 Open 28/04/2017 19:14:20 28/04/2017 19:12:32
Contoso Alpha Case July 2015 Open 28/04/2017 19:14:20 28/04/2017 19:12:27

Contoso Alpha Case July 2015 Open 28/04/2017 19:14:20 28/04/2017 19:12:35

Contoso Alpha Case July 2015 Open 28/04/2017 19:14:20 28/04/2017 19:12:31

Improper management transactions (LD-2018-002) Open 04/04/2018 18:03:40 04/04/2018 18:19:55

Inactive Exchange Mailboxes Open 28/04/2017 19:14:19 28/04/2017 19:12:27

Personal Harassment Investigation LD-2018-001 Open 04/04/2018 17:31:03 04/04/2018 17:38:33

Personal Harassment Investigation LD-2018-001 Open 04/04/2018 17:31:03 04/04/2018 17:44:42

Project Alpha Foxtrot Open 28/04/2017 19:14:20 28/04/2017 19:12:27

Stock Trading Case Reference 2017-001 Open 28/04/2017 19:14:20 28/04/2017 19:12:27

Stock Trading Case Reference 2017-001 Open 28/04/2017 19:14:20 15/02/2018 17:16:56

In-place Holds and Litigation Holds

An eDiscovery case can include one or more in-place holds to ensure that Office 365 keeps information even if
someone tries to remove or edit it. Each hold has a search query to find the information, which stays in the source
location until something happens, like a user trying to remove or edit the content. At this point, Office 365 steps in
and keeps a copy of the original content. Holds applied by Exchange and SharePoint or by Office 365 retention
policies and eDiscovery cases all keep content in-place. It is an efficient mechanism that avoids the need to duplicate
information unnecessarily.

Litigation or legal hold is a somewhat cruder but very effective mechanism available for Exchange Online mailboxes.
A litigation hold places the entire mailbox on hold for a set period or indefinitely. Again, all items stay in place but
every deletion results in Exchange keeping an item. As you can imagine, many of the items contained in mailboxes fall
into the banal category and are of no interest whatsoever to discovery actions. However, there are instances where it
is necessary to keep literally everything so that there is no chance of missing anything which might remotely be of
interest. A litigation hold keeps everything in a mailbox, as will an in-place hold that has no search criteria. You can
use litigation holds alongside in-place holds. All mailbox content is held when a mailbox is subject to both types of
hold.

Placing a mailbox on litigation hold is easy. First, select the mailbox owner’s account in the Office 365 Admin Center,
and go to the Mail Settings section. Scroll down to find litigation hold and click Edit . Move the Turn on litigation
hold slider to On to enable the hold and reveal to reveal the extended properties Figure 20-22 . Click Save to set the
hold.

In this example, 120 days is the retention period. Leave this field blank to put the mailbox on indefinite hold. The text
input to the note field will appear in the backstage section of Outlook, where its text informs the user that their
mailbox is on hold and how to find out more about what this status means, especially if local laws need companies to
inform users if they place a hold on mailboxes. The URL is to give a link to a web page with the compliance policy or
similar information about the circumstances that cause the company to put mailboxes on hold. No check occurs to
ensure that the web page exists and is reachable.
Figure 20-22: Putting a mailbox on litigation hold

Behind the scenes, several mailbox properties are updated:

LitigationHoldEnabled : Set to True.

LitigationHoldDate : Set to the time and date when the hold is placed on the mailbox. For example, 1-May-2018
19:45:50.
LitigationHoldOwner : Set to the account that placed the mailbox on hold.
LitigationHoldDuration : Set to the retention period. In this case, 90.00:00:00, or 90 days.
RetentionComment : The free-text note entered to inform the user that their mailbox is on hold. You don’t have
to enter this comment if you do not want to.
RetentionUrl : The URL pointing to a page holding information about the compliance policy.

You can also set litigation hold on by setting mailbox properties in EAC or with PowerShell. Here is an example of
putting a mailbox on litigation hold using PowerShell is:

[PS] C:\> Set-Mailbox –Identity "Ben Owens" –LitigationHoldEnabled $True

–LitigationHoldDate "1-May-2018 19:45:00" –LitigationHoldOwner "Administrator"

–LitigationHoldDuration 90.00:00:00 -RetentionComment "Your mailbox is on hold"

–RetentionUrl "http://www.contoso.com/compliance.htm"

A hold can be indefinite. If you want to pass a duration in days, Microsoft's documentation states that the limit is 2555
days (7 years), but the cmdlet is happy to accept 100,000 days (>273 years). I doubt anyone now alive will be
worrying if such a hold expires before its due date.

Ingesting Items for Review from Non-

Office 365 Sources
Companies that need to supervise employee communications need coverage over more than just email. Employees can
conduct business using a variety of consumer and business services including Twitter, Facebook, Bloomberg, HipChat,
Thomson Reuters, and BlackBerry messaging. To cater for the problem, Microsoft has signed deals with a set of third-
parties who specialize in the extraction of data from different communication systems to create connectors to extract
data from those systems and ingest the data into Office 365. The basic approach is:

A connector from a selected partner (like Actiance or ArchiveSocial) connects to the source data using whatever
API is available. The connection runs on a scheduled or ongoing basis to find and extract data of interest.
The connector uses Exchange Web Services to connect to an Azure endpoint for the ingestion of data into Office
365.
Data flows across the connector to either:
 User mailboxes, if a match exists between the identifier used by Office 365 (usually the User Principal Name) and
the identifier used by the source service. For instance, if the corporate Twitter account logs in as
TwitterService@tenant.com and an Office 365 account exists with the same identifier, a match exists, and the
data extracted from Twitter goes to that mailbox. Because you do not want someone to be able to access the
information brought into Office 365 via the connector, the items go into the Purges folder within Recoverable
Items. The items are indexed and discoverable but invisible to anyone who logs into the mailbox.
Connector mailboxes, set up explicitly as a target for data ingested into Office 365 through a connector. In this
case, the items go into the Inbox folder because someone usually needs to check the items and decide where to
keep the items over the long term.

As items flow into Office 365 through the connector, a separate set of agents watch the Exchange Web Services
traffic to apply supervision policies.

When the data reaches Office 365, you can apply the data governance policies that exist within the tenant. In other
words, you can assign retention policies to the mailboxes used by the connectors to ensure that you keep the ingested
content for the desired retention period.

Reporting what happens

Administrators often associate reporting and auditing with searching. After all, if you have found something that
needs further investigation, you probably want to figure out where it came from and who has been accessing the
content. All of which brings us to the topic of Office 365 Reporting and Auditing.
Chapter 21: Auditing and Reporting
Office 365
Tony Redmond

Office 365 gathers and stores audit information to track actions taken by users as they interact with Exchange Online
mailboxes, SharePoint Online sites, and other applications. After a slow start, most of the Office 365 applications and
Azure Active Directory now incorporate auditing capabilities to track how users and administrators create, update,
and remove data. This chapter reviews how the Office 365 auditing system gathers audit events from multiple sources
into one repository for administrators to query in multiple ways, including Office 365 Cloud App Security or third-
party products. Many different Office 365 sources generate audit events, including Exchange Online, so we review
administrative audit events and mailbox audit events to answer questions such as who did something to a mailbox or
who did something to information held in a mailbox. Finally, we look at the question of reporting Office 365 and
consider the value delivered by both the in-built reports and third-party solutions.

Office 365 Auditing

The ability to reliably audit user and administrative operations is an important part of any compliance strategy.
Although it is good to be able to enable auditing in an application like Exchange Online, that approach creates some
difficulties for a suite like Office 365. This is especially true when lines blur between applications as new functionality
such as Office 365 Groups, Teams, and Planner draw on components and data from different applications.

Figure 21-1: Office 365 Auditing Architecture

The background and history of some of the basic Office 365 workloads as on-premises applications means that each
has its own way to enable auditing, its own ideas about what audit data should be extracted and how that data should
be stored, and its own methods for administrative access to the audit data such as those described later for Exchange
Online mailbox and administrative auditing. The result is that you end up with a mess of inconsistencies and no way to
assure anyone that the audit data stays secure and immutable. The solution is the creation of a unified auditing
framework covering the whole of Office 365. The architecture of the Office 365 auditing framework is shown in Figure
21-1 .

The components of the Office 365 Audit architecture are:

Data feeds flow from the various Office 365 workloads. The different workloads running inside Office 365
have varying abilities to capture audit data. It is therefore important to normalize the data retrieved from across
Office 365 so that the audit events exist in a consistent format. As data is ingested into the auditing data mart,
Office 365 applies a common schema to events recorded in an individual workload, like Exchange Online, to
make sure that all the events include a consistent set of essential information such as the date and time for an
event such as the user identity, the client IP address, client type, the action taken, and the object accessed.
 Today, Office 365 captures events from Exchange Online, SharePoint Online (including OneDrive for Business),
and Azure Active Directory as well as applications like Yammer, Teams and Sway and administrative functions
such as eDiscovery operations. The schema accommodates the need for applications to capture information
specific to their activities through product-specific schemas built on top of the common schema. For example,
SharePoint Online supports product-specific schemas for file operations and sharing. See properties of audit log
entries for more information.
The Office 365 Audit data mart ingests the data feeds. Office 365 keeps audit data for tenants in the data
mart (otherwise known as the Office 365 audit log) for 90 days (accounts with Office 365 E3 licenses) or 365
days (accounts with Office 365 E5 licenses or accounts with Office 365 E3/Exchange Online Plan 1 licenses that
also have the Advanced Compliance add-on license). Office 365 Cloud App Security downloads audit data into its
own store and holds it there for 180 days. If you need to keep audit data for a longer period, third-party products
like Quadrotech Radar Security and Audit usually store Office 365 audit data in their own repositories for as long
as a customer is willing to pay for the storage. The data mart is immutable because administrators cannot
remove or change audit records. The data mart must ingest audit entries before records are accessible for
reporting purposes. This usually happens within an hour or so but can take up to 24 hours, depending on the
source workload, other activities running within the service, and system events such as software updates. For
this reason, the view of audit data is more precise when working with data a day or so after events occur than it
is in the short term.
Three access methods are available to consume the audit data. Because searches use an Exchange Online
PowerShell cmdlet, before you can view data in the audit log, your account must hold the Exchange View-Only
Audit Logs or Audit Logs role. These roles are assigned to the Exchange Online Compliance Management or
Organization Management role groups and can be assigned to other role groups as needed.

The Audit log search (described below) in the Search & Investigation section of the Security and Compliance
Center. The log search supports ad-hoc queries through a GUI with the ability to export discovered records to a
CSV file for later examination.
The Exchange Online Search-UnifiedAuditLog cmdlet can search the audit data from PowerShell.
The Office 365 Management Activity API is available to developers to build third-party tools for audit reporting or
analysis (or to extract data from Office 365 for injection into a different audit store – a GitHub example is
available ). The Office 365 Management Activity API is a REST-based web service. The API aggregates actions
and events drawn from the source workloads into tenant-specific content blobs, classified by their type and the
content they hold (such as Exchange Online, SharePoint Online, or Azure Active Directory). Tenants can use the
Management Activity API to build their own audit analysis tools in situations where PowerShell is unviable, as in
the case of large-scale tenants where the number of audit records generated daily is too high for PowerShell to
be able to process in any reasonable time.

Workloads that generate audit events : The intention is that audit events should be available and reportable from
all workloads active inside Office 365. It takes time to implement the concept. For now, the set of auditable sources
include:

Exchange Online administrative events.

Exchange Online mailbox events such as when messages are sent by mailbox delegates (only for user and shared
mailboxes, not for public folder mailboxes or group mailboxes). These events only appear for mailboxes that you
enable for auditing.
Office 365 Groups (created and updated by many different applications). These events are collected by the Azure
Active Directory workload.
SharePoint Online and OneDrive for Business file and folder events.
SharePoint Online and OneDrive for Business site administration events.
SharePoint Online and OneDrive for Business synchronization events.
Azure Active Directory events (like account logons and failed login attempts).
eDiscovery (events like creating or running a content search), including the use of cmdlets to perform searches
and other eDiscovery activities.
Sway.
Power BI (see instructions on how to enable auditing for Power BI ).
Teams, including the creation and removal of teams, channels, connectors, and tabs plus membership
management.
Flow, including the creation and deletion of flows and assignment of permissions.
Stream, including the creation, removal, and editing of videos, channel activities, uploading and sharing of
videos, and even when someone likes a video.
Dynamics 365.
Workplace Analytics.
Kaizala, including exports of chats and report summaries.

You can find extra information about Azure Active Directory audit events in the Azure portal. At present, the older
portal offers more options than the new . Not all Azure Active Directory events appear in the Office 365 audit mart.
For instance, if you need to investigate failed or suspicious logins, more data is available through the Azure portal .

Some consider that the gathering of audit data from multiple workloads gives Office 365 a Security Incident Event
Management (SIEM) or Cloud Access Security Broker (CASB) capability. However, Office 365 auditing lacks the
analysis and investigation features typically found in SIEM or CASB products, or Microsoft’s own Cloud App Security.
The big advantage Office 365 auditing holds over third-party products is the way that the various Office 365
workloads integrate auditing into their operations and generate audit events for the Office 365 audit log.
Enabling the Office 365 Audit Log in a Tenant
Office 365 collects information about user activity all the time. Some of this information is kept within a workload and
can also be transmitted to the Office 365 audit log. Other data is accessible through the Microsoft Graph. Before you
can search for user and administrator activity in the Office 365 audit log, you must enable searches. If you go to the
Search & Investigation section of the Security and Compliance Center and select Audit log search and see a link
to Start recording user and admin activity , you know that auditing has not been enabled for the tenant. Click the
link to create the audit log and turn on the ingestion process to bring audit data in from all the Office 365 workloads
that contribute to the log.

If you see the Turn on auditing button (Figure 21-2 ), you know that searches have been disabled for some reason.

Figure 21-2: Turning on the flow of events into the Office 365 audit log

You can also enable audit log searches with PowerShell by running the Set-AdminAuditLogConfig cmdlet from the
Exchange Online module:

[PS] C:\> Set-AdminAuditLogConfig -UnifiedAuditLogIngestionEnabled $True

The first time you run this command in a tenant, you create the audit log and enable the ability to search the log. You
only need to run the command thereafter if someone pauses audit log searches and you wish to reenable searches.
Pausing cannot be controlled through the Security and Compliance Center, but it can with PowerShell:

[PS] C:\> Set-AdminAuditLogConfig -UnifiedAuditLogIngestionEnabled $False

Disabling audit log searches also pauses the ingestion process. The workloads continue to capture events, but the
events don’t show up in the audit log until the pause is lifted. When that happens, the flow of events resumes into the
audit log. It might take some time before all the workloads learn about the pause lifting and the ingestion of events
into the log stabilizes. During this time, events might not be ingested into the audit log.

ISV Access to Audit Data

In the same way that it encourages third-party developers to exploit the reporting data available for tenants through
the Office 365 Reporting web service (described later), Microsoft encourages ISVs to use the Office 365 Management
Activity API to access the audit data in the Microsoft Graph and develop solutions that generate in-depth activity
reports, including suspicious or out-of-norm actions, data visualization and analysis to aid planning and oversight for
tenants, and to incorporate audit activity in operational dashboards. In short, these solutions will help tenants
understand who is accessing their content and whether their compliance framework is working.

Searching Office 365 Audit Events

When a tenant is enabled for auditing, events from the different workloads begin to flow into the Office 365 audit log.
The full set of auditable events is documented online . Because different methods of auditing are used by the Office
365 workloads, it should come as no surprise that events flow into the audit log at different intervals after the
workloads note details of the underlying actions. Although the exact periods will ebb and flow with demand and
volume, the following delays are normal:

Exchange Online : Audit events are available within 30 minutes. The events captured are those set by mailbox
and administrative audit settings for the tenant.
SharePoint Online and OneDrive for Business : Audit events are available within 30 minutes. Usually events
show up within 15 minutes. SharePoint on-premises servers capture audit events according to the auditing
settings for the site collection. SharePoint Online automatically generates these events and forwards them to the
Office 365 audit log.
Azure Active Directory : Audit events for logins are available within 30 minutes and other directory activities
within 24 hours.
All other workloads : Audit events are available within 24 hours.
Figure 21-3: Executing a search of the Office 365 audit log

Selecting Audit log search displays a form to collect parameters to create a search query to execute against the
audit data. Figure 21-3 shows the interface, including the drop-down list holding the auditable actions that you can
select to include in the search. The process is:

Select the audit events that you want to look for from the drop-down Activities list. You can combine events from
all the available Office 365 audit sources in a single search, subject to the 5000-item limit on the number of
entries returned by a search. You might find that some operations are not available in the picker but can be
included in a PowerShell search. This is because it can take time to update the picker after events join the set
supported for audit capture.
Define the date range to use (remembering that audit events are only available for a limited period). The default
is to search for the last seven days.
Select the user or users that performed the action you are looking for or leave the Users field blank to retrieve
data for all users. The drop-down list shows all active users in the tenant, including guest users.
If you are searching for audit events relating to SharePoint or OneDrive for Business, you can add supplementary
information for the search such as the name of a file, folder, or URL. If you want to search for a word in a
document title, give the full word rather than a partial substring. For example, if you want to search for events
related to a documented called “Reporting and Auditing”, a search will find it if you specify “Auditing,” but will
not for “Audit”.
Click Search .

The usual approach to searching is to start by looking for a specific activity that occurred in a certain time range and
then refine the search to focus in on a more precise set of events. You can refine the search by specifying the user
whose activity you are interested in or a specific file, folder, or site to which an item belongs. You can combine a wide
range of activities drawn from different Office 365 workloads into a single search. Each event that you include in a
search will probably increase the number of entries returned by Office 365 and might make it more difficult to find
the precise information that you want.

In this instance, the search results listed in the results pane are for “Checked in File” events, so the search will find
records associated with files checked into SharePoint Online document libraries. When the Security and Compliance
Center retrieves search results (Figure 21-4 ), some details are visible about the found events, including the name of
the checked-in file. To speed access to the audit data, the portal fetches and displays the most recent 150 entries. If
more matching entries exist, the portal retrieves them in batches of 150 records as you scroll down through the
results. In total, the results pane can display 5,000 events. If more than this amount meets the search criteria, the
results pane displays the most recent 5,000 events. Although you can examine the details of large data sets of audit
records on-screen, you might lose interest in the search by the time you scan through thousands of lines of audit data.
Figure 21-4: Viewing the results of a search of the Office 365 audit log

When an audit search returns results, you can use the Filter results button to filter the set by date, IP address, user,
activity, item, or detail. Office 365 applies the filter to the set of audit records downloaded from the server and
displays what it finds.

To capture audit data for further analysis, you can export the audit events to a CSV file. Click Export results to
begin. You can select to export the set of information shown on the screen (Save loaded results - up to 5,000 events)
or the complete set (Download all results ). However, the maximum number of audit events that can be exported at
one time is 50,000. If you end up downloading 50,000 events to a CSV file, there’s a fair chance that more matching
audit events exist that are not in the downloaded set, so it is a good idea to do another search to find events that
might be missing (perhaps by using a different date range) and then merge the two sets of results together.

Auditing and Secure Score: If you enable auditing for your tenant, your Microsoft Secure Score goes up by 15
points. However, enabling auditing is not enough. It’s more important to review what’s gathered in the audit log
regularly to understand what happens in the tenant and be able to detect when something out-of-the ordinary occurs.

User and System Events

It is also important to recognize that both user- and system-initiated operations generate audit events. For example, if
someone uses the Office Document Cache to hold offline copies of files in SharePoint Online or OneDrive for Business
sites, a background synchronization process checks the sites periodically to ensure that the local cache holds up-to-
date copies of the online files. The synchronization process shows up as a series of “Viewed File”, “Accessed File”, and
“Downloaded File” events and might lead the observer to conclude that the user has accessed many files over a brief
period.

Another example of an event that occurs as a by-product of user activity is the Accessed File event logged for the
JPEG file for a user’s profile photo. It is a fact that someone accessed the file, but it is unlikely that the fact will be
important in any sense except in circumstances when you absolutely must prove that someone looked at someone
else’s profile photo. Examples of system-initiated activity include events recorded for the user “app@sharepoint ”,
which relate to background processing of SharePoint and OneDrive sites while those generated by “NT
AUTHORITY\SYSTEM (Microsoft.Exchange.ServiceHost) ” belong to Exchange Online background jobs used to update
the organization configuration. For instance, each time a user updates a file in a SharePoint library, you should see a
matching app@sharepoint event as the crawler re-indexes the updated content. app@sharepoint also appears in
“Added user or group to SharePoint group” events when users join the membership of an Office 365 group and that
change replicates to the SharePoint group for the team site.

For these reasons, you should take care to restrict the extraction of events to a manageable quantity by using a filter
such as a limited time or events for selected users. In addition, you should ask yourself why an event might show up
rather than jumping to any conclusions. Microsoft recognizes that the amount of audit data generated by a search can
be excessive occasionally and that a filter to separate user-started from system-initiated events might be a sensible
future enhancement.


IP addresses in audit records : If you look up the IP addresses reported in audit logs, you might find some that
seem to be in unexpected places. The IP address might be for a home router connected to an ISP or it might be an IP
address managed by Microsoft and assigned to one of their datacenters. For this reason, products that use IP-based
geolocation sometimes report that a connection comes from places that you know the user has never been. For
instance, occasionally an audit record pops up to say that I accessed SharePoint Online from Helsinki. Much as I like
the Finnish capital, I have not been there for at least 15 years. However, one of Microsoft’s EMEA datacenters is
outside Helsinki and that is where the IP address originated.

Examining an Audit Record

You can select an item listed in the audit search results to view its full detail. Figure 21-5 shows an example of an
audit record for a document updated in a SharePoint Online document library. The information exposed here is exactly
what is in the audit log. We will get to the details of how to parse audit data with PowerShell soon to consider the
purpose of each of these fields in the audit record. You can see more information about the audit event by clicking the
More Information link.

Figure 21-5: Viewing details of an audit record

To export audit data, Office 365 creates a CSV file with a name starting with “AuditLog” to hold the information. You
can open the file (Figure 21-6 ) or save it for further processing.

Figure 21-6: Audit data exported as a CSV file and opened with Excel

The contents of the CSV are raw in that the AuditData column that holds the information about an activity record is
not formatted as nicely as it is when you see it in the query form or when you view an individual audit record on
screen. In fact, the details about the action taken by a user to cause the generation of the audit entry is in JavaScript
Object Notation (JSON) format, so further processing is needed to reformat the data for easier reading.

Chatty Applications : Some of the Office 365 applications generate more audit events than the others. While
Exchange Online can generate many events daily, if you use mailbox auditing, SharePoint Online usually generates
most events per user operation. Updating a document can generate five or six events – or even more if people
synchronize the document library to their PCs with OneDrive for Business. On the other hand, Teams does not
generate many audit events and Planner does not support auditing.

Viewing User Activity

The Home section of the Security and Compliance Center allows you to search for individual users. This is not just a
matter of checking that a user exists in the directory. Instead, if you select a user, you see a detail card with two tabs.
The Summary tab shows the user’s status. First, you can see whether the user holds any administrative role for the
Security and Compliance Centre. Second, you see an overview of how tenant policies and function cover the
information belonging to the user:

Archive : Has the user an archive mailbox.

Retention : Does the user’s account come within the scope of Office 365 retention policies.
Data loss prevention : Does the user’s account come within the scope of Office 365 DLP policies (not Exchange
transport rule DLP checking).
Mobile management : Are mobile devices linked to the user account managed with InTune.

The Activity tab displays entries from the Office 365 audit log recoding recent activities of the user (Figure 21-7 ). If
something strange appears, you can go to the audit log to perform a full search or create an activity alert (explained
later in this chapter) to flag when a certain action happens.

Figure 21-7: Audit events describing user activity

The Home page for the Security and Compliance Center also flags selected activities and offers to search the audit log
for those records. For example, you might see a prompt to check which users changed their passwords recently.
Clicking the link starts an audit log search for the “Changed User Password” event.

Searching Office 365 Audit Data with PowerShell

The Search-UnifiedAuditLog cmdlet accesses Office 365 audit data from PowerShell. Like many of the cmdlets used
for search operations, Search-UnifiedAuditLog needs a little work before it produces some useful results. By default,
Office 365 returns 100 records unless you specify otherwise by using the ResultSize parameter, which can request a
set of up to 5,000 records. If you need to deal with very large numbers of audit entries, you can use the
SessionCommand parameter and pass the ReturnLargeSet value, which allows you to retrieve up to 50,000 records.
However, that data is unsorted and might have duplicates.

When using the cmdlet to retrieve large numbers of audit entries, it is best to use the SessionCommand parameter
and the SessionId parameter. SessionId is an optional string parameter that you use as an identifier for a command
run several times to receive paged data. Using SessionId , you can run the same command multiple times to receive
blocks of audit data until you reach the end of the matching records. Each block holds the number of records specified
in the ResultSize parameter. You can use the ReturnNextPreviewPage value to have Office 365 return the data sorted
in descending date order (newest record first) and remove any duplicates.

For example, to discover who changed the file in the document library used for this book over a certain period, we can
issue a command like the one shown below. Although it is not obvious here, the AuditData property holds the
interesting data. In this case, because we know the period that we’re interested in, the full date and time is specified
to delimit the start and end of the search. You can pass a date on its own, in which case Office 365 uses 00:00 as the
time to begin (or end) the search. If you use a date that’s more than 90 days in the past, Search-UnifiedAuditLog
returns an error and tells you the first possible date you can use.

[PS] C:\> Search-UnifiedAuditLog –StartDate '01-July-2017 09:00' –EndDate '20-Jul-2017 17:00'

–ObjectIds 'Ch 20' –Operations FileModified -SessionCommand ReturnNextPreviewPage -ResultSize 1000 |

Format-Table UserIds, CreationDate, AuditData

UserIds CreationDate AuditData

------- ------------ ---------

tony.redmond@office365itpros.com 13/07/2016 14:35:44 {"CreationTime":"2016-07-13T14:35:44"

If you examine the AuditData property for an audit record, you will see that it consists of a set of multiple attribute-
value pairs separated by a comma (in JSON format). Up to 3,060 characters can be stored in the property – if more
audit data is available, it is truncated. To format the audit data and make it easier to follow, specify the Formatted
parameter with the Search-UnifiedAuditLog cmdlet.

[PS] C:\> Search-UnifiedAuditLog –StartDate '01-May-2018 09:00' –EndDate '20-Jun-2018 17:00'

–ObjectIds 'Ch 20' –Operations FileModified -Formatted -SessionCommand ReturnNextPreviewPage

RecordType : SharePointFileOperation

CreationDate : 13/06/2018 14:35:44

UserIds : tony.redmond@office365itpros.com

Operations : FileModified

AuditData : {

"CreationTime": "201 8-06-13T14:35:44",

"Id": "74746ae0-2fee-4399-18ce-08d3ab2af8fb",

"Operation": "FileModified",

"OrganizationId": "b662313f-14fc-43a2-9a7a-d2e27f4f3478",

"RecordType": "SharePointFileOperation",

"UserKey": "i:0h.f|membership|1003bffd805c87b0@live.com",

"UserType": "Regular",

"Version": 1,

"Workload": "SharePoint",

"ClientIP": "83.197.88.148",

"ObjectId": "https:// office365itpros.sharepoint.com/sites/O365ExchPro/Shared Documents/Fourth Edition Files/Ch 20 -

Reporting and Auditing.docx",
 "UserId": "tony.redmond@redmondassociates.org",

"EventSource": "SharePoint",

"ItemType": "File",

"ListItemUniqueId": "d9189cf3-8c85-4ae9-9c21-c94dfe76c988",

"Site": "acfe74d8-edfb-436d-924b-e018666605ee",

"UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79
Safari/537.36 Edge/14.14385",

"WebId": "a2aba197-5f1d-4864-a2c7-4daf0ff6379b",

"SourceFileExtension": "docx",

"Site Url": "https://office365itpros.sharepoint.com/sites/O365ExchPro/",

"SourceFileName": "Ch 20 – Audit and Reporting.docx",

"SourceRelativeUrl": "Shared Documents/ Fourth Edition Files"

ResultIndex : 1

ResultCount : 125

The audit data is now more legible, and we can see many items of interest, including:

RecordType : The workload that generated the record. Values include:

AzureActiveDIrectory: For example, add a member to a group.

MicrosoftTeams: For example, a user logs onto Teams.
ExchangeAdmin: For example, an update of mailbox properties.
SharePointFileOperation: For example, a client downloads a file.

CreationTime : The date and time in UTC format when the user performed the activity. If your time zone isn’t
UTC, you need to adjust this value to get to a local time.
Operation : In this example, FileViewed is a SharePoint Online operation logged when someone accesses an
item in a document library. You can see examples of other operations in the drop-down menu for the search form
(shown in Figure 21-3 ). The PowerShell search examples described below look for many different events.

OrganizationId : Is a unique GUID for the Office 365 tenant.

UserKey : The identity (in this case, achieved through membership of an Office 365 group) used to gain access
to the item.
Workload : The name of the application workload that logged the event. In this case, it is SharePoint Online.
Other values are Exchange Online, OneDrive for Business, and Azure Active Directory.
ClientIP : The IP (V4 or V6) address of the client workstation where the action originated is logged.
ObjectID : The full path to the object is logged. We can see that this resolves to a document in a SharePoint
Online document library.
UserID : The Office 365 identifier for the account that caused the action to occur.
UserAgent : The client used to invoke the action. In this instance, a user uploaded a file to the document library.
SourceFileName : The name of the file.
UserType : The type of user that performed the action. “0” indicates a normal user; “1” indicates an action taken
by an administrator, while “2” means that the action was taken by a Microsoft datacenter administrator or
datacenter system account. Because we used the Formatted switch for the Search-UnifiedAuditLog cmdlet, Office
365 translates the numeric value 0 to “Regular.”
EventSource : Only SharePoint Online uses this field. It is either SharePoint or ObjectModel.

The ResultIndex and ResultCount properties are interesting if you need to keep track of a large record set.
ResultIndex tells us the record number within the set returned. ResultCount tells us the total number of records
returned. We can therefore say that the record shown is the first of 125 returned in the set. With the increasing
number of workloads generating audit data for the Office 365 audit data mart, it is likely that a casual search will
return hundreds if not thousands of records. It is therefore important to be as specific as possible with search
parameters to restrict the number of records returned.

A common need to search the audit log is when you need to focus on a certain operation performed against a specific
object. In this example, we are looking for events logged when someone accesses and updates a Word document
stored in a SharePoint Online document library. Note that this is just the beginning of an investigation because the
same document name might exist in several libraries (think of the infamous “test.doc”) and you would need to check
each audit entry to understand whether it belongs to the document you need. But this is enough to start a search to
see if any entries exist.

[PS] C:\> Search-UnifiedAuditLog –StartDate 1-May-2018 –EndDate 1-Jul-2018

–RecordType SharePointFileOperation –ObjectIds "Ch 2 – Making the cloud decision.docx" -Operations “FileModified” | Format-Table
UserIds, CreationDate, Operations –AutoSize

UserIds CreationDate Operations

------- ------------ ----------

tony.redmond@office365itpros.com 07/04/2017 21:28:09 FileModified

tony.redmond@office365itpros.com 03/04/2017 15:20:32 FileModified

The record type shows that we are interested in SharePoint Online data. The other valid record types are listed here
and the schema used for the various types of audit records is documented here . It takes time for administrators to
become accustomed to the information contained in Office 365 audit records and to figure out how best to use this
data when confronted with questions such as “who interacted with a document” or “who created new documents in
this period.” Experience with the PowerShell cmdlets and some trial and error soon shows that the audit log is a
surprisingly useful source of information.

Practical Examples of Using Audit Information

As discussed above, audit records hold a wealth of information in the AuditData property. To make the information
more approachable, we must break out the important pieces from the JSON-formatted data. Some trial and error is
necessary to figure out what information interests you from the audit data. The guide to detailed properties in audit
log records is helpful in this respect.

Office 365 Group Creation

For example, let’s assume that we want to discover who created Office 365 groups, including the workloads used to
create the groups. The relevant event to look for is “Add Group.” In this example, we search for groups created
between two dates and then process each record to extract information about who created each group, the group
name, and the workload used. You can see how we use the ConvertFrom-JSON cmdlet to convert the JSON data into
an array that we then query for different fields.

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 10-Jul-2018 -EndDate 10-Sep-2018 -Operations "Add Group" -ResultSize 1000)

If ($Records.Count -eq 0) {

Write-Host "No group creation records found." }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime

User = $AuditData.UserId

Action = $AuditData.Operation

Status = $AuditData.Status

Workload = $AuditData.Actor[2].Id

GroupId = $AuditData.Target[0].Id

GroupName = $AuditData.Target[1].Id }

$Report += $ReportLine

}}

$Report | Select Timestamp, Workload, User, GroupName

Our report should look something like the example below. In this case, we see details of the creation of six groups
using a variety of workloads from Yammer to Planner (ProjectWorkManagement ) to Teams. The Microsoft.Exchange
workload means that the group was created by an Exchange client such as OWA. If you see a workload named with a
value like “12128f48-ec9e-42f0-b203-ea49fb6af367,” it is the Teams PowerShell module.
TimeStamp Workload User GroupName

--------- -------- ---- ---------

2018-04-25T08:58:38 ProjectWorkManagement James.Ryan@office365itpros.com O365Grp-James Plan

2018-03-03T22:54:00 Microsoft Teams Services Kim.Akers@office365itpros.com O365Grp-Acquisitio

2018-03-01T11:25:19 Microsoft.SharePoint Joe.Richards@office365itpros.com Nice Airport Watch

2018-02-17T14:30:57 Microsoft.Exchange Brian.Weakliam@office365itpros.com Brian's Nelson Gro

2018-02-17T14:13:57 Microsoft.Exchange Vasil.Michev@office365itpros.com Sandboxes

2018-02-17T14:07:37 Microsoft Teams Services James.Abrahams@office365itpros.com O365Grp-Potholers

2018-02-15T15:08:28 Microsoft.YammerEnterprise marc.vigneau@office365itpros.com CEO Working Group

All the examples in this section generate an ordered hash table called $Report. If you end up with many entries to
review or you want to format the data into a nice-looking report, you might like to export the hash table to a CSV file.
This is easy to do with the simple command:

[PS] C:\> $Report | Export-CSV c:\temp\Report.csv -NoTypeInformation

We can then open the CSV file with Excel to format the data as desired or load the CSV file into a tool like Power BI to
generate graphs, views, and reports about the audit data.

Truncated Records : In July 2018, Microsoft changed the ingestion process for Azure Active Directory events.
Unfortunately, due to a programming error, truncated data is written into the events in the Office 365 audit log. At
the time of writing, Microsoft is investigating the problem. The actual data is available in Office 365 Cloud App
Security, the Azure audit log, or in third-party systems, if those systems use the Graph API to fetch the audit data
instead of relying on the audit log. The code shown above does not work as it should and will not until Microsoft fixes
the underlying problem.

User Sign-ins
In another example of how to use the same technique to interpret audit events, here’s what you might do to report
user sign ins to Office 365 workloads. To make things more complicated, we use two different events, one from Azure
Active Directory (to handle most workloads) and the other from Teams. Because we can expect to retrieve many audit
records for these operations, we also pass the ReturnLargeSet value to the SessionCommand parameter and specify
that we will accept up to 5,000 records.

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 15-Feb-2018 -EndDate 15-Apr-2018 -Operations UserLoggedIn,
TeamsSessionStarted -SessionCommand ReturnLargeSet -ResultSize 5000)

If ($Records.Count -eq 0) {

Write-Host "No Audit records found for user logons." }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

If ($AuditData.Workload -eq "MicrosoftTeams")

{ $Client = $Auditdata.ObjectId

$Status = "OK"}

Else

{ $Client = $AuditData.ExtendedProperties[0].Value

$Status = $AuditData.ExtendedProperties[3].Value }

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime
 User = $AuditData.UserId

Action = $AuditData.Operation

Client = $Client

IpAddress = $AuditData.ActorIpAddress

Status = $Status

Workload = $AuditData.Workload }

$Report += $ReportLine }

Failed User Sign-ins

Understanding failed user login events is also interesting because these events might be caused by a hacker trying to
penetrate your tenant.

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 15-Feb-2018 -EndDate 15-Apr-2018 -Operations UserLoginFailed -SessionCommand
ReturnLargeSet -ResultSize 5000)

If ($Records.Count -eq 0) {

Write-Host "No audit records found for failed logins." }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime

User = $AuditData.UserId

Action = $AuditData.Operation

Status = $AuditData.ResultStatus

IpAddress = $AuditData.ActorIpAddress

Error = $AuditData.LogonError

UserAgent = $AuditData.ExtendedProperties.value[0] }

$Report += $ReportLine }

$Report | Sort User, Timestamp | Select Timestamp, User, IpAddress, UserAgent

Who Changed a File

Here’s how to search the audit log for records for a specific document name, perhaps to answer the eternal question
of who last changed a file. The search looks for audit events created when someone edits a file (a FileModified
operation) or creates a file (a FileUploaded operation). If the AutoSave feature is enabled, you will probably see
multiple edit records for a file over a short period. As you can see, because we deal with audit records generated by
SharePoint, different information is in the AuditData field.

[PS] C:\> $FileName = (Read-Host "Enter file name to search")

$Start = (Get-Date).AddDays(-15)

$End = (Get-Date)

$Records = (Search-UnifiedAuditLog -StartDate $Start -EndDate $End -Operations FileModified, FileUploaded -ObjectIds $FileName -
ResultSize 1000)

If ($Records.Count -eq 0) {
 Write-Host "No audit records found for file names beginning with" $FileName }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime

User = $AuditData.UserId

Action = $AuditData.Operation

SiteUrl = $AuditData.SiteUrl

Site = $AuditData.SourceRelativeUrl

File = $AuditData.SourceFileName

IpAddress = $AuditData.ClientIP

App = $AuditData.UserAgent }

$Report += $ReportLine }

To list the audit records in your preferred order, use a command like this:

[PS] C:\> $Report | Sort File | Select Timestamp, User, File, SiteUrl

Guest User Activity

Here’s how to review the activity of guest users in your tenant. Applications like Office 365 Groups, Planner, and
Teams create guest accounts in your tenant directory when external users receive invitations to join groups or teams.
Over time, some of these accounts might become obsolete and candidates for removal. Obviously, you only want to
remove unused accounts, so this code scans the audit log and the message tracking logs to look for evidence of guest
user activity over the last 90 days.

[PS] C:\> $Guests = (Get-AzureADUser -Filter "UserType eq 'Guest'" -All $True| Select Displayname, Mail,
RefreshTokensValidFromDateTime | Sort RefreshTokensValidFromDateTime)

Write-Host $Guests.Count "guest accounts found. Checking last connections..."

$StartDate = (Get-Date).AddDays(-90) #For audit log

$StartDate2 = (Get-Date).AddDays(-10) #For message trace

$EndDate = (Get-Date)

$Active = 0

$EmailActive = 0

$Inactive = 0

$TeamsSpo = 0

ForEach ($G in $Guests) {

Write-Host "Checking" $G.DisplayName

$Recs = $Nul l

$UserId = $G.Mail

# Handle account whose guest invitation is not redeemed

If ($Userid -eq $Null) {$UserId = "NullString"}

$Recs = (Search-UnifiedAuditLog -UserIds $UserId -Operations UserLoggedIn, TeamsSessionStarted -StartDate $StartDate -EndDate
$EndDate)

If ($Recs -eq $Null) {

Write-Host "No connections found in the last 90 days for" $G.DisplayName " account created on"
$G.RefreshTokensValidFromDateTime -Foregroundcolor Red

# Check email tracking logs because guests might receive email from Groups. Account must be fully formed for the check

If ($UserId -ne "NullString") {

$EmailRecs = (Get-MessageTrace –StartDate $StartDate2 –EndDate $EndDate -Recipient $G.Mail)

If ($EmailRecs.Count -gt 0) {

Write-Host "Email traffic found for " $G.DisplayName "at" $EmailRecs[0].Received -foregroundcolor Yellow

$Active++

$EmailActive++ }}

Elseif ($Recs[0].CreationDate -ne $Null) {

Write-Host "Last connection for" $G.DisplayName "on" $Recs[0].CreationDate "as" $Recs[0].Operations -Foregroundcolor Green

$Active++

$TeamsSpo++ }

Write-Host ""

Write-Host "Statistics"

Write-Host "----------"

Write-Host "Guest Accounts " $Guests.Count

Write-Host "Active Guests " $Active

Write-Host "Active on Teams and SPO " $TeamsSPO

Write-Host "Active on Email " $EmailActive

Write-Host "InActive Guests " ($Guests.Count - $Active)

Guest User Access to Documents

Our next example shows how to search the audit log for File Accessed events for guest users. The idea is to
understand precisely what files in document libraries guest users have opened. We could expand the search to include
File Modified events if you want to know what files guest users update, but this search is enough to prove the point.

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 1-Apr-2018 -EndDate 20-Jun-2018 -Operations FileAccessed -ResultSize 2000 | ?
{$_.UserIds -Like "*#EXT#*" })

If ($Records.Count -eq 0) {

Write-Host "No SharePoint file access records found for guest users." }

Else {

Write-Host "Processing" $Records.Count " SharePoint file access audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

If ($AuditData.SourceFileName -NotLike "*aspx*" -And $AuditData.SourceFileName -NotLike "*jpg*" ) {

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime
 User = $Rec.UserIds

Action = $AuditData.Operation

Workload = $AuditData.Workload

URL = $AuditData.SiteUrl

Document = $AuditData.SourceFileName }

$Report += $ReportLine }

}}

After retrieving the data, here’s how we might sort it by user to discover what documents individual guest users have
accessed.

[PS] C:\> $Report | Sort User, Document | Format-Table TimeStamp, User, Document -AutoSize

TimeStamp User Document

-------- ---- --------

2018-03-22T22:54:39 john_contoso.com#ext#@Tenant.onmicrosoft.com Summit 2018.one

2018-03-29T11:58:41 mary_contoso.com#ext#@tenant.onmicrosoft.com Summit 2016.one

2018-03-29T11:28:41 mary_contoso.com#ext#@tenant.onmicrosoft.com Updates.docx

2018-03-15T08:37:32 terry_contoso.com#ext#@tenant.onmicrosoft.com Amazon Blurb.docx

The extracted audit data can be viewed in different ways. For example, to find out how many accesses occurred to
each document, you could do this:

[PS] C:\> $GroupData = $Report | Group-Object -Property Document

[PS] C:\> $GroupData | Sort Count -Desc | Select Name, Count

Name Count

---- -----

Ch 5 - SharePoint Online and OneDrive For Business - Final.docx 17

Project Ambush.docx 11

Interesting data.docx 1 0

Financial Arrangements 2019.docx 9

Ch 8 - Clients.docx 8

Ch 3 - Office 365 Basic Workloads.docx 6

Budget 2019.docx 1

Who Used the SendAs Permission?

The Office 365 audit log ingests mailbox audit records from Exchange Online. In the past, you might have used the
Search-MailboxAuditLog cmdlet to look for audit records. For example, a common question is who used the SendAs
permission to send a message impersonating a shared or user mailbox. We can find out by running a command like
this:

[PS] C:\> Search-MailboxAuditLog -Identity "Customer Complaints" -LogonTypes Delegate -StartDate "1-Nov-2018 12:00" -EndDate "3-Nov-
2018 17:00" -ShowDetails | ? {$_.Operation -eq "SendAs"} | Select LogonUserDisplayName, LastAccessed

LogonUserDisplayName LastAccessed

-------------------- ------------

James Ryan 2 Nov 2018 12:13:35

James Ryan 2 Nov 2018 11:57:33

You can still use the Search-MailboxAuditLog cmdlet to search for mailbox events, but as those events end up in the
Office 365 audit log it might be more convenient to search there. Here’s how to look through the log for information
about delegates sending messages for another user with the SendAs permission:

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 1-Nov-2018 -EndDate 2-Nov-2018 -Operations "SendAs" -ResultSize 1000)

If ($Records.Count -eq 0) {

Write-Host "No Send As records found." }

Else {

Write-Host "Processing" $Records.Count "audit records..."

$Report = @()

ForEach ($Rec in $Records) {

$AuditData = ConvertFrom-Json $Rec.Auditdata

$ReportLine = [PSCustomObject][Ordered]@{

TimeStamp = $AuditData.CreationTime

User = $AuditData.UserId

Action = $AuditData.Operation

Status = $AuditData.ResultStatus

SentBy = $AuditData.MailboxOwnerUPN

SendAs = $AuditData.SendAsUserSmtp

Item = $AuditData.Item.Subject }

$Report += $ReportLine

}}

$Report | Select Timestamp, Action, User, SendAs

TimeStamp Action User SendAs

--------- ------ ---- ------

2018-11-02T12:13:28 SendAs James.Ryan@office365itpros.com Customer.Complaints@office365itpros.com

2018-11-02T11:57:29 SendAs James.Ryan@office365itpros.com Customer.Complaintsoffice365itpros.com

Reporting Document Labels

The same technique of grouping and sorting data from audit records can be used to report other aspects of Office
365. For instance, if you want to know what the most popular label applied to classify documents is, you would search
for ComplianceSettingChanged events, extract the audit data, and put the events into an ordered hash table. You can
then group and sort the table after extracting the records for labels applied to files (rather than folders or lists).
Something like this will work.

[PS] C:\> $GroupData = $Report | ? {$_.ItemType -eq "File"} | Group-Object -Property Label

[PS] C:\> $GroupData | Sort Count -Desc | Select @{n="Office 365 Label"; e={$_.Name}}, Count

Office 365 Label Count

---------------- -----

Office 365 for IT Pros eBook Content 47

Approved 29

Draft 18

Archive Retention 12

Audit Material 9
 2

Contractual Information 1

The blank entry in the list above is for audit events logged when a user removes a label from a document. Another
point to note about document classification events is that SharePoint can assign a default label to documents added to
a library. When this happens, SharePoint inserts the label name into the UserId property for the audit event where a
user principal name is normally found.

If you have Office 365 E5 licenses, you can use the Label Activity Explorer (see Chapter 19) to see what’s happening
with labels, but PowerShell gives you control over what you report.

SharePoint Sharing Events

SharePoint logs audit events when users generate sharing invitations with people inside and outside the tenant. Three
separate events are recorded.

SharingSet : Someone shares a document with someone else (inside or outside the tenant).
SecureLinkCreated : SharePoint creates and sends a secure link to the target user. This only happens for external
users as users with accounts in the tenant directory can use their accounts to access the shared document.
SecureLinkUsed : The target user uses the secure link to access the document. Again, this only happens when
external users access a shared document.

You can search for these audit records with a command like:

[PS] C:\> $Records = (Search-UnifiedAuditLog -StartDate 28-Mar-2018 -EndDate 26-Apr-2018 -Operations SharingSet, SecureLinkUsed,
SecureLinkCreated -ResultSize 2000)

If you then interpret the audit records, you see this kind of sequence when a sharing invitation is generated and used
by an external user.

TimeStamp : 2018-04-16T16:21:42

User : tony.redmond@office365itpros.com

SharedWith : SharingLinks.dd7d0f94-7757-4646-afcd-c1b824f8676d.Flexible.7a5975c0-8ac2-4167-ba4b-70296cd8df9d

SharedType : SPO Sharing Link

Document : Migration Datasheet.pdf

Action : SharingSet

TimeStamp : 2018-04-16T16:21:43

User : tony.redmond@office365itpros.com

SharedType : SPO Sharing Link

Document : Migration Datasheet.pdf

Action : SecureLinkCreated

Event : <Type>View</Type>

Event : <Permissions granted>Read</Permissions granted>

TimeStamp : 2018-04-16T16:29:45

User : urn:spo:guest#james.pierrott@yandex.com

SharedType : SPO Sharing Link

Document : Migration Datasheet.pdf

Action : SecureLinkUsed

Between the records for file accessed and file sharing, you can create a complete picture of what documents are
accessed by external users.

Activity Alerts and Alert Policies

Searching the Office 365 audit log to find items of interest rapidly becomes boring. It also creates the potential that
you might overlook important audit events. Office 365 has two approaches to automate checking of user activity
within a tenant and alert administrators when something out-of-the-ordinary occurs.

Activity alerts are available to all business tenants. An activity alert checks events recorded in the Office 365
audit log for different conditions and fires when those conditions occur.
Alert policies build on the concept of activity alerts and apply extra intelligence to the events recorded in the
Office 365 audit log. Instead of firing when events occur, policies look for patterns of events such as a certain
number of file downloads over a brief period. Microsoft includes ten default alert policies to help tenants
understand their use and handle common conditions, including notification of malware attacks or instances when
someone is assigned administrative permissions for Exchange Online. Three of the alert policies are available to
Office 365 E1 and E3 tenants. Use of the other policies is licensed through Office 365 E5, or the Threat
Intelligence or Advanced Compliance add-on.

In both cases, Office 365 notifies administrators via email and it is then up to the people notified to act to resolve the
detected problem. Only accounts that hold the Organization Configuration role for the Security and Compliance
Center can create activity alerts or alert policies.

Activity Alerts
Experienced administrators know when something is not quite right. At least, their suspicions heighten when they see
certain things happening. Activity alerts help administrators keep up to date with what is happening inside the tenant.
An activity alert is a request to inform nominated individuals when a certain event occurs. After you create an alert,
Office 365 checks the stream of audit data flowing from individual workloads to detect matching events. If Office 365
detects a match, it generates an email notification to the people registered for the alert.

An activity alert can check for multiple actions across multiple Office 365 accounts. You can select any of the events
logged in the Office 365 audit log for monitoring. For example, you could have an alert that fires when someone
checks a file into a document library or uses the Send As permission to send email on behalf of another user. To create
a new alert policy, go to the Alerts section of the Security and Compliance Center and select Manage Alerts , then
click New alert policy (see note below). You can then add the following information to create the new alert:

A unique name for the alert. For example: Deleted File Alert.
A description of what the alert does. For example: “This alert fires when members of the Executive Committee
delete a file from a SharePoint document library”.
Alert type: This can be Custom, in which case you can select one or more audit events for activities that you want
the alert to fire, or “Elevation of privilege”, which fires when someone raises their privileges within the tenant.
Users: For custom alerts, you can specify the names of the users that you want the alert to cover.
Alert recipients: The email addresses of those who will receive notifications of the alert. The person who creates
an alert automatically receives alerts.

You can also create an activity alert by clicking the New alert policy button when performing a search of the Office
365 audit log. In this case, the parameters for the current search pre-populate the new alert with details of the events
to check. One thing to remember is that Alerts don’t support filtering based on a file or folder name, so you cannot
use them to check changes made to a specific item. For instance, if your search is for file check-ins for a document
called “Budget” and you use the search to create an alert, Office 365 generates alerts for all file check-ins and not just
for that document. In addition, activity alerts do not support date ranges, so the alert will fire for any matching
activity from when you create it.

Alert Policies
Alert policies try to capture the instinct of experienced administrators to detect patterns of problematic activity, using
the ability of software to keep looking for matching patterns repeatedly. When a match occurs between real-time
activity as captured in the Office 365 audit log and the settings defined in a policy, Office 365 triggers an incident and
notifies one or more recipients. The recipients are then responsible for resolving the incident. Conceptually, you can
break down alert policies into four stages:

Understand the normal ebb and flow of user activity to know what you expect to happen within the tenant. This
is the activity baseline and can be set manually through your own observations and experience or automatically
by Office 365.
Define the characteristics of activity that cause concern and watch for incidents when those characteristics
occur. These characteristics are the activity, condition, and threshold checked by an alert policy.
Monitor the events recorded in the Office 365 audit log against the conditions defined in alert policies.
Alert administrators through email notifications when policy violations occur.

Office 365 can trigger alerts for every instance of a certain activity, such as when an administrator grants another
user elevated permissions. Office 365 can also trigger events based on aggregation. For instance, users download 50
files from a SharePoint library within 30 minutes. That might be evidence of a hard-working user. On the other hand,
it might be a sign that someone is grabbing some valuable intellectual property that they plan to take with them to
another job. A more complex form of aggregation is when Office 365 sets the threshold for a trigger by analyzing up
to a week’s worth of activities to understand the normal level of activities within the tenant. If something then
happens that greatly exceeds the expected norm, Office 365 notifies the recipients defined in the policy.

Being able to create policies to check for specific events and patterns helps administrators keep an eye on what
happens within a tenant. To show the value of alert policies, Office 365 creates the ten default alert policies described
in Table 21-1 . You can amend the recipients for alerts generated by the default alert policies, but you cannot change
the other settings. The first three policies are available to all Office 365 E1, E3, and E5 tenants. The others are
licensed through Office 365 E5 (E5), or the Threat Intelligence (TI) or Advanced Compliance (AC) add-ons.

Alert Conditions Severity

Creation of Fires when a user creates a rule to redirect or forward their email outside the tenant. Low
forwarding/redirect
rule.

Elevation of Fires when an administrator assigns another user account administrative permissions Low
Exchange Admin for Exchange (for example, adding an account to the Organization Management role
privilege group). This might be perfectly normal, but it might also be a sign that someone is
going to use those privileges to do something that they should not.

Messages have Fires when the number of messages waiting to be delivered outside the tenant exceeds High
been delayed 2,000 and the level exists on the queue for an hour.

Malware campaign Fires when an unusually (compared to baseline) number of messages with malware High
detected after arrive in user mailboxes., Apart from flagging an alert, Office 365 removes the
delivery problematic messages. [E5/TI]

Malware campaign Fires when Office 365 detects that attackers try to send an unusual volume of Low
detected and messages with known malware to your tenant. Office 365 blocks the malware and
blocked messages do not arrive in user mailboxes. [E5/TI]

Malware campaign Fires when an unusual number of malware detections occur in SharePoint Online and High
detected in OneDrive for Business sites in the tenant. [E5/TI]
SharePoint and
OneDrive.

Unusual increase Fires when there is an increase in the normal volume of email reported by users as High
in email reported phish. [E5/TI]
as phish.

Unusual external Fires when external users with access to SharePoint or OneDrive for Business sites in Medium
user file activity the tenant perform an unusual number of activities like accessing, downloading, and
removing files. [E5/TI/AC]

Unusual volume of Fires when users share an unusual number of files in SharePoint or OneDrive for Medium
external file Business sites with external people outside the tenant. [E5/TI/AC]
sharing

Unusual volume of Fires when users remove an unusual number of files from SharePoint or OneDrive for Medium
file deletion Business sites within a brief period [E5/TI/AC]

Table 21-1: Default Office 365 alert policies

Like any default policies, the default alert policies cover some generic situations that might or might not apply to your
tenant, which is why you can define your own policies. The following settings form an alert policy:

Name and Description . These settings are for administrative convenience and you can enter anything you like.
The name appears in dashboards, so it should convey the intent of the alert policy. The description can be used to
 note who created or last amended the policy and what it is intended to do.
Category : To help track alerts, you can classify policies into the following categories, which you can use to sort
alerts in the View alerts page:

Data governance. For example, users download more than a certain number of files to their PC over a defined
period.
Data loss prevention. For example, misuse of sensitive data.
Permissions. For example, elevation of permissions.
Mail Flow.
Threat management. For example, malware outbreaks.
Others. This category exists for tenants to use as they wish.

Severity : You can assign alerts detected by the policy to be Low, Medium , or High . Obviously, the more
destructive a condition is, the higher its severity should be.
Activity : The activity that the policy tracks. Alert policies do not yet cover all the activities recorded in the
Office 365 audit log because the intention is that alert policies can deliver near real-time notifications about
problem and not all workloads feed events into the audit log that quickly. Microsoft will probably extend coverage
across Office 365 workloads over time. For now, policies can cover most activities in the SharePoint, OneDrive,
and Exchange workloads. You can only select a single activity per policy. If activity policies do not support an
event you want to check, you can create an activity alert to do the job.
Conditions : For most activities, you can define conditions that must exist before Office 365 fires an alert. For
example, a user downloads a file to a computer with a specific IP address. It is also possible to configure alerts to
fire every time a user performs the activity or when users download files from a specific site.
Threshold : How often an activity must occur within a period before a problem condition exists. The threshold
can also be unusual activity, which is when Office 365 compares activity against the baseline for the tenant.
Setting too low a threshold usually results in many notifications, which can hide real problems (the lowest
threshold is 3 activities). Setting too high means that Office 365 might not send notifications for situations when
administrators need to act. It is easy to tune a threshold after noting how many notifications administrators must
process to reach a point where notifications arrive when real problems exist. You can also set a daily limit for an
alert policy so that a policy can only ever generate a certain number of notifications in a single day.
Email notifications : You can define that alert notifications go to a list of recipients when Office 365 invokes the
policy. Notifications can go to accounts within or outside the tenant. The account that creates a new policy
automatically receives notifications while the default policies send notifications to the tenant administrators.

The new policy wizard steps through these sections to create a new alert policy. Figure 21-8 shows how to set the
activity and threshold for an alert policy.

Figure 21-8: Defining the conditions for an alert policy

OneDrive Synchronizations and Alerts : If you implement alert policies for files downloaded from SharePoint or
OneDrive for Business, you should set the threshold for the volume of matches to be higher than the largest number
of files in a library synchronized with the OneDrive sync client. If you set a lower number, you will receive alerts each
time the OneDrive sync client refreshes its copy of the library.
Handling Alerts
When Office 365 detects a pattern of events matching the threshold and conditions set in an alert policy, it triggers an
alert and generates an email to the recipient list defined in the policy. The notification tells the recipient:

The severity level for the alert.

The policy that triggered the alert.
The date and time of the alert in UTC.
The activity that triggered the alert.
The details of the alert. For example, how many matched activities occurred in what timeframe.

The alert message has an Investigate button. When clicked, the link opens a page with the details of the triggered
alert using an URL like https://protection.office.com/#/viewalerts?id=3e8ff308-1c04-5824-ae00-08d4acf0322c . After
viewing the incident, you can resolve or suppress the alert. When you resolve an alert, you record a status for the
alert (Active, Investigating, Resolved, Dismissed) and any comments to justify the status. The new status and
comments and your name become part of the history of the alert. If you suppress an alert, you tell Office 365 how long
you want to suppress email notifications for similar alerts and select from 1 day, 7 days, 30 days, or 365 days. Office
365 continues to trigger new alerts during the suppression period, but you will not receive email about them.

The dashboard for the Alerts section has widgets for alert trends (number of incidents over the last two weeks), alerts
by severity, recent alerts, alert policies, and other alerts. These widgets give an overview of what is happening in the
tenant. More details of recent alerts are available through the View alerts choice. Figure 21-9 shows how
administrators process alerts.

Figure 21-9: Handling alerts

The top left screen shows that Office 365 has triggered several alerts. Inside a busy tenant, you might have many
alerts showing up daily. Filters are available to focus on alerts belonging to a certain category or severity level. If you
select an alert, you can see the details of that alert (this is the same information revealed when you click the
Investigate button in a notification message. In this instance, we can see that 11 “download file to computer”
activities happened to trigger the alert. Details of the activities recorded in the audit log are visible by clicking the
View activity list link (lower left), where we see that a certain user downloaded multiple files. In fact, this kind of
download activity is typical of a synchronization by a OneDrive client of documents stored in SharePoint Online and
OneDrive for Business sites and it is unsurprising to see alerts of this nature flagged when a low threshold exists.
Therefore, you need to set thresholds at a level where that normal user activity does not trigger alerts.

Where are my old Activity Alerts ? If your account has an E5 license, the Security and Compliance Center adjusts
its UI to show you Alert Policies instead of Activity Alerts. You can continue to access Activity Alerts through a link in
the “Other Alerts” section of the Alerts dashboard or by inputting the URL
https://protection.office.com/#/managealerts . Or there is always PowerShell.

Activity Alerts and Policies and PowerShell

Some PowerShell cmdlets to manipulate activity alerts and alert policies are in the Security and Compliance cmdlet
set (see Chapter 4 for information). The Get-ActivityAlert cmdlet reports the set of alerts defined for a tenant. For
example, this command lists all the known alerts with the description, the operations that are being checked for the
alert (for instance, filecheckedin is the name of the event logged when someone checks in a file to a SharePoint
document library), the identities (UPNs) of the users being monitored, and the mode of the alert (Enforce means that
the alert is active). Other interesting information includes NotifyUser (the people who receive the alerts), and
LastModifiedBy , which tells you the administrator who last updated the alert.

[PS] C:\> Get-ActivityAlert | Format-Table Identity, Description, Operation, UserId, Mode -AutoSize

The other cmdlets are:

New-ActivityAlert : Creates an activity alert.

Set-ActivityAlert : Amends an existing activity alert.
Remove-ActivityAlert : Removes an activity alert.

This example shows how to use the New-ActivityAlert cmdlet to create a new alert. In this case the alert checks for
the SendAs and SendOnBehalf actions when taken by two specified users.

[PS] C:\> New-ActivityAlert -Name "Monitor Send As and Send on Behalf" -Description "An alert to monitor the Send As and Send on
Behalf of actions" -Operation SendAs, SendOnBehalf -NotifyUser Kim.Akers@Office365ITPros.com -UserId "Ben.Owens@Office365ITPros.com",
"Jack.Healy@Office365ITPros.com"

As an example of using Set-ActivityAlert to update an activity alert, here is how to configure two recipients for an
alert. Both external and internal email recipients can receive alerts. It is also possible to add the SMTP address of a
distribution group or an Office 365 Group as an alert recipient, in which case the alerts go to the groups. You cannot
add a group as an alert recipient through the GUI.

[PS] C:\> Set-ActivityAlert -Identity "Monitor File Check-ins" -NotifyUser "James.Smith@SomeDomain.com",

"Kim.Akers@Office365ITPros.com"

To turn an alert off, you set its Disabled property to $True .

[PS] C:\> Set-ActivityAlert -Identity "File Downloads Alert" -Disabled $True

Finally, this example uses the Remove-ActivityAlert cmdlet to remove an activity alert without pausing for
confirmation:

[PS] C:\> Remove-ActivityAlert -Identity "Monitor File Check-ins" -Confirm:$False

It is certainly possible to use these cmdlets to do the job they are designed for, but it is usually easier and quicker to
create and edit activity alerts through the Security and Compliance Center. The names of the events that you can
check are sometimes not obvious and there are over a hundred to choose from. It is easy to make a mistake typing in
the command to create an alert that encompasses multiple events for multiple people.

Although activity alerts give an automated method to detect specific events after they occur, you must remember that
notifications are only possible after the workloads feed in audit data. SharePoint and Exchange events fire
notifications reasonably quickly because the audit data is available within 30 minutes. An event for something like a
Power BI event might take longer.

Alert policies are more complex than alerts and it is usually best to manage them through the Security and
Compliance Center. However, a set of cmdlets are available:

Get-ProtectionAlert : Reports the alert policies defined in the tenant.

Set-ProtectionAlert : Amends an alert policy.
New-ProtectionAlert : Creates an alert policy.
Remove-ProtectionAlert : Removes an alert policy. You cannot remove a default alert policy.

For example, to list the set of default alert policies, connect to the Security and Compliance Center with PowerShell
and run this command:

PS C:\> Get-ProtectionAlert | ? {$_.IsSystemRule -eq $True} | Format-Table Name, Category, AggregationType, AlertScenario

Name Category AggregationType AlertScenar

---- -------- --------------- ---------

Malware campaign detected after delivery ThreatManagement AnomalousAggregation Protection

Unusual increase in email reported as phish ThreatManagement AnomalousAggregation Activity

Unusual volume of external file sharing DataGovernance AnomalousAggregation Activity

Elevation of Exchange admin privilege AccessGovernance None Activity

Creation of forwarding/redirect rule ThreatManagement None Activity

Mails have been delayed MailFlow CustomAggregation MailFlow

Malware campaign detected and blocked ThreatManagement AnomalousAggregation Protection

Malware campaign detected in SharePoint and OneDri ThreatManagement AnomalousAggregation Activity

Unusual volume of file deletion DataGovernance AnomalousAggregation Activity

Unusual external user file activity DataGovernance AnomalousAggregation Activity

The AggregationType property tells us whether Office 365 triggers alerts for every occurrence (None ), based on the
volume of the activity in a time window (SimpleAggregation ), or when the volume of activity greatly exceeds the
baseline for the tenant (AnomalousAggregation ).

Office 365 Cloud App Security

The Office 365 Cloud App Security application (previously called Advanced Security Management) is part of the E5
plan and is also available as an add-on for the other enterprise plans. Unless you use groups to scope the coverage of
policies (see section later), every user in the tenant must have a license for Cloud App Security as it is not possible to
exclude the audit data for individual users from the anomaly detection and analysis.

Office 365 Cloud App Security (CAS) is accessed through the Alerts section of the Security and Compliance Center
where Manage Advanced Alerts connects to a special version of Microsoft Cloud App Security (available for
purchase as a standalone product) designed for Office 365. You can upgrade to the full version of Microsoft Cloud App
Security and include other data sources in the mix. Office 365 Cloud App Security keeps up to six months of data
about user activities, or twice as much security data as is normally collected and analyzed for a tenant. Data begins to
be collected as soon you enable Cloud App Security for a tenant.

When a tenant opts-in to use Office 365 Cloud App Security, a link is created between your tenant and an equivalent
tenant automatically created within the Cloud App Security infrastructure. The link allows audit data to be extracted
from Office 365 and analyzed by the Cloud App Security analytics engine, which detects suspicious activity and other
potential problems. It takes about a week after a tenant is enabled before a satisfactory model is created of its normal
activity and build a baseline that suspected anomalies can be measured against.

Office 365 Cloud App Security is part of a long-term plan to give customers much better oversight about what is
happening in their tenant based on the data accumulated in the Office 365 audit log, with the major advantage of the
approach being that no agents or other software needs to be deployed to support the gathering and analysis of the
data to detect the threats that might lie in the anomalies that are picked up. Analyzing the audit data also reveals how
the actions taken by individual users might compromise the security of the organization through suspicious behavior,
such as someone downloading all the documents from a library holding confidential information within a brief period.
Other indications are considered, such as suspicious IP addresses that might originate from anonymous proxies or
known botnets.

Office 365 Cloud App Security allows administrators to create tenant-specific policies to fire alerts when specific
events happen or when a specific pattern of actions occur. For instance, you could create a policy that will alert
administrators by email or SMS whenever certain conditions occur. Microsoft includes a preconfigured “General
anomaly detection” policy to get the ball rolling. This policy covers common conditions that should cause suspicion,
such as a user logging in from two places widely separated in distance within a brief period. You can add other
anomaly detection policies to highlight specific activities that are of concern to the organization. For example, you
could create a policy to look for attempted log-ins from IP addresses outside the corporate IP range. You can tailor
policies to turn off or on different risk factors or to increase sensitivity to a risk.

Alerts
Figure 21-10 shows how alerts appear in the Office 365 Cloud App Security console. In this case, a set of alerts have
been signaled because users have connected to Teams from countries that are not their normal location. Connections
are noted from France and Switzerland for one account, pointing to someone on a road trip, while another user is
flagged from Bulgaria as an infrequent connection. This is possibly because the user has not connected from this
location for a while.
Figure 21-10: Reviewing Office 365 Cloud App Security alerts

If more information is needed to understand the pattern behind a user’s behavior or another aspect of an alert, such
as the IP address, you can click different aspects of the item to have Cloud App Security reveal more information. For
example, if you click on Tony Redmond (Figure 21-11 ), you see all the events logged for that user for the last month
(and can select other periods from the last seven days to the last six months). Three interesting points arise from the
analysis.

A busy Office 365 user generates many audit events gathered from many applications.
SharePoint Online generates more audit events than any other Office 365 application. Not all the SharePoint
events happen through the traditional browser interface as many inside Office 365 occur through other
applications, such as when a user uploads a file to Teams.
Peak activities need explanation. In this instance, some activity caused the massive spike in daily events logged
on May 14 (a restructuring of files across SharePoint sites generated most of the events).
Figure 21-11: Analysis of a user’s activity displayed in audit events for a month

Resolving Alerts
Alerts are rated a high, medium, or low risk. The risk level is calculated using behavioral analytics to compare normal
user interaction with Office 365 against the information contained in the audit data. The analytics are based on
Microsoft’s collected knowledge about the threats that exist, and their origin gathered from across Office 365 and
other cloud services. Assigning a risk value allows an administrator to filter for high risk alerts and prioritize their
resolution.

Another example of an alert is when an account is detected to have elevated permissions (a “New admin user” alert).
Again, if the permissions were assigned purposely, the alert can be resolved, and Office 365 Cloud App Security knows
that it does not have to signal the issue again. However, it could be the case that someone has been assigned
permissions in error or that they hold permissions for too long, in which case the resolution is different and might
need the account to be suspended or to have its permissions adjusted. User accounts can also be suspended as an
action contained in a policy to ensure that action is taken to protect the organization without needing an
administrator to do something manually. Suspended users show up in Office 365 as blocked users. If this turns out to
be the wrong thing to do, you can reverse the suspension from Cloud App Security or the Office 365 Admin Center.

It is possible that an alert highlights an event that is uninteresting or invalid. In these instances, you can dismiss the
alert or mark it as a false positive. These actions are recorded in the Activity Log and the fact that the user’s location
or their admin status is valid is considered by Cloud App Security when it processes audit and other data to detect
anomalies and suspicious activity in the future.

Filters
Filters are available to focus in on one or more of the Office 365 applications or to look for selected users. The latter
filter is valuable when you might be concerned about the activities of a certain individual. You can also search for
high, medium, or low severity alerts or for alerts that have been previously dismissed or resolved. You can also filter
by category (access control, compliance, configuration control, privileged accounts, sharing control, and threat
detection). The filters can be combined to focus in on certain actions, meaning that even a very large volume of alerts
can be quickly refined to produce a set of alerts that need to be examined. You can also export alerts to a CSV file if
needed.

Audit entries extracted from Office 365 can be examined in the activity log along with other logged items, such as
those recorded when an administrator resolves or dismisses an alert. Again, a range of filters are available to reduce
the number of log entries down to a manageable amount. In the example shown in Figure 21-12 , filters extract events
relating to document check-outs by users based in Bulgaria in a certain period. Note the choice in the top right-hand
corner of the screen to create a new policy based on search criteria, meaning that you can easily create a new policy
to create alerts if similar events occur in the future.

Figure 21-12: Browsing the Office 365 Cloud App Security Security Activity Log

Policies
The ability to create customized policies to check events and trigger alerts when predetermined conditions occur is
one of the most powerful features in Cloud App Security. Using templates or from scratch, you can create various
policies to check for various kinds of activity captured in events, including:

Activity policy.
Anomaly detection policy.
App discovery policy.
Cloud discovery anomaly detection policy.
File policy.

These policies help administrators to master the vast quantity of events that busy tenants generate. For example, a
file policy can check for events when users share documents holding sensitive information with people outside the
tenant. You can enable some Data Loss Prevention (DLP) checking to look for specific forms of data, like credit card
numbers and take governance actions, like reporting the problem to a site owner, if the checks discover someone
sharing a file when they should not. Administrators can see the results of the policy in the dashboard and opt to
receive updates via email or SMS text messages.

It is important to emphasize that Cloud App Security does not replace the DLP policies that you can deploy for Office
365 workloads. Alerts are often reviewed sometime after the fact rather than being actioned at once rather than being
brought to the attention of users through visual clues embedded in clients like Outlook or applications like Word.
Instead, checking audit events for problems gives the tenant an added layer of protection. The same is true for
governance as actions to prevent users making mistakes are usually better taken through classification and retention
policies deeply integrated into the Office 365 workloads.

One issue for non-U.S. customers is that Cloud App Security is based on an Azure data store that runs in a U.S.
datacenter. However, only audit data and information about tenant users and groups is moved to the Azure data store
and personal information belonging to tenant users stays within Office 365. Microsoft plans to extend Cloud App
Security so that its data is stored in other datacenter regions in the future. When this happens, Cloud App Security
data for a tenant will be stored in the same region as Office 365.

In some respects, apart from the analytics used by Cloud App Security to pick up suspicious activity by correlating
events, the technology is not rocket science. You could argue that a skilled administrator who knows what is
happening in their tenant is likely to be able to detect and resolve the same kind of issues that Cloud App Security
highlights. However, an application like Cloud App Security scores through its ability to handle massive quantities of
information of the type generated by audit events and to reduce the mass down to what is important. A human can do
this too, but will struggle with:
 The volume of data to process (especially as the environment scales).
The time needed to recognize complex suspicious audit events and to learn the characteristics that mark new
threats.
The need to be consistent in how events are treated.

It is also likely that the human administrator will forget that some events have happened (or not) in the past, so when
something happens, they must consider the event on its merits. Computers are better at remembering things, so
Cloud App Security quickly recognizes when an event is rare (and therefore potentially out of the norm) or normal.

In addition, the machine learning that lies behind analytics is much faster at correlating events to detect suspicious
activity. Once software learns what it should be looking for, it generally produces more consistent results than a
human can, 24 hours a day, 365 days a year, which is why applying technology to automate the collection and
validation of information drawn from multiple sources is a good approach to understanding the kind of threat
introduced by how individuals behave.

SIEM integration with Cloud App Security : Office 365 supports the integration of Cloud App Security with third-
party SIEM servers. A SIEM agent runs on the server to pull alerts from Cloud App Security so that you can integrate
Office 365 alerts alongside alerts generated from other parts of your IT infrastructure. Details of how to perform this
integration are available online .

Scoping Policies to Groups

If you only want to use Cloud App Security to check the activities of a limited set of people, you can tailor policies to
limit them to defined groups. These are Cloud App Security groups rather than Azure Active Directory groups. If you
want to use Azure Active Directory groups, you must import them into Cloud App Security through the User groups
choice in the cogwheel menu. Once imported, you can edit Cloud App Security policies and add a group as a filter.
Because Cloud App Security then scopes the alerts to just the users in the group, the licensing requirement for Cloud
App Security only extends to those users. Make sure that all policies are scoped to groups as otherwise you need to
license every user in the tenant.

Third-Party Audit Alternatives

The Office 365 audit log is available to third party through the Office 365 Management Activity API to build their own
version of an audit investigation tool. One example is Quadrotech Radar for Security and Audit (Figure 21-13 ), which
provides administrators with a different approach than used in either the standard Office 365 audit report or Cloud
App Security. The interface is customizable with filters and pivot tables to help administrators isolate audit events as
they investigate incidents or look for anomalous situations. Quadrotech’s solution does not include some of the high-
end machine-learning functionality and policy-driven event checking found in Cloud App Security, but it is much
cheaper and is therefore an interesting choice to investigate if you think you need more than the standard Office 365
audit report delivers.
Figure 21-13: Quadrotech Radar Security and Audit

Apart from taking a different approach to reporting, third-party products usually allow tenants to store audit data for
longer periods. This is a big advantage for large enterprises who often need to hold audit information for years to
meet regulatory requirements.

Exchange Online Administrative

Auditing
Administrative auditing is a mechanism to track the operations performed by administrators when they invoke
cmdlets to manage Exchange Online. Another way of putting this is that the audit entries allow the age-old question of
“who did that?” to be answered. Administrative auditing is enabled by default for Exchange Online. However, you
have little control over what is audited because Microsoft does not allow tenants to change most of the settings in the
audit configuration.

Auditing is performed by the Admin Audit Log agent, which is active on every Exchange mailbox server. The agent
evaluates cmdlets as they run against the audit configuration to decide whether the use of the cmdlet needs to be
logged. If so, the agent creates an item holding details of the cmdlet and its parameters in the Inbox of the audit
mailbox. The audit agent creates separate reports for each object if you execute an action that is performed against
several objects. For example, if you use Get-Mailbox to fetch a list of mailboxes from a database and then use Set-
Mailbox to place the mailboxes on litigation hold, the audit agent creates a separate audit event for each mailbox as it
is updated.

You can also write your own entries into the audit log with the Write-AdminAuditLog cmdlet. This is intended to allow
administrators to document actions performed in scripts. Up to 500 characters of text can be inserted in the comment
parameter, which is captured in the CmdletParameters property of the audit entry:

[PS] C:\> Write-AdminAuditLog –Comment "NYC mailboxes placed on litigation hold"

Accessing Administrative Audit Log Entries

You can use four methods to access Exchange Online administrative audit events:

Exchange Online administrative audit log events flow to the Office 365 audit log and can be included in audit log
searches performed through the Security and Compliance Center or third-party products. This is the preferred
method because it uses a tool set that is under active development.
Some of the Security and Compliance reports available in the Office 365 Admin Center consume administrator
audit log entries. For example, the “mailbox litigation holds” and “mailbox content search and hold” reports both
use this data.
Some of the reports available in the auditing tab of the compliance management section in EAC, including the
“admin audit log report,” use this data. Possibly even more interesting is the “External admin audit log report,”
which details the changes made to the tenant’s Exchange Online configuration by the Office 365 administrators.
The Search-AdminAuditLog and New-AdminAuditLogSearch cmdlets can be used to search administrative audit
logs through PowerShell. See Chapter 8 in the Companion Volume for some examples of running Exchange audit
searches with PowerShell.

Exchange Online Administrative Audit Configuration

The Get-AdminAuditLogConfig cmdlet reveals the administrative auditing configuration used by Exchange Online (and
in some cases, Office 365). An edited version of its output is shown below:

[PS] C> Get-AdminAuditLogConfig

AdminAuditLogEnabled : True

LogLevel : None

TestCmdletLoggingEnabled : False

AdminAuditLogCmdlets : {*}

AdminAuditLogParameters : {*}

AdminAuditLogExcludedCmdlets : {}

AdminAuditLogAgeLimit : 90.00:00:00

UnifiedAuditLogIngestitionEnabled: True

UnifiedAuditLogFirstOptInDate : 27-Jul-2016 09:00:00

LoadBalancerCount : 1

RefreshInterval : 10

PartitionInfo : {}

Apart from AdminAuditLogEnabled , which tells us that administrative auditing is in effect, the important settings in
the list have the following meanings:

LogLevel : None, meaning that optional informational is not captured in audit entries. What is captured is data
for CmdletName (the name of the cmdlet executed), ObjectName (the object that the cmdlet ran against),
CmdletParameters (parameters passed to the cmdlet), Caller (the account used to run the cmdlet), Succeeded
(whether the cmdlet was successful), and RunDate (the date and time when the cmdlet ran).
AdminAuditLogCmdlets : *, specifying what cmdlets should be audited. In this case, the asterisk means that
any cmdlet that creates or changes the properties of an object is recorded. Note that Exchange Online creates far
fewer audit entries because it can manage fewer objects. For example, databases and servers are not
manageable. Audit entries are never captured for cmdlets that simply retrieve information, like Get-Mailbox or
Get-TransportRule .
AdminAuditLogParameters : *, meaning that all parameters passed to the audited cmdlets are recorded. For
instance, if you use the Set-Mailbox cmdlet to modify a setting on a mailbox, the parameter passed to the cmdlet
is captured in the audit entry.
AdminAuditLogExcludedCmdlets : Logically, as all cmdlets are recorded, none are excluded.
AdminAuditLogAgeLimit : The default value shown is 90.00:00:00, meaning that audit entries should be kept
for 90 days. Administrative events are stored in an Exchange Online arbitration mailbox that is inaccessible to
administrators. You cannot change the retention period.
UnifiedAuditLogIngestionEnabled : If True, audit events from Office 365 workloads are ingested into the
Office 365 audit log. See the earlier section on enabling audit event ingestion for the Office 365 audit log. The
associated UnifiedAuditLogFirstOptInDate property records the date when audit log ingestion started.

Exchange Online Mailbox Auditing

Email administrators are often asked “who did what in that mailbox?” It might be that someone sent a message from a
mailbox in error or someone removed an item without permission. For whatever reason, the question always seems to
end up on the desk of the unfortunate administrator. The audit events captured by mailbox auditing are in three
categories of access, each of which has its own configuration:

Owner : Operations performed by the mailbox owner. Normally there is little point in auditing owner operations
as the owner has full control over their own mailbox. Because normal user interaction with a mailbox usually
generates many audit items, owner level auditing should only be enabled when absolutely needed.
Delegate : Operations performed by another user who has delegate access to the mailbox via the SendAs ,
SendOnBehalfOf , or FullAccess permissions. These actions are the usual focus for auditing, especially when
multiple delegates have access to a shared mailbox or for mailboxes that hold confidential information, such as
the copies of items retrieved by eDiscovery searches that are stored in discovery mailboxes.
Administrative : Operations performed by programs that connect to the mailbox using a special administrative
MAPI access such as mailbox moves.

When auditing is enabled for a mailbox, Exchange captures audit entries for a set of default events for each of the
three access categories listed above. If needed, you can customize the audit configuration to capture specific events
for a mailbox. In addition to forcing Exchange to generate audit events, enabling auditing for mailboxes makes the
audit events available to the Office 365 audit log. Mailbox auditing only functions for user and shared mailboxes. It
does not apply to public folder mailboxes or group mailboxes.

Enabling Auditing for a Mailbox

Until they reconsidered in July 2018 (see section below), Microsoft did not enable auditing for newly created
mailboxes by default, which mean that tenant administrators must run the New-Mailbox cmdlet to enable auditing
after they create new user or shared mailboxes. In the following example, we enable auditing for the shared mailbox
used by the Customer Services team.

[PS] C:\> Set-Mailbox –Identity "Customer Services" –AuditEnabled $True

We can now examine the effect of enabling auditing for the mailbox by examining what actions are captured for the
three audit categories. To do this, we run Get-Mailbox to look at the AuditAdmin , AuditDelegate , and AuditOwner
properties:

[PS] C:\> Get-Mailbox –Identity "Customer Services" | Format-List Audit*

AuditEnabled : True

AuditLogAgeLimit : 90.00:00:00

AuditAdmin : {Update, Move, MoveToDeletedItems, SoftDelete...}

AuditDelegate : {Update, SoftDelete, HardDelete, SendAs...}

AuditOwner : {UpdateFolderPermissions, UpdateCalendarDelegation }

A truncated list of the actions specified for AuditAdmin and AuditDelegate . To see the complete set of audit actions,
expand a property as follows:

[PS] C:\> Get-Mailbox –Identity "Customer Services" | Select –ExpandProperty AuditDelegate

Update

Move

MoveToDeletedItems

SoftDelete

HardDelete

FolderBind

SendAs

SendOnBehalf

Create

UpdateFolderPermissions

The audit configuration shown above tells us that auditing is active for the mailbox, that Exchange keeps audit items
for 90 days, and the set of captured actions for the three categories of access. The set of audited owner actions is
minimal because otherwise a vast quantity of audit records would be created. From the set of actions listed for
Delegate and Admin access we can see that details of item updates, deletes (both hard and soft), and send as
operations are captured. Table 21-2 lists the different actions configurable for capture in each access category.

Action Description Owner Admin Delegate

Create An item is created in the mailbox Yes Yes Yes

Copy An item is copied to another folder No Yes No

FolderBind A client opens a mailbox folder, including the original logon No Yes Yes
to the mailbox. Folder bind actions are consolidated to
remove excessive entries from the audit log. A single entry is
posted per folder for a 24-hour period.

SendAs A message is sent from the mailbox using the SendAs No Yes Yes
permission

SendOnBehalf A message is sent from the mailbox using the No Yes Yes
SendOnBehalfOf permission

SoftDelete An item is moved to the Recoverable Items folder Yes Yes Yes

HardDelete An item is permanently removed from the Recoverable Items Yes Yes Yes
folder

Update The properties of an item are updated Yes Yes Yes

Move An item is moved into another folder Yes Yes Yes

MoveToDeletedItems An item is moved into the Deleted Items folder Yes Yes Yes

MailboxLogin The owner logs in to the mailbox. Yes No No

MessageBind A message was viewed in the preview pane or opened to be No Yes No

read

UpdateCalendarDelegation Record delegate permissions assigned to the calendar. Yes No No

UpdateFolderPermissions Record changes to folder permissions, such as allowing a Yes Yes Yes
user to view a calendar. This action is not available in on-
premises systems.

UpdateInboxRules Record changes to rules to process messages arriving into Yes Yes Yes
the inbox.

Table 21-2: Mailbox actions captured for auditing purposes

The UpdateCalendarDelegation and UpdateInboxRules actions were introduced in 2018. Exchange includes these
actions in the default set of audit actions assigned to new mailboxes, but you might need to assign it to older
mailboxes if you want to collect audit records for these actions.

The HardDelete action does not record audit events for items removed using the Shift+Delete key combination as this
operation only bypasses the Deleted Items folder to move items directly into the Recoverable Items folder.
Shift+Delete operations are logged as SoftDelete actions, which also happens when users remove items from the
Deleted Items folder individually or by emptying the entire folder.

You can change the set of captured actions by running the Set-Mailbox cmdlet. For instance, here is how to enable all
possible audit actions for delegate access.

[PS] C:\> Set-Mailbox –Identity "Customer Services"

–AuditDelegate Create, FolderBind, SendAs, SendOnBehalf, SoftDelete, HardDelete, Update, Move, MoveToDeletedItems,
UpdateFolderPermissions

To disable auditing for a category, input “None” for the action list.

[PS] C:\> Set-Mailbox –Identity "Customer Services" –AuditDelegate None

After you enable auditing for a mailbox, the execution of any of the defined actions causes Exchange to generate an
audit item. If the Office 365 audit log is enabled for the tenant, the ingestion process takes mailbox audit items,
normalizes them to match the Office 365 audit format, and incorporates them in the audit log to make these items
searchable like items from other workloads. In addition, Exchange holds audit items in the user’s mailbox in the
special “Audits” sub-folder under Recoverable Items. The items stay there for the defined retention period after which
the Managed Folder Assistant removes them. You can keep audit items for up to 24,855 days, 3 hours, 14 minutes, and
7 seconds (just over 68 years and an excellent Exchange trivia question).

[PS] C:\> Set-Mailbox –Identity "Customer Services" –AuditLogAgeLimit 24855.03:14:07

To find out how many items exist in the Audits folder for a mailbox, run this command:

[PS] C:> Get-MailboxFolderStatistics –Identity 'Customer Services' -FolderScope RecoverableItems |

? {$_.Name –eq "Audits"} | Format-Table Name, ItemsInFolder, FolderSize

Name ItemsInFolder FolderSize

---- ------------- ----------

Audits 10884 36.74 MB (38,522,242 bytes)

Odd Delegate Audit Records : You might find some audit records for SendAs operations by a delegate for a
mailbox that you know has no delegates and wonder what’s going on. The answer is usually when a background
process impersonates the user to send email. For instance, this happens when Planner generates email notifications
when creating new tasks or completing a task (the ClientInfoString property is “Client=REST,” showing that Planner
used the Graph API for this action). The messages that Planner sends are in the Sent Items folder of the mailbox. You
can match them with the audit records.

Making Sure Auditing is Enabled for All Mailboxes

If you want to include mailbox audit events in the Office 365 audit log, you must enable mailbox auditing for the
mailboxes for which you want to collect information. Normally, you want audit data for user and shared mailboxes,
and it is best to enable auditing as part of the mailbox creation process. It does not make sense to enable auditing
only for a subset of mailboxes as this creates the potential that some needed to investigate for a specific incident will
be missing. The right approach is to enable auditing for all user and shared mailboxes. This command finds all user
and shared mailboxes that do not have auditing enabled, enables auditing, and updates the audit configuration to
capture certain delegate activity.

[PS] C:\> Get-Mailbox -Filter {AuditEnabled -eq $False} -RecipientTypeDetails UserMailbox, SharedMailbox | Set-Mailbox -AuditEnabled
$True –AuditDelegate Create, FolderBind, SendAs, SendOnBehalf, SoftDelete, HardDelete, Update, Move, MoveToDeletedItems,
UpdateFolderPermissions

Because Exchange does not enable auditing for new mailboxes automatically, you need to run the command to find
and enable auditing for new mailboxes periodically, perhaps as a scheduled job.

Mailbox Auditing Enabled by Default

In July 2018, Microsoft announced that they had begun the process to enable mailbox auditing by default across
Exchange Online. This long-awaited and welcome decision makes Exchange Online consistent with SharePoint Online
for user actions and will roll out gradually across Office 365 to be complete for all business tenants by the end of
2018. The new approach uses a tenant-wide configuration setting called AuditDisabled to control mailbox auditing.

[PS] C:\> Get-OrganizationConfig | Select AuditDisabled

False

By default, the AuditDisabled setting is $False , meaning that Exchange automatically audits a set of default actions
for all mailboxes, even if the AuditEnabled setting for a mailbox is $False.

After mailbox auditing is enabled by default, you can still change the set of audit events captured for specific
mailboxes by running the Set-Mailbox cmdlet as described above. The downside is that if Microsoft chooses to update
the default set of audit actions in the future, you must reset the custom audit settings for those mailboxes.

To opt out of default mailbox auditing and stop Exchange capturing any mailbox audit data for the Office 365 audit
log, change the setting to $True.

[PS] C:\> Set-OrganizationConfig -AuditDisabled $True

Mailbox Auditing Bypass

When default auditing is in place, if you want to exclude mailboxes from auditing, you must use the Set-
MailboxAuditBypassAssociation cmdlet. For example:

[PS] C:\> Set-MailboxAuditBypassAssociation -Identity "Kim Akers" -AuditBypassEnabled $True

The Set-MailboxAuditBypassAssociation cmdlet was designed to stop audit events from service mailboxes flooding the
audit log. This might have been acceptable in an on-premises environment when the cmdlet first appeared in
Exchange 2010. Today, it is not recommended to exclude any Exchange Online mailbox from auditing in this manner.
First, the tools available to search the audit log are much better inside Office 365 and a few extra audit events will
make no difference. Second, when an account is excluded from mailbox auditing, Exchange does not capture events
for any mailbox the account can access. An account could therefore open a shared mailbox and remove items in that
mailbox without the recording of any audit data. Last, if you experience something like a Business Email Compromise
attack, you need as much audit data as possible to collect to understand how the attack develops and what was its
impact.

To see if any mailbox auditing bypasses are in place, run the Get-MailboxAuditBypassAssociation cmdlet and filter for
accounts with a bypass. It is quite normal to have many mailbox association records returned, including records for
guest user accounts and system accounts as these are created when new accounts are created in the tenant. For
example, this command finds accounts with mailbox audit bypass enabled.

[PS] C:\> Get-MailboxAuditBypassAssociation | ? {$_.AuditBypassEnabled -eq $True} | Format-Table Name, WhenCreated,

AuditBypassEnabled

Name WhenCreated AuditBypassEnabled

---- ----------- ----------------- -

Kim Akers 02/12/2014 15:20:42 True

To disable a mailbox audit bypass, run Set-MailboxAuditBypassAssociation:

[PS] C:\> Set-MailboxAuditBypassAssociation -Identity "Kim Akers" -AuditBypassEnabled $False

Supervision Policies
Supervision policies allow organizations who need to perform surveillance of employee communications to check that
nothing untoward exists in messages. Although it might seem sneaky and reminiscent of a “big brother” approach to
employees, certain industries have regulations that force companies to ensure that they have a supervision policy in
place, perhaps only for employees that work in specific areas. For example, FINRA , the U.S. Financial Industry
Regulatory Authority, enforces rules governing the activities of brokers and dealers to ensure market transparency
and fairness.

In the past, on-premises organizations had to write and install special software to capture and examine messages sent
between employees. The software ran in the Exchange transport pipeline to capture copies of messages and route the
copies to people who are skilled in regulations and compliance, who review the traffic to ensure that no problems
exist. Companies often used these “transport sinks” together with transport rules to prevent certain groups of users
communicating with each other.

In 2015, Microsoft introduced Supervisory Review Policies (SRPs) to do the job of the custom software found in on-
premises systems. SRPs copy a certain percentage of messages exchanged between defined individuals to a discovery
mailbox. Reviewers then access the mailbox to examine the messages and decide if all is well or if they need to take
some action because a monitored user did something dubious.

Supervision Policies are the second generation of SRPs and reached general availability in May 2017. They are part of
Office 365 Advanced Data Governance, which means that all accounts coming under the scope of a supervision policy
need to have an Office 365 E5 license or the Advance Compliance Plan. You manage supervision policies through the
Data Governance section of the Security and Compliance Center. Existing SRPs continue to function, but it is
obviously best to move to supervision policies as soon as possible. No migration functionality exists to move older
SRPs or the data under review for these policies to new supervision policies, so you must remove the old SRPs and
recreate them as supervision policies. As you cannot access the old SRPs through the Security and Compliance
Center, you must remove them by running the Remove-SupervisoryReviewPolicy cmdlet.

Components of a Supervision Policy

A supervision policy includes the following components:

Reviewees : The people (individuals or groups) whose communications need to be reviewed. You can use email
distribution groups to define the reviewees for a policy.
Conditions : Queries and conditions to find items for review. Like content searches, queries are in KQL syntax.
However, supervision policies only support relatively simple queries when compared to content searches.
Conditions include size limits and types of attachments.
Sample size : The percentage of messages satisfying the policy conditions captured for review. You can capture
100% of messages or a smaller amount.
Reviewers : The people who will access the items captured for review to decide whether the content of those
items comply with the regulations in force. The reviewers use OWA or Outlook to access items for review using
an add-in provided by Microsoft. The add-in allows the reviewers to assign a compliance status to an item and
make comments to justify their decision.
Reports : A set of reports about supervision policy activity in the Reports section of the Security and Compliance
Center.

You can deploy multiple supervision policies within a tenant, each of which monitors the communications for a distinct
group of reviewees. Supervision policies only cater for Exchange messaging today. Communications through Yammer
or Microsoft Teams are not subject to review. Items are captured in user and group mailboxes to record the contents
of chats but as these items do not flow through the transport system, they are not assessed against the policies
expressed in supervision policies.

With the structure of a supervision policy in mind, before you can create a new policy, you should answer the following
questions:
 What mailboxes come within the scope of the policy?
Mail traffic between users includes a vast variety of content. How do we focus in on the messages for review? For
example, are we looking for a specific code word or terms?
What is an adequate sample for review? You can start with 100% to check that the policy works and captures the
messages that you expect and then throttle back to a more practical level of perhaps 20%.
Who will examine the captured traffic? Obviously, the reviewers need to understand what constitutes a problem
when they see it in a message. They also need to understand what is a serious violation that needs immediate
action and what is not so serious.
What happens when a violation occurs? All violations have consequences but some violations have graver
consequences than others. An escalation process might be needed to allow reviewers to pass violations on to a
higher authority for decision about how to handle a matter, including potentially reporting a case to a regulatory
authority.

User communications. Before the tenant launches a supervision policy, it is only fair to inform the users whose
traffic is checked about the issues that the policy checks for and the consequences that flow if a violation is
detected and proven. Clearly this is an area where the company legal and HR departments have a key role to
play.

With answers to these questions known, you can go ahead and create a policy. Only users who hold the Supervisory
Review permission can work with supervision policies. To manage the role group, go to the Permissions section of the
Security and Compliance Center. You can also use PowerShell. For example, to discover those in the role group, use
this command:

[PS] C:\> Get-RoleGroupMember -Identity “Supervisory Review"

To add someone to the role group, use to Add-RoleGroupMember cmdlet.

[PS] C:\> Add-RoleGroupMember -Identity Supervisory Review" -Member “Sanjay Ramaswamy”

Creating a New Supervision Policy

Supervision policies are managed through the Governance section of the Security and Compliance Center. Creating a
new supervision policy goes through several steps. The first is to name the new policy and give some administrative
details about the policy. Users do not see this information as they have no idea that supervision policies are in use
unless you tell them. You cannot change the name of a supervision policy after it is created.

The next step selects the reviewees, the users whose communications the supervision policy will check. The reviewees
illustrated in Figure 21-14 include a mixture of email distribution groups (dynamic groups are supported) and
individual accounts. When you save the policy, Office 365 expands the membership of the selected groups and adds
any mailboxes from the membership to the set of reviewees. This is a one-time operation and users added afterwards
to a group do not come under the scope of the policy. If you use groups, you can exclude individuals from the group
membership from policy review.

Figure 21-14: Deciding the users who come within the scope of the supervision policy

The next stage is to define the characteristics of the messages that will trigger capture of copies for review. If you
want to check every message sent to, from, and between the individuals defined earlier, do not define any conditions
except the direction of traffic (see below) and go to the next step. In most cases, it makes sense to narrow the scope
for the monitored traffic to focus on what is important and reduce the workload for the reviewers.

The first condition is the direction of the message traffic to monitor. This is one or more of the following:

Inbound : email sent to the people included in the policy from users who do not come under the scope of the
policy. Traffic can come from other users within the organization or people outside the organization.
Outbound : email sent by the people included in the policy to anyone else.
 Internal : email sent between the people included in the policy.

You can then narrow the traffic further by creating conditions to filter messages selected for checking. Figure 21-15
shows that the policy will check traffic for all sources and isolate messages that mention the word “Project” within
three elements of the word “Contoso”. For the query to match, the words must appear in the order given. In this case,
the query will match items containing “Project Contoso” but not items with “Contoso project.” To match both, we need
to include both phrases.

Figure 21-15: Setting conditions to check message traffic

The available conditions are as follows:

Specified words or phrases appear in the message body (or its subject). Use KQL syntax to define the query, but
do not try to get too complex (see note below). The queries are evaluated as messages pass through the transport
pipeline. Alternatively, you can add a condition that checks for messages where the specified words are not
present. You will see an error if the specified query is too complex for a supervision policy.
Any attachment to the message includes or does not hold the specified words or phrases.
The message has an attachment of a specific type or does not have an attachment of a specific type.
An attachment is larger than a specified size (in KB, MB, or GB).
Overall message size is larger than a specified size.

You can define multiple conditions to find messages of interest, such as those with certain words and have a
PowerPoint attachment.

Supervision Policy Queries : It is important to understand that supervision policies do not run content searches to
find the items for review. Instead, agents running in the transport pipeline use the queries to find matching items as
messages flow through the pipeline. This is the reason why the queries that you can use with supervision policies are
much simpler than those used for searches.

The next step is to specify the percentage of matching traffic to capture for review (Figure 21-16 ). You might start out
by capturing 100% to assure yourself that the policy is picking up the right messages and then throttle back to a less
demanding regime so that reviewers have fewer items to check.
Figure 21-16: Deciding what percentage of matching traffic to review

Finally, you nominate the people who will check the message traffic captured by the policy. Although Figure 21-17
shows that items captured by the policy will be reviewed by two individual accounts, this is another place where it can
be best to use a distribution group. When a supervision policy is used in production rather than testing, you must
make sure that the policy has enough reviewers assigned to handle the expected workload generated by the volume of
captured items, including when some reviewers are ill, on vacation, or unavailable for another reason.

Figure 21-17: Defining the reviewers for the policy

After all parts of the policy are input, click Next to review the settings for the new policy followed by Finish to create
the policy. Office 365 then sets up the review mailbox to hold captured messages and activates the policy. It can take
up to 15 minutes before a policy is fully effective and begins to capture messages. You cannot make changes during
the provisioning process. Instead, wait until the portal reports the policy status as “Active” and then make the change.

Updating a Supervision Policy

After a policy is active, you can make changes to it and publish those changes. To edit a policy, select it in the portal.
The portal displays details of the policy including some statistics about items reviewed after capture. The statistics are
at the bottom of the screen, so you might have to scroll down to see them. The agents responsible for processing
supervision policies update statistics for a policy as reviewers deal with items placed in the review mailbox. Office 365
updates the statistics as the reviewers assigned in the policy process captured items using the Supervisory Review
add-in. You can use the statistics to understand the volume of items captured by the policy and to see a snapshot of
how reviewers are dealing with the captured items.

Click Edit policy to invoke the wizard to step through the different elements of the policy and make changes.
Alternatively, click the Edit link shown beside each element of the policy to change that. For instance, to change the
percentage of items captured, click the Edit link beside “Percentage to review,” enter a new percentage, and click
Save .

Office 365 publishes changes to supervision policies faster than the creation of new policies because the review
mailbox used by the policy already exists. Thus, a change to a policy is effective within a few minutes.

Processing Messages for Review

The older Supervisory Review Policies use transport rules to copy messages for review. This is an effective way of
collecting messages because all email must pass through Exchange’s transport system for onward delivery. The
downside is that a danger exists that the transport rules created for SRPs might compromise the rules that already
exist within a tenant and vice versa. For this reason, Microsoft has moved away from transport rules and the current
architecture selects messages using mailbox agents that run in the background.

Reviewing Selected Content

Supervision policies use review mailboxes to store the copies of messages selected for review. As users send
messages, a mailbox agent checks items passing through the transport pipeline to find any that match the criteria set
in the policy. If the agent finds a match, it copies that message to the review mailbox for the policy. You can see the
name of the review mailbox in the settings for a policy. Look for “Supervision mailbox” where you will see the email
address of the review mailbox listed as something like:

SupervisoryReview{24c680ac-5005-4f0d-bc54-99574ba33d0f}@tenant.onmicrosoft.com

Users cannot send email to a review mailbox and these mailboxes do not show up in any administrative interface,
including PowerShell. The only evidence of their existence is their email address and the way that reviewers interact
with the review mailbox as a shared folder. Accessing a review mailbox is simple if you use OWA because Office 365
automatically installs the supervisory review add-in for all users when an administrator adds them as reviewers to a
policy. The add-in connects all review mailboxes that a user can access when they open an OWA session. Each review
mailbox shows up as a form of shared folder in OWA’s folder list. Things are more complex if you want to use Outlook
desktop to access a review mailbox because you need to create a new Outlook profile to include the review mailbox.
See this support article for details.

The task of the reviewer is to look through the captured items held in the review mailbox and decide whether any
compliance violation exists. In Figure 21-18 , we see that the reviewer has access to two review mailboxes, each
showing up as a shared folder in OWA’s resource list. After opening a review mailbox, the reviewer sees the set of
messages awaiting review waiting in the Inbox. The process to navigate between items and read their content is the
same as used within your own mailbox.

To process an item, the reviewer opens it in the list, reads the content to satisfy themselves that they understand the
content and whether any violation exists. They can send the message to someone else to seek advice or reply to the
sender of the message to ask them about the content and what the sender meant when they sent the message.
However, they cannot remove an item from a review mailbox.

When the reviewer is happy that they can complete their review of an item, the reviewer opens the Supervisory
Review add-in, which is added automatically to the OWA read message screen for reviewers. The add-in uses a form
to allow the reviewer to classify the item in one of four categories:

Compliant . The item is compliant and meets all applicable regulations.

Non-compliant . The item is obviously non-compliant and needs investigation to resolve the matter.
Questionable . Some questions still exist whether the content in the item satisfies applicable regulations.
Resolved . A compliance issue might have existed with the item, but the issue is now resolved.

These are generic explanations of what each category means and a tenant is free to use their own interpretation.

Figure 21-18: Reviewing items captured by a supervision policy in a review mailbox

To process an item, the reviewer must use the form to assign it one of the four classifications detailed above (Figure
21-19 ). They can add comments to explain their decision. For example, if the item is questionable, the reviewer might
explain why they consider the item to be in this state and the actions they have taken to resolve the matter. The
reviewer then clicks to assign the classification to the item.
Figure 21-19: Resolving an item discovered by a supervision policy

Office 365 then updates the item properties with the classification and comments and moves the item into a sub-folder
of the same name as the assigned classification. The resolved items stay in the review mailbox as they might be
subject to further review or perhaps be needed as documentary evidence for a compliance investigation. The items do
move even if they appear to stay in the Inbox, as you can confirm by switching focus to another folder and then back.

Do not underestimate the amount of work involved in processing a couple of hundred review items, a volume that a
busy tenant can easily generate daily. Apart from the opening and reading of each item, the reviewer must understand
the context of the communication and how the content fits with regulations. It is obvious that hours of work might be
consumed to process items at a high level of accuracy.

Removing a Supervision Policy

To remove a supervision policy, select it from the GUI and then take the Delete policy action in the details pane.
Alternatively, you can run the Remove-SupervisoryReviewPolicyV2 cmdlet. In either case, Office 365 removes the
policy and the review mailbox, including all the messages awaiting review. This is not an issue from a compliance
perspective because the original messages are in user mailboxes. If needed for eDiscovery, the mailboxes are probably
on-hold and the users cannot remove the messages.

PowerShell and Supervision Policies

As with most Office 365 applications associated with Exchange Online, PowerShell underpins supervision policies. You
can create, amend, and remove policies with PowerShell, but the truth is that most will be happy to work with policies
through the Security and Compliance Center.

Like other policies, a supervision policy divides into general settings and rules. To view the supervision policies that
exist in a tenant, use the Get-SupervisoryReviewPolicyV2 cmdlet. This cmdlet is available in both the Exchange Online
and Security and Compliance Center PowerShell modules, but the cmdlets to add new policies, amend their
properties, or remove policies are only in the Security and Compliance Center module. In this example, we see that
two policies are active and enabled.

[PS] C:\> Get-SupervisoryReviewPolicyV2 | Format-Table Name, PolicyStatus, LastModifiedBy, Enabled, ProvisioningStatus

Name PolicyStatus LastModifiedBy Enabled ProvisioningStatus

---- ------------ -------------- ------- ------------------

SRP: Ethical Trading Policy Active Tony Redmond True Provisioned

Monitor Project Contoso communications Active Tony Redmond True Provisioned

Each policy will have one rule, which we can see by running the Get-SupervisoryReviewRule cmdlet. Here we list the
properties to reveal the name (a GUID is used more recently to identify rules), the sampling rate, and the rule that
states the type of traffic to monitor, the reviewees, and the query and conditions used to find messages to check.

[PS] C:\> Get-SupervisoryReviewRule | Format-Table Name, SamplingRate, Condition



Name : 8eb4f4da-a06e-4ca4-bfe0-87054bc8dce2

SamplingRate : 100

Condition : ((((Direction:Inbound) OR (Direction:Outbound) OR (Direction:Internal))) AND ((("(Project NEAR(3) Contoso)"))) AND

(((Reviewee:Marc.Vigneau@office365itpros.com) OR

(Reviewee:James.Gangley@office365itpros.com) OR Reviewee:James.Abrahams@office365itpros.com) OR

(Reviewee:Andy.Ruth@office365itpros.com))) AND (NOT(((Reviewee:Ed.Banti@Office365itpros.com)))))

Name : SRP: Ethical Trading Policy Rule 1

SamplingRate : 100

Condition : ((((Direction:Inbound) OR (Direction:Outbound) OR (Direction:Internal))) AND ((("""Trading Profit"" NEAR(5)

""Deal"""))) AND (((Reviewee:SREmployees@Office365itpros.com))))

New-SupervisoryReviewRule -Policy "SRP: Ethical Trading Policy" -Condition "((((Direction:Inbound) OR (Direction:Outbound))) AND
(((""Trading Profit"" NEAR(10) Deal) OR (""Large"" NEAR (10) ""Funds Transfer"") OR (""SEC 17A-4""))) AND
(((Reviewee:Administrator))))" -Name "SRP: Ethical Trading Policy Rule 1"

The more adventurous might consider managing the finer details of supervision policies through PowerShell. The rest
of us will use PowerShell to check things out when we are not quite sure what is happening.

Reporting Supervision Policies

A supervision report is available in the Reports dashboard of the Security and Compliance Center. The report depends
on information fetched from a tenant covering all supervision policies, which Office 365 then processes centrally to
create data about items captured for review. Two cmdlets are available to interrogate the reporting data. Get-
SupervisoryReviewReport records the statistics for the number of items reviewers assign to each classification for
policies active in the tenant. For example, to see what decisions reviewers took for a certain named policy on 12-May-
2017:

[PS] C:\> Get-SupervisoryReviewReport -Policies "Monitor Project Contoso Communications" -StartDate 12-May-2017 -EndDate 12-May-2017
| Format-Table Reviewer, TagType, ItemCount

Reviewer TagType ItemCount

-------- ------- ---------

Nancy.Anderson@office365itpros.com Compliant 5

Nancy.Anderson@office365itpros.com Non-Compliant 1

Nancy.Anderson@office365itpros.com Questionable 1

Nancy.Anderson@office365itpros.com Resolved 2

The results tell us that only one reviewer was active on the day and that they processed a total of 9 items.

The Get-SupervisoryReviewPolicyReport cmdlet generates a summary of the work done by reviewers and some added
information that is useful for reporting purposes. In this example, we extract the reporting data for the same policy on
the same day:

[PS] C:\> Get-SupervisoryReviewPolicyReport -Policies "Monitor Project Contoso Communications" -StartDate 12-May-2017 -EndDate 12-
May-2017 | Format-Table Tagtype, ItemCount

TagType ItemCount

------- ---------

Compliant 5

HitPolicy 16

InPurview 29

New-Reviewed 9

Non-Compliant 1
Not-Reviewed 7

Questionable 1

Resolved 2

If no review activity occurred on the specified dates, you will not see counts for items placed into the various
classifications. The interesting data here are:

InPurview : Count of items that match the reviewees.

HitPolicy : Count of items that match the reviewees and all conditions specified in the policy, including the
sample percentage.

In this case, 29 items that matched the reviewees passed through the transport pipeline but only 16 of those matched
the policy settings to create a copy for review. The reviewer processed nine of the items, so seven are unprocessed
from the workload generated on the day. It is not usually a good sign when reviewers cannot get to process items
within a day of their generation.

If you look at the properties of a policy through the Security and Compliance Center, the statistics revealed there
reflect the current disposition of review items. In fact, the statistics are counts of items in the different folders of the
policy mailbox used for review items, so it is easy to fetch and display the data. By comparison, data is not available to
the reporting cmdlets until between 36 and 48 hours following the events. Such a delay is the norm for cmdlets that
report Office 365 activity and is because of the need to fetch data from individual servers before the data can be
processed centrally.

Reporting Office 365

Despite many requests over the years, Microsoft has never delivered good reporting facilities for on-premises
applications. Instead, they leave the task to administrators or ISVs who create reporting products to come up with
ways to extract and report information about objects and operations in sensible ways. Sources such as the IIS logs
hold valuable information about client connections and file operations for SharePoint while Exchange messaging
tracking logs hold useful data to learn about the volume of messages sent as well as to create lists of users who make
heavy use of the system (and conversely, those who are inactive). Active Directory is a reliable source for server and
organization configuration, while the system registry is yet another place from which to fetch configuration data.

Office 365 Reporting Web Service and DataMart

The Office 365 Reporting Web Service is a now old and partially deprecated interface to the Office 365 Reporting
DataMart, a repository that gathers and combines data feeds from Exchange Online, Exchange Online Protection,
Skype for Business Online, Azure Active Directory, and other Office 365 sources. The reporting web service is only
available to Office 365 tenants that have enterprise plans. To populate the DataMart, Microsoft extracts and
aggregates a range of information from the various workloads periodically. You can see the kind of information
available in the DataMart by running the Office 365 reporting PowerShell cmdlets .

Retiring some reporting cmdlets : Microsoft is moving to a new platform for the Office 365 Reporting Web
Service based on the Microsoft Graph . A consequence of the change is the deprecation of many cmdlets used to
report information about various aspects of Office 365. See this page for information about Office 365 usage reports
available in the Graph.

The example below shows the output from the Get-MailTrafficReport cmdlet with parameters used to fetch details of
inbound messages for a specific day. The different events reported are described in the online documentation . Briefly,
the Goodmail event type notes the count of unique messages delivered after passing through the malware and anti-
spam filters; “BCL” stands for bulk complaint level and is the level of certainty as determined by Exchange that
messages are bulk mail (anything marked with BCL0 is considered personal email). The other figures are for the
amount of traffic that pass through different stages of malware prevention.

[PS] C:\> Get-MailTrafficReport -Direction Inbound -StartDate 7-Sep-2018 -EndDate 8-Sep-2018

Domain Date Event Type Direction Action Message Count

------ ---- ---------- --------- ------ -------------

7 Sep 2018 00:00:00 AtpGoodMail Inbound Allow 1

7 Sep 2018 00:00:00 BCL0 Inbound 81

7 Sep 2018 00:00:00 BCL1 Inbound 50

7 Sep 2018 00:00:00 BCL3 Inbound 1

 7 Sep 2018 00:00:00 BCL4 Inbound 2

7 Sep 2018 00:00:00 BCL5 Inbound 2

7 Sep 2018 00:00:00 BCL6 Inbound 1

7 Sep 2018 00:00:00 GoodMail Inbound 146

7 Sep 2018 00:00:00 NonSpam_ContentScanPassed Inbound 143

7 Sep 2018 00:00:00 Spam_ContentScanFiltered Inbound 10

7 Sep 2018 00:00:00 Spam_SenderBlocked Inbound 2

7 Sep 2018 00:00:00 SpamContentFiltered Inbound 12

7 Sep 2018 00:00:00 SpamDBEBFilter Inbound 1

7 Sep 2018 00:00:00 SpamIPBlock Inbound 6

7 Sep 2018 00:00:00 SpoofMail Inbound GoodMail 2

7 Sep 2018 00:00:00 TransportRuleMessages Inbound 137

The Get-MailTrafficReport cmdlet can report the last 14 days of message data, but some of the other reporting
cmdlets support retrieval of up to 180 days of information depending on the source that they draw from. In fact,
sometimes the reporting cmdlets return data older than 180 days. This is because Office 365 uses a background
process to age out and remove obsolete reporting data. The background process is “lazy” in that it is resource
constrained and therefore might leave obsolete data in place for some time after its expected removal.

Using PowerShell to Create Custom Office 365 Reports

Administrators have used PowerShell for years to generate reports and many examples of scripts to produce nicely-
formatted reports are available on the web. You can certainly use PowerShell to do the same to create your own
version of some of the standard Office 365 reports and to build reports that meet your specific needs. For example, we
might want to know the size of user mailboxes. A quick PowerShell query reveals the answer:

[PS] C:\> Get-Mailbox –RecipientType UserMailbox | Get-MailboxStatistics | Sort ItemCount

-Descending | Format-Table DisplayName, TotalItemSize, ItemCount -AutoSize

DisplayName TotalItemSize ItemCount

----------- ------------- ---------

Tony Redmond 4.116 GB (4,419,110,350 bytes) 36529

Jeff Guillet 1.62 GB (1,739,456,346 bytes) 10411

Vasil Michev (Technical Guru) 1.081 GB (1,161,234,561 bytes) 7019

Deirdre Smith 933 MB (978,334,934 bytes) 6207

Paul Cunningham 1010 MB (1,059,413,045 bytes) 6146

Reports like this hold valuable data and are useful on an ad-hoc basis. However, when you set out to use PowerShell
with Office 365, you should realize that:

PowerShell is an excellent method to query data but is less good when the time comes to format reports for
printing. PowerShell can output data in CSV format that can you can process in Excel to create charts but
generating a report and a few charts in Excel is different from creating a clear and well laid-out report.
PowerShell has limited ability to analyze data over an extended period (the cmdlets that retrieve data from the
Office 365 reporting data mart sometimes do not give a very granular level of information).
PowerShell is throttled by Office 365 if a script attempts to process details of hundreds or thousands of
mailboxes. Office 365 throttles PowerShell processing to ensure that no job soaks available resources and so
reduces service to other tenants. It is good if your script can avoid the need to call the Get-Mailbox cmdlet to
create a set of mailbox object for processing as this is a resource-intensive activity that becomes a candidate for
throttling. One way around this is to create lists of mailboxes and store them in CSV files that can then be opened
and processed by other scripts.
Some Office 365 components do not support PowerShell or make their data available to PowerShell. Exchange
Online has good coverage, but you will not be able to generate reports for Teams or Yammer.

Finally, if you do create your own PowerShell reports, you must be prepared to check and potentially update them
regularly to ensure that code does not break when Microsoft introduces updates to cmdlets, new functionality, or new
versions of PowerShell into Office 365. On the upside, some will like the challenge of writing and updating their own
reports, others will find that it is easier and more efficient in the end to buy a reporting package from a company that
specializes in this space, and it is always great to be able to reuse the many contributions of PowerShell scripts and
code snippets that people make to the community to form the basis of your solution.

Office 365 Admin Center Reports

Possibly because more of a need exists to help administrators manage resources like licenses and user accounts in a
multi-tenant environment (when you use your own hardware, no one really cares what you do with it), Office 365
includes much better reporting facilities than is available for the on-premises applications. The Office 365 Admin
Center includes a Reports section (Figure 21-20 ) that offers a choice of reports covering various aspects of the
service. You don’t need to grant someone full administrative access to see these reports as a Report Reader
administrative role can be assigned to any licensed user (see Chapter 4). Two links to reports are available in the
Office 365 Admin Center:

Usage : How much use people make of basic Office 365 workloads, like how many messages users send and
receive using Exchange or files viewed or edited with SharePoint.
Security and Compliance : This link used to give access to a set of reports associated with mail flow and
compliance activities. The link now connects to the Reports section of the Security and Compliance Center.

The detail and coverage of the reports available in the Office 365 Admin Center has improved over time and certainly
now cater for all the basic requirements that a tenant needs for activity reports. Microsoft makes up to 180 days of
reporting data available to tenants.

Figure 21-20: Standard Office 365 usage reports

You can use Export to save the data used for reports to a CSV file. Alternatively, if you want to preserve the graphics,
you can either take a screen shot or use the browser’s print to PDF choice to create a PDF file holding the
information. It is also worth mentioning that you can access some more reports through an Azure Active Directory
subscription (some of which need an extra license). Naturally these reports focus on directory-centric activities such
as password resets, anomalous sign-ins, and rights management, but they are useful in understanding the complete
operating environment.

Third party reporting products make their money by offering a wider range of reports that go to a more granular level
of detail than is available in the standard reports. They also usually give access to information gathered for more than
the standard 180-day reporting window or incorporate data gathered from a variety of sources within a customer’s IT
environment. In comparison to the standard Office 365 reports, which tend to focus on tracking usage, ISV products
often place greater emphasis on analysis to help tenants understand what the data means and how to improve
operations in aspects such as license management.

To stay in business, third party reporting products must stay ahead of what Microsoft delivers in the Office 365 Admin
Center. Better analysis and insight into a business is always welcome and this is exactly the advantages that you
would hope to gain by using a third-party product. To address the need to dive deeper into how workloads and users
are performing, Microsoft gives access to the underlying reporting data via the Office 365 Reporting web service to
allow third party developers build tailored solutions to satisfy specific customer needs that the out-of-the-box reports
do not address.

Security and Compliance Reports

The reports available in the Security and Compliance Center cover the functionality managed through the center,
including anti-malware activities, data governance, data loss prevention, and supervision policies. The dashboard
gives a quick overview of activities across the tenant. You can then click on a dashboard widget to see the full report.
Some of the reports allow you to create a schedule report, which means that Office 365 creates a report once a week
(or once a month) and sends it to you via email. Other reports support filters to focus in on specific information or the
ability to export the underlying data to a CSV file.

Anonymized Report Data

Some companies are uncomfortable with the idea that administrators can see what might be sensitive data about user
activity in the reports available in the Office 365 Admin Center. For example, it is easy to discover how many
messages the CEO sends and receives or how much Skype for Business conferences he or she attends. If this is a
concern, you can opt to show anonymized data in the reports. This means that Office 365 replaces the display names
for user accounts with an obfuscated GUID that cannot be associated with an account. To display anonymized data in
reports, go to the Settings section of the Office 365 Admin Center, select Services & Add-ins and go to the Reports
section. You can then enable anonymized identifiers. After a few minutes, the anonymized data replaces display names
(Figure 21-21 ).

Figure 21-21: Anonymized user data in the Office 365 email activity report

Microsoft 365 Usage Analytics Power BI Content Pack

Microsoft offers tenants a Power BI content pack (previously known as the Office 365 adoption content pack) to help
understand the usage patterns for some of the Office 365 applications. A content pack is a collection of pre-packaged
graphs and charts that you can load into Power BI. You only need a basic Power BI license to access the content pack.
Together, the graphs and charts form a dashboard for a tenant divided into four areas:

Adoption: Tracks the usage of Office 365 and basic workloads like Exchange and SharePoint.
Communication: Tracks how people use email, Skype (meetings), and Yammer to communicate.
Collaboration: Tracks how people use SharePoint and OneDrive for Business.
Activation: Tracks the activation of Office 365 ProPlus, Project, and Visio licenses.

The current iteration of the content pack includes good coverage of Exchange Online, SharePoint Online, OneDrive for
Business, Skype for Business, Teams, and Yammer. It does not cover other applications like Stream or Planner, so the
dashboard covers usage of basic workloads rather than giving a comprehensive view of all activities within Office 365.
It is likely that Microsoft will build out coverage of the missing applications over time.

The dashboard helps administrators to understand the adoption rate and usage for various aspects of Office 365 over
the prior twelve months (Figure 21-22 ). For instance, how many people use SharePoint and how many use Exchange –
or all the covered applications. This kind of information can tell you if you have a license management problem where
people have licenses for an application that they do not use.
Figure 21-22: Graphs generated with the Microsoft 365 Usage Analytics content pack

Another use of the dashboard is to look at usage trends, such as whether email traffic increases over time. These
insights into what is happening within the tenant can help you to understand changes in user behavior. For instance, if
you make a determined effort to convince people to store documents in SharePoint and OneDrive and access the files
there rather than attaching them to email for circulation within the company, a steady uptick in file storage should
result. If it does not, then the campaign is unsuccessful, and you might need to take a different approach to convince
people to use “cloudy attachments”. Another example is in Skype for Business, where a reduction in email traffic
should match an increase in minutes consumed in instant messaging and conversations. Still another is to measure
whether the introduction of Teams causes users to move some of their collaboration with fellow employees away from
email to conversations within team channels.

Microsoft gathers data from across Office 365 to aggregate and refine the data to refresh the dashboard. The content
pack updates weekly by default, but the backend Office 365 service refreshes data daily, so a tenant can amend the
dataset scheduled refresh interval to receive daily updates. However, the actual data lags the current date by between
36 and 48 hours as Microsoft needs this time to gather information and process it for the dashboard. Given the scale
of Office 365 and the amount of data that Microsoft must process to generate aggregate statistics for tenants, this
delay is understandable. The setting for anonymized data for the reports available in the Office 365 Admin Center
governs whether usernames or anonymous identifiers appear in the dashboard.

For now, the Office 365 Power BI content pack offers limited coverage of what a tenant does with the service. Even so,
it is a good start and a useful addition to the reporting tools available to administrators.

Independent Reporting Products

Few tenant administrators are skilled at generating easy-to-read reports that highlight important data. Programmatic
access to the Reporting DataMart is available to allow software developers to use the data belonging to tenants, if
tenants grant permission for this access. The intention is to allow vendors who specialize in software monitoring and
reporting to incorporate Office 365 data into their products to give Office 365 customers with a much better view of
what happens inside a tenant than you can get using the basic reports. Apart from the obvious goodness inherent in
having a common repository to consult to retrieve operational data about Office 365, the combination of Reporting
DataMart and choice of APIs (including PowerShell) avoids the need for people to create their own ways to extract
information from Office 365.
Figure 21-23: Quadrotech Radar for Office 365

The number of independent software vendors who create reporting solutions for Office 365 continues to expand. Table
21-3 lists some of the better-known ISVs active in this area. Most deal with pure Office 365 environments while some,
like ENow Software, build off their extensive experience of monitoring and reporting on-premises servers to
incorporate reporting about components commonly deployed in hybrid configurations such as ADFS and Azure Active
Directory Connect. Because these products are independent of Office 365, you must pay extra, usually priced on a
cost per seat (licensed account) basis. Quadrotech (Figure 21-23 ) extends an interesting offer to small tenants as it
does not charge anything for tenants with fewer than 50 seats.

Company Product

Quadrotech Radar for Office 365

Exoprise Performance Monitoring for Office 365

Kaseya 365 Command

ENow Software MailScape 365

Syskit Syskit Monitor

CoreView Office 365 Reports

Table 21-3: Third-party reporting products for Office 365

Each product plays to its own strengths and employs different approaches to data extraction, analysis, and reporting.
Among the criteria you can use to figure out whether you need to use a third-party product instead of the standard
Office 365 reports include:

Data longevity : Office 365 stores tenant reporting data for 180 days. This might be insufficient for your needs if
 you want to chart the usage of Office 365 (and the individual services) over a longer period for planning and
analysis purposes. Ask for how long the vendor stores tenant reporting data. In addition, ask about the security
and privacy controls that are in place to ensure that tenant data cannot be compromised. Ask about how often
data is refreshed from Office 365 to understand how up-to-date the reports are.
The number and type of reports : A limit set of standard Office 365 reports are available in number and in
what they report. For example, there are no reports for mobile device usage, perhaps because Microsoft would
prefer tenants to use InTune for mobile device management. A third-party reporting vendor is likely to have
hundreds of different reports in their product.
Intelligent views : Looking at a single Office 365 workload is interesting if you want information about that
application, such as the number of active mailboxes in Exchange Online. Things become more interesting when
you can compare data extracted from one workload against another. For example, if you have a project to make
more use of OneDrive for Business, you will obviously be interested in the number of sites in use and the files and
quota used in those sites. It is also interesting to know if the growing use of OneDrive affects email volume and
usage.
Filtering and Pivoting : Viewing tables of data through a browser is acceptable for small tenants but rapidly
becomes problematic when the number of reported objects grows. For instance, it is difficult to make sense of a
table holding details of thousands of mailboxes. The ability to use filters and to view data through pivot tables are
huge advantages for large tenants.
Access : The reports available in the Office 365 Admin Center need some level of administrative access. This is
inconvenient when users other than administrators want access too. For instance, a company might want to
chargeback costs for Office 365 to the operating units of the business. This activity is usually performed by
accountants, not Office 365 administrators, so removing the requirement to have administrative permissions to
be able to access reports can be very useful.
Delivery : Not everyone wants to browse a web site to access reports. Some products allow reports to be
automatically extracted and emailed to users on a scheduled basis
Scale : How well does the product scale up to deal with the predicted full load of the tenant? Good demos
performed against a 5-user tenant might not be so impressive when confronted with the data generated by
20,000 users.

Coverage : How many reports does the product generate? How many of those reports do you think you need?
Are any obvious reports missing?
Applications : Is the product focused on any part of Office 365 or does it offer good coverage across Exchange
Online, SharePoint Online, OneDrive for Business, Yammer, and Skype for Business Online? How quickly is the
product updated to accommodate new Office 365 features such as Teams, Planner, and Office 365 Groups? Does
it support hybrid environments? Can it be extended to cover on-premises applications? Does the product use
other Office 365 sources such as Azure Active Directory to supplement the data retrieved from the reporting web
service?
Support : How is support provided? Is support available locally?
Cost : How is the cost of the product calculated?
Deployment : Is any extra software needed to create or view the reports? Is the product fully web-based or do
you have to install some software on a workstation or servers to access the reports?

No one product is the best choice for every situation. The best approach is to spend the time to test the software in
your own environment and make the decision based on what you discover there.

Gaps and changes : All third-party reporting products and Microsoft’s own reports use a single source of truth: the
reporting data collected for a tenant. Some third-party products copy that data so that it is retained for longer
periods and to perform additional analysis. However, you need to be aware of two issues about reporting data
originating from Office 365. First, it can change because Microsoft alters the way they gather data. This is what
happened when Microsoft changed the way that the Get-MailboxStatistics cmdlet reported system messages stored
in user mailboxes (described in Chapter 6). Second, hiccups and operational glitches sometimes stop an Office 365
workload providing data to the reporting data mart, which then creates gaps in reports. Sometimes the gaps are
filled in after normal service is restored, sometimes the gaps remain. Keep your eyes open and make sure that if a
gap appears, you make Microsoft and/or the third-party reporting vendor aware of the issue.

Service Accounts
In general, all reporting products use a service account belonging to a tenant to retrieve data for the tenant. A service
account is a special dedicated account that is authorized with the proper permissions (such as membership of the
Exchange View-Only Organization Management role group) necessary to access information from the Office 365
Reporting web service for a tenant on a regular basis.

Extracting More Detail About Tenant Activity

Office 365 provides a lot of information about how a tenant consumes resources, but the information is often quite
high-level and not as granular as you might like. In addition, the reporting service makes data available for a limited
timeframe, usually with a maximum horizon of 180 days, which makes the reporting information provided by the
service difficult to use for long-term planning. For example,
 Figure
21-24: Viewing the files usage report for SharePoint Online

illustrates the growth in SharePoint Online usage over time based on the number of files stored in sites. The report
can show data going back over a maximum of 180 days, but you cannot go back any further to explore questions such
as how much growth occurred in the last year or last two years. Note that the bottom part of the report lists the
individual sites and the storage that they occupy.

Figure
21-24: Viewing the files usage report for SharePoint Online

Reporting products often use PowerShell to retrieve more information from Office 365 and Azure Active Directory to
build out the data necessary to fill in the knowledge gaps. For example, it is all very well to know that a total of 55
distribution groups exist within a tenant, but it is even better to know the membership of each group and whether the
groups are in active use or just taking up dead space in the GAL. To make long-term reports possible and to assure
speedy access to information, reporting vendors often download tenant data periodically to repositories that they
manage. The data can then be used to analyze trends such as the growth in mailboxes or messaging activity over time.
Each product has its own set of reports and its own unique way to interpret and present information.

Because extra reporting capabilities are enabled by Azure Active Directory Premium, this can be included in the “add-
on reporting” category for Office 365. For example, if you want detail about password-based activity such as user
resets, you need an Azure Active Directory Premium license to access that report. For more information about the
reports that need a premium license, see the Premium Reports section under the Reports tab for Azure Active
Directory in the Azure portal.

Protection is Important
Given all this focus on auditing and reporting, we should consider how we can protect data. Chapter 22 examines the
topic of Data Loss Prevention and how some basic rules can help to educate users about the need to protect
confidential information. And if they do not learn, the same rules can enforce blocks on email and file sharing.
Chapter 22: Data Loss Prevention
Tony Redmond

It would be nice if users did everything right all the time, but that is not going to happen any time soon. It is human nature
to err, and one big error is to include confidential or sensitive data in email. Data Loss Prevention (DLP) is a technology
designed to help users to understand when they mishandle sensitive data. First introduced for Exchange, Office 365 DLP
policies cover Exchange, SharePoint Online, and OneDrive for Business too, with plans in place to extend coverage further
to protect all forms of data within the service.

Tenants deploy DLP through policies that capture the business requirements to protect sensitive data. This chapter
describes:

The concepts behind Data Loss Prevention (DLP).

How Office 365 implements DLP.
The differences between “unified” Office 365 DLP policies and those implemented through Exchange transport rules
(ETR).
How to build suitable policies to help Office 365 applications prevent the loss of sensitive data.
How to know whether your policy is effective.

Data Loss Prevention is not a magic bullet and its capabilities do not extend to every method that someone can use to
share data with another person. For instance, it is entirely possible that someone could paste sensitive data like a credit
card number into a Facebook conversation. You can handle that issue through third-party monitoring software, but user
education might be equally effective. DLP is just another part to fit into the compliance puzzle that contributes to the
overall security of information within companies.

Stopping Leaks
The need to prevent the loss (or leakage) of sensitive data such as credit card numbers, personal information like passport
numbers, and so on is well understood, possibly because there have been many instances in which company or personal
information has been compromised by being made public deliberately or accidentally. Graphic examples exist of where
millions of records held by large public companies were lost in some form. The result can be brand erosion, legal
expenses, customer loss, and an almost guaranteed public relations disaster. It is easy for a user to attach a document and
send it to someone else; it is also easy to make a mistake that ends up with information going to where it should not. For
example, you might not notice that Outlook has auto-completed an address with the wrong recipient and end up sending a
confidential attachment outside the company. The speed of modern email systems makes any attempt to recall messages
futile, so if someone realizes that they have made a mistake, all they can do is call the recipient and ask them to remove
the content from their system.

Companies devote enormous effort to protect IT systems. Much of the focus in the past has been on erecting traditional
security barriers to stop attackers penetrating internal systems. This approach might keep out hackers and scammers, but
it does nothing to prevent employees from causing data loss, usually by accident. DLP is not new and vendors have been
offering solutions for the last decade or so. Microsoft entered the space with Exchange 2013. Although a late entrant to
the DLP market, Microsoft enjoys certain advantages over third-party solutions because of their ability to deeply integrate
DLP checks into clients and servers:

Integration of DLP functionality to protect email into Outlook (both MSI and click-to-run variants). Outlook is a
critical desktop application in large enterprises. The integration allows text to be examined using a mixture of
algorithms including pattern matching and regular expressions as text is typed into a message body to detect possible
inclusion of sensitive data considered to be of concern to the company. OWA also includes the ability to perform DLP
checking. Both clients display policy tips to users when they detect sensitive data so that users become more aware
of the kind of data that they are dealing with before they send messages with that content.
Integration of DLP functionality into the desktop versions of other Office applications such as Word, Excel, and
PowerPoint. Although not as rigorous as the DLP checking in Outlook, inclusion of checks throughout the Office suite
means that users have less chance of making mistakes.
The ability to implement checking at points where a guarantee can be given that data will be examined. For Exchange
Online, checking runs in the transport system as all messages must pass through it before leaving the system or
delivered to internal recipients. Sensitive data can therefore be intercepted in messages by applying DLP transport
rules. For documents stored in SharePoint Online and OneDrive for Business, a crawler to detect sensitive
information checks files as users add them to document libraries. The crawler is already indexing all content stored in
SharePoint and OneDrive for Business sites, so adding a check for sensitive content there creates the desired
oversight.

The need to protect against the disclosure of confidential data does not exist in a vacuum. To be effective, it is important to
incorporate DLP into an overall corporate compliance strategy alongside other tools to protect data both inside and
outside the organization. Applying Office 365 sensitivity labels to encrypt and protect content help in this respect,
especially when circulating documents to third parties.

Office 365 displays policy tips when it detects DLP violations to emphasize that user education and policy communication
is very important. People must know why the policy tips appear and what they should do next. It is also important to stress
that DLP is not a guaranteed block against users sharing sensitive data. If someone wants to send content outside the
company, they will be able to find another way to do so.

DLP is a premium feature available to users with an Office 365 E3 (and higher) or Exchange Online E2 subscription.
However, DLP transport rules will block messages in violation of policy no matter with what license a user has. The
importance of DLP and its growing use within Office 365 is seen in statements made by Microsoft at the Ignite 2017
conference, when they reported that DLP use inside tenants had a year-over-year increase of 600% while the number of
Office 365 users protected by DLP grew by 750% in the year to September 2017.

Office 365 Data Loss Prevention

Microsoft’s goal is to implement a common DLP capability across all Office 365 workloads. The first step in the process
was to create the ability to apply DLP policies to information held in SharePoint Online and OneDrive for Business sites.
The next step was to extend the framework used for SharePoint Online to cover Exchange and so have policies that could
process the two major Office 365 workloads. In line with the general trend to develop software for the entire service, the
focus within Office 365 is now to extend the capabilities of these policies so that they cover all the data customers store
within Office 365. Like the other service-wide policies, you manage Office 365 DLP policies through the Security and
Compliance Center.

Covering multiple Office 365 workloads with a single DLP framework is a good goal for Microsoft to work towards.
However, it is important to recognize that Office 365 DLP policies have limited capabilities to protect leakage through
Exchange compared to ETR-based policies. Today, Office 365 DLP policies can detect the misuse of sensitive data and the
transmission of that data outside the organization, but that is not quite the same as being able to deploy the wide range of
conditions and exceptions supported by transport rules. The advantage of using Office 365 DLP policies is that you must
only manage a single set of policies. The downside is that you lose some functionality. That might not be a big deal if you
use relatively simple policies, but it will be if you explore more complex scenarios. The only way to be sure is to test.

Over time, Microsoft intends to close the functionality gap and make Office 365 DLP policies as capable as their workload-
specific counterparts. That work is already well started and some of the email-based conditions are supported by unified
policies. Although the unified policies are now the focus for DLP within Office 365, the workload-specific versions will
continue because they are needed for the on-premises servers. Because of the current disparity in functionality, Microsoft
cannot commit to when it will be possible for tenants to migrate away from ETR-based policies. The following factors
should be considered when considering moving away from ETR-based policies to Office 365 DLP policies.

Office 365 DLP policies are synchronized to Exchange Online but are not visible through EAC. You can only access
these policies through the Security and Compliance Center or PowerShell.
Office 365 DLP policies are not integrated into the Outlook Windows desktop client in the same way as the Exchange
DLP policies are. This means that Outlook does not detect policy violations during message composition. However,
the Exchange Online transport engine will block any messages when they are sent if they violate an Office 365 DLP
policy. Office 365 DLP policies are integrated into the OWA message compose screen and scan for policy violations
every time OWA saves a draft of a message.
The rules implemented in ETRs always take precedence over those in Office 365 DLP policies when processing email.
This is natural because ETRs are explicitly designed to process email. However, it also means that messages that are
blocked by an ETR will never be checked by an Office 365 DLP policy. Messages that pass through the ETRs are
processed by Office 365 DLP policies and could be blocked at this point if the rules detect a condition that calls for
this action. If a “stop processing” action is invoked by an ETR, it will not affect the processing of Office 365 policies.
Unlike ETRs, which have a priority order that can easily be manipulated by an administrator, the rules for Office 365
DLP policies are executed according to the creation date of the policy. In other words, the rules belonging to a policy
created on 1-Dec-2016 are executed before the rules of a policy created on 2-Dec-2016.
Office 365 DLP policies do not support the custom sensitive data types created with digital fingerprinting . If you
have ETRs that depend on these data types, Office 365 DLP policies cannot offer a similar solution. You can create
custom sensitive data types (defined as an XML document to define the characteristics of the custom data type) to
describe data types such as employee numbers or identifiers for use with Office 365 DLP policies and use the data
types with Exchange, SharePoint, or OneDrive for Business. Creating an XML rules package for a custom sensitive
data type is not as easily as scanning a document to create a fingerprint, but it is a potential solution.

DLP is an area under constant development and new or altered features can appear at any time. For this reason, it is
sensible to confirm the effectiveness of your policies with a periodic check.

Sensitive Data Types

The basis for a DLP policy is knowing what kind of sensitive data you want to protect and how you want to protect the
data. Several common data types come to mind when you consider what kind of information you prefer users to not
mishandle. Credit card numbers or passport numbers are obvious examples of data usually regarded as sensitive.
Although credit cards use a worldwide format, personally identifiable information (PII) and other sensitive data types
differ from country to country.

It would not make much sense if Microsoft gave tenants a blank sheet to define their own view of how to describe sensitive
data. Mistakes would happen, and inconsistencies in how different forms of sensitive data were described would abound.
Fortunately, Office 365 comes with over 80 definitions for sensitive data that you can use immediately within a DLP policy
without first having to figure out what essential characteristics might best describe a particular type of data. For example,
there are definitions for:

ABA routing numbers.

Credit card numbers.
U.S. Social Security numbers.
Canada bank account numbers.
European Union debit card numbers.
 Australian passport numbers.
German driver's license numbers.
European Union common sensitive data types such as national identity numbers.

Over time, it is likely that the inventory of sensitive data types will expand to cover other types of data used in other
countries. You can see the current set of sensitive data types (or classifications) used by Exchange Online by running the
Get-DataClassification cmdlet:

[PS] C:\> Get-DataClassification | Sort-Object Name | Format-Table Name, Description –AutoSize

Name Description

---- -----------

ABA Routing Number Detects ABA (American Bankers Association) routing number

Australia Bank Account Number Detects Australian bank account number

Australia Driver's License Number Detects Australian driver's license number

Australia Medical Account Number Detects Australian medical account number

Australia Passport Number Detects Australian passport number

Australia Tax File Number Detects Australian tax file number

Canada Bank Account Number Detects Canadian bank account number

A sensitive data type is defined by describing its characteristics in the form of some recognizable pattern, often captured
in a regular (regex) expression. For instance, a social security number is a nine-digit number usually formed in three
groups of three, two, and four digits separated by hyphens. Many passports have some alpha characters followed by seven
digits, and so on.

Custom Classifications
Although a large set of out-of-the-box sensitive data types exist, organizations usually have their own idea of what sensitive
data might be. You can therefore create your own custom sensitive data type through the Classifications section of the
Security and Compliance Center. Custom sensitive data types can be used in DLP policies, Office 365 retention policies, or
auto-label policies.

To begin, you need to know how to describe the sensitive data type through a regular expression and some extra evidence
to give confidence that data in the defined format is sensitive and not just a random collection of letters and numbers. For
example, as noted above, the social security number data type is described by a number in the format 999-99-9999 with
added evidence coming from the keywords “SSN” or “Social Security” near (in terms of characters) the number. The same
is true for credit cards, where the 16-digit number is confirmed by words such as “credit card,” “Visa”, ”MasterCard”,
“expiry date”, and so on.

Let’s assume that we want to use customer project numbers as a sensitive data type. Project numbers use the format AAA-
9999, and terms like “project number” can be the added evidence. In Classifications, click Custom sensitive and then
Create , and enter the name of the new custom type and a brief description. You then set up the definition for the new
type.

Figure 22-1 shows what our definition might be. The key thing is the pattern used to identify the new data type. In this
case, we use a regular expression of \s[A-Z]{3}-[0-9]{4}\s to pick up project numbers like “PRJ-1384” or “PRO-1847.” This
expression is called the primary pattern element. (Sites like regex101.com are useful to help form regular expressions).

Notice that the confidence level is set to 80%. This means that DLP is 80% confident that it matches the sensitive data
type if it detects the primary pattern in content. To increase the confidence level even higher, we can include a keyword
list to provide supporting evidence. Input the keywords (you can use different combinations that make sense to you) in a
comma-separated list with the keywords in quotes. The Proximity value at the top is set to 200 characters, meaning that a
match is found if the pattern is detected within 200 characters of a keyword.
Figure 22-1: Definition for a custom sensitive data type

When you save a custom definition, Office 365 offers you the choice to test it. You need at least two text files for testing.
One should contain a set of values that you expect the sensitive data type to detect; the other contains values that it
shouldn’t. For a more comprehensive test, you can create a suite of text files to check various combinations against the
new type.

Browse to select a file for testing and then click Classify . Office 365 opens the test file and checks its contents against
the custom sensitive data type just like it would in a DLP policy. You’ll then see the results of the test and the matches
(Figure 22-2 ). Once the new custom sensitive data type passes its tests, you can go ahead and deploy it like any of the out-
of-the-box types. You’ll also see it show up in PowerShell with your tenant listed as the publisher. In this output, we see the
sensitive data type we just defined together with others created by document fingerprinting for use with Exchange DLP
policies (covered later).

[PS] C:\> Get-DlpSensitiveInformationType | ? {$_.Publisher -ne "Microsoft Corporation"} | Format-Table Name, Publisher

Name Publisher

---- ---------

U.S. Tax Documents Office365ITPros.onmicrosoft.com

U.S. Tax Form W-8BEN Office365ITPros.onmicrosoft.com

U.S. Tax Form W-4 (2015) Office365ITPros.onmicrosoft.com

U.S. Tax Form 1040 (2014) Office365ITPros.onmicrosoft.com

U.S. Tax Form 1040EZ Office365ITPros.onmicrosoft.com

Customer Project Identifier Office 365 for IT Pros

Figure 22-2: Testing a custom sensitive data type

Office 365 DLP Policies

To understand how Office 365 DLP policies work, we’ll go through the process of setting up a new policy and see what
happens when the policy is operational. After we finish with Office 365 DLP policies, we will focus on how DLP policies for
Exchange use transport rules for their implementation.

Checking Documents for Sensitive Data

Despite the obvious differences that exist between processing email and documents, the same need exists to find a single
point in the system to apply policies in a reliable manner. The use of transport rules for Exchange DLP policies guarantee
that the transport system can apply policies to all messages as they pass through. A similar natural chokepoint does not
exist inside SharePoint Online, so Microsoft needed to take a different approach to offer protection for SharePoint and
OneDrive for Business sites.

To ensure that it can detect DLP violations when present in documents, SharePoint Online uses a mixture of real-time
policy evaluation together with a special “crawler” process and a background timer job to find documents that have
sensitive data. Microsoft refers to this as a mixture of synchronous and asynchronous checking that applies equally well to
new documents as users add them and existing documents as users update content.

Synchronous or real-time evaluation does not mean the kind of in-place checking that occurs in Outlook when the body of
messages is examined for sensitive data as the user inputs text. Nor is it like the OWA implementation where message
content is sent to the server for examination on a regular basis. Real-time checking for SharePoint and OneDrive
documents means that the content is examined when documents are added, changed, or shared.

If SharePoint detects a violation during real-time examination, it displays the policy tip set in the policy, but only in
applications that support the necessary user interface (at the time of writing, only the 2016 versions of Word, Excel, and
PowerPoint can display DLP policy tips). Policy tips also appear in the browser interface to sites. Policies might allow users
to override the blocks imposed on documents holding sensitive information. If so, users can do this through the browser by
opening the document properties, viewing the policy tip, and clicking Override (see Figure 22-5 ). In the document
properties, you can see the time that the crawler checked the document and discovered the violation under “Recent
Activity”, where the action appears as an edit performed by the “System Account.”

Crawlers create content indexes by scanning all the document libraries in sites to look for new or changed material to
index. The DLP crawler looks for sensitive data types in new or changed documents and can detect them soon after an
update occurs (as fast as indexing occurs). However, background processes are subject to throttling and checking might
not happen if servers are under a heavy processing load. When load reduces, the crawler detects any lingering violations
soon afterwards as it processes items for content indexing. Excel posed a difficulty in terms of how it stored data in
spreadsheets and Microsoft had to create a special file handler to ensure that DLP could detect sensitive data types in
these files.

When SharePoint detects a violation, it applies the rules specified in the DLP policy. If the policy rules block access to
items with sensitive data outside the organization, SharePoint and OneDrive for Business incorporate the check into the
file sharing dialog so that users cannot commit DLP violations (Figure 22-3 ). The idea is that prevention at the outset is
better than fixing a problem after a violation occurs.
Figure 22-3: DLP blocks a user sharing a OneDrive for Business file

The exception is that SharePoint does not generate and send incident reports when the crawler detects a violation in
content that existed before a rule became active. If SharePoint generated incident reports when it scanned content after
an administrator adds a new rule or updates an existing rule, it is easy to imagine how the results might be a mail storm of
thousands of incident reports. In a practical sense, it would be impossible to go through all the incident reports and
resolve all the newly-detected violations. DLP still enforces any blocking actions contained in the rules; suppression occurs
only for the incident reports.

Default DLP Policy

It is always good to have a building block to start with when approaching the introduction of new technology. To help
tenants start with DLP, Microsoft includes a default DLP policy to protect credit card information for Office 365 tenants
where DLP is not already in use. Credit cards are the most common sensitive data protected by active DLP policies, so
they are a natural choice for companies to start their DLP journey. If your tenant has a default DLP policy, you must edit
and enable it to make the policy active.

Creating Office 365 DLP Policies

After becoming used to DLP policy structure and operation, the next step is to create a policy tailored for your
organization. You can edit the default DLP policy or create a new one. To create a new Office 365 DLP policy, go to the
Data Loss Prevention section of the Security and Compliance Center and click Create a policy. Figure 22-4 shows two
policies in place within a tenant, one of which is active (status = “On”) and the other is being tested.
Figure 22-4: The DLP section of the Security and Compliance Center

Before you go ahead and create a new policy, it is worthwhile to chart out what kind of sensitive data you want to protect
and where are that data kept. Every DLP policy follows the same basic structure:

Locations : What parts of Office 365 to protect. You can choose from Exchange, SharePoint, and OneDrive for
Business. Office 365 DLP policies do not currently cover Teams or Planner, but the SharePoint content associated
with these apps are protected.
Rules : What kind of protection you want to achieve. Rules specify what kind of sensitive data you want to protect,
conditions that say what must happen before a violation occurs, and actions that the policy takes afterwards. Actions
include user notifications and the creation of incident reports.

You can then start to flesh out the policy by inputting the different sections of the policy. The wizard used by Office 365 is
divided into these parts:

The information to protect : To make it easier to create policies, Office 365 gathers sensitive data types into sets of
data that organizations commonly must protect in certain fields of activity. For instance, if you want to avoid the loss
of personally sensitive data, you can select Privacy to see sets of data such as “U.S. Patriot Act” or “U.K. Data
Protection Act”. Each set is a template for a policy and when you select a set to use as the basis for a policy, Office
365 automatically creates the necessary DLP rules to protect the information covered by the template. If none of the
sets work for you, select Custom , which allows you to construct your own rule set.
Name your policy : If you select a template, its name is used as the policy name. You can overwrite the name with
your own choice. You can also enter a description for the policy. Some organizations used this to refer to internal
documentation for the policy, including information about who created the policy.
Choose Locations : You can opt for “All locations in Office 365,” meaning all Exchange Online mailboxes and all
SharePoint Online and OneDrive for Business sites, or you can select to include or exclude specific SharePoint sites,
OneDrive for Business accounts, or Exchange mailboxes (you can use a distribution list to specify the mailboxes to
include or exclude). When you opt to include or exclude specific SharePoint or OneDrive locations, you enter the URL
for the user sites that you want to protect (up to a maximum of one hundred sites). Note that the administrators who
create DLP policies need to have at least view-only permission to the sites to which the policy is to apply. Microsoft
suggests that you create a special security group to hold compliance administrators and that you add this group to
the Visitors group for each site collection over which a policy applies. Separate steps are necessary to assign
eDiscovery permission for OneDrive for Business sites because these sites are for personal use. Apart from using
distribution lists as a one-time mechanism to add mailboxes to a policy, DLP policies do not support scoping, which
would allow you to say that a policy only applies to a certain group of users based on a filter applied against Azure
Active Directory.

Policy settings : Define the conditions under which Office 365 invokes the policy and the actions that Office 365
takes, including whether to generate DLP incident report and what information about a potential violation to include
in the report. You can select the default settings to apply the processing copied from the template to handle potential
violations when Office 365 detects a violation in content shared information inside or outside the organization, or
Advanced settings , which allows you to access the individual rules within the policy. A policy created from a
 template is likely to have at least two rules, each of which might need your review to ensure that the settings in one
rule does not interfere with settings in other rules. We will discuss the different settings in more detail later.
Enable policy : Define whether the rule is on or off, or if it is in test mode.
Review your settings : Before you save a policy, you can review its settings to ensure that everything is as you
expect. Click Create to continue.

When you create a new DLP policy or update an existing policy, Office 365 checks the policy to ensure that it is valid. For
example, you might have input the URL for a specific site instead of a site collection. If this happens, Office 365 flags the
error and you must fix the problem before you can save the policy. Once everything is satisfactory, Office 365 provisions
the new policy to the various workloads to enact the policy. A certain amount of complexity is involved to publish a policy
to all the workloads as multiple servers are involved. Microsoft’s SLA is for a new or updated policy to be effective within
an hour of publication, but the experience of many is that this SLA is often breached and that you might have to wait up to
a day before everything settles down and new DLP rules are active.

A further delay occurs before workloads begin to detect violations. SharePoint relies on the crawler within the content
indexing process to detect sensitive data within documents. SharePoint does not index content immediately after an item
is updated and (depending on system load) the crawler might take between ten minutes to an hour before it processes an
updated document.

Much of the flow of creating and managing Office 365 DLP policies follows the same approach as Exchange Online DLP
policies, including the ability to create a new policy from one of the templates supplied by Microsoft. These templates span
the same collections of sensitive data types supported by Exchange Online such as the "U.S. Personally Identifiable
Information (PII) Data" collection. You can also create a DLP policy from scratch and add in the sensitive data types
against which you want to check.

Office 365 DLP policies do not use transport rules as their chokepoint to detect sensitive data in Exchange. Instead, Office
365 checks items during the content indexing process. Office 365 DLP policies do not have the same range of processing
possibilities supported by transport rules. Note that if a document violates multiple DLP policies, DLP applies the most
stringent policy on the basis that it is better to be safe rather than sorry.

Mobile Policy Tips : The OneDrive for Business mobile apps for Android, iOS, and Windows Mobile support DLP policy
tips. The clients download the DLP policy from Office 365 when it connects, and the application is then able to scan text
input into documents for sensitive data types and, if detected, display the correct policy prompt.

Default Rule Settings

If you choose not to use the advanced settings to customize the rules copied into a policy from a template, Office 365, the
policy applies common settings for all rules. These settings cover:

How Office 365 notifies users when a policy violation occurs . It is usual to allow Office 365 display policy tips
(like the one shown in Figure 22-5 ) to users to tell them that a problem exists with a document.
How many instances of sensitive data exists in an item before Office 365 flags a violation . The default is
that 10 instances of the same sensitive data type (for example, a credit card number) must be present in a file or
message before a violation occurs. You can reduce or increase the number as necessary. If you want to make a rule
easier to match, reduce the instance count (for example, from 10 to 7). Likewise, to make it harder to match, increase
the count.
Who receives the incident report and what is in the incident report . Typically, incident reports go to a DLP
administrator, who checks the incident to assess whether it is valid and, if so, acts to resolve the incident.
Who can access the content and whether users can override policy . If you decide to restrict access to content
that violates a DLP policy, you can control who can continue to access content while it contravenes policy. As shown
in Figure 22-5 , The normal situation is to block external access to the content while allowing internal users
continued access. You can also decide whether users can override policy if they give a business justification

Figure 22-5: Customizing the access to content setting for a rule in a DLP policy

When you opt to block external users from seeing sensitive content that violates a DLP policy, the SharePoint and
OneDrive sharing dialog changes to include a warning to users if they try to share sensitive content outside the tenant.
The warning appears in real time and helps to educate users about the need to protect sensitive content.

Advanced Rule Settings

When you opt to use Advanced Settings for a DLP policy, you can tailor the rules copied from the template (or created
from scratch) to meet your exact needs. The settings for rules are:

Rule name : Free-text name assigned to the rule.

Conditions : The type of sensitive data present in content and the number of instances of that data that cause the
rule to fire. For example, a rule might ignore content where a single credit card number is present but fire when it
finds more than three instances of credit card numbers. You can also define whether the rule checks for information
circulating inside the organization or when shared externally. You can also add some email-based characteristic,
including specifying the sender’s IP address, what the recipient domain must be, and what kind of file an attachment
is. Figure 22-6 shows how to change the number of detected sensitive data instances for a policy to check (in this
case, the minimal number of U.S. Social Security Numbers that exist in an item must be 2 to fire the rule). The
Report an issue link allows the user (because of a policy setting) to give a business justification to override the
policy. When a user justifies the presence of sensitive data in a document, Office 365 records their reasons and
releases the block on sharing.
Exceptions : Define whether any conditions exist that will cause the rule not to fire. The exceptions include if content
has a specific form of sensitive data (or label), the sharing of content inside or outside the organization, or a
characteristic of email (IP address, attachment type, and recipient domain).
Actions : What happens when a violation occurs. The action is to restrict access to the content by removing
permissions for everyone except the primary site collection administrator (or global administrator), site owner, and
the person who last changed the document. Office 365 automatically restores the original permissions when the
owner or person who last changed the document acts to remove the offending content and bring the document back
into compliance. Encryption is available to protect email with sensitive content, but only for policies scoped to apply
to Exchange content. See the note below.
User notifications : DLP supports two forms of notifications. First, Office 365 can send email notifications to the site
owner and person who last changed the document to inform them that a problem exists and why (for instance,
someone shares the item with external people and includes a debit card number). A link in the message allows the
recipient to open the document and fix the problem detected by the policy. The second method is to flag the problem
visually with a policy tip. You can customize the text of the policy tip to add your own advice to users.
User overrides : The rule can allow users to override the block because of a false positive. In other words, the user
(who knows the content and can put it into business context) says that the rule is incorrect to flag a document as out
of compliance. When this happens, Office 365 removes the block and records the text given by the user to justify the
override for auditing purposes.
Incident reports : You can arrange for email notifications to go to one or more people when a policy detects
problems. These notifications are known as incident reports. Each report has enough information about the problem
to allow an administrator or compliance officer to understand whether a problem really exists and if so, whether they
need to take further action.

Figure 22-6 also illustrates the ability of a DLP rule to combine and/or checking for combinations of sensitive data and
Office 365 labels. In this instance, a policy copies rules from the U.S. HIPAA template. Two groups are present initially (PII
Identifiers and Medical Terms) and the rule checks for content that holds both PII data and medical terms (as defined in
some large medical dictionaries of disease classifications available within Office 365). A change made to this policy uses
the Add group button to create a new group called “PII Confidential Information” and specifies that the content must also
have the Confidential label. Although no one is likely to use the combination of rule sets and labels shown here in practice,
it does illustrate the powerful checking capability that exists within DLP policies.

You can alter the settings for each rule in a policy. When you finish working with the settings for a rule, click Save to
return to the rule set so that you can edit the next rule. When all the rules are to your liking, click Next to return to the
policy.
Figure 22-6: Specifying notification settings for an individual rule in an Office 365 DLP policy

Confidence Levels
DLP policies match data found in documents and email against sensitive data types by looking for patterns and other
forms of evidence. Combining a pattern and other evidence builds confidence that data found is the kind that the policy is
looking for. The higher the percentage confidence level specified by a rule, the more evidence Office 365 must find before
it can match a rule. For instance, at 65% confidence level, a rule matches a credit card number if it finds a number in the
right format (sixteen digits) that passes Luhn’s algorithm to establish that the number could be a credit card. To increase
the confidence that the number is a credit card to 85% confidence, the rule looks for other evidence like a keyword (such
as “Visa” or “Amex”) or an expiry date in the MM/YY format.

If a policy includes multiple rules that check for the same sensitive data type, the suggestion is to:

Set the minimum and maximum confidence level for the rule that takes the least aggressive action to a number such
as 65 (percent).
Set the confidence level range for the rule that takes more aggressive action from a point just above the first rule (in
this case, 66) up to 100.

The idea is that content with lower confidence matches get an action with less impact on the user (like displaying a policy
tip) while content with higher confidence matches get a more restrictive action (like being blocked).

Policy Tips
Figure 22-7 shows how a DLP policy tip appears to a user when a problem after a problem is found in a document. The
issues that the user must deal with are clearly shown.
Figure 22-7: A policy tip flags a document stored in OneDrive for Business

Overriding a policy tip through the Report an issue link allows a user to go ahead and share a document when a DLP
policy would otherwise blocks this action. SharePoint records overrides in the Office 365 Audit Log in a “DLPRuleUndo ”
audit entry. The ExceptionInfo section of the audit entry captures the override text:

"Reason": "Override",

"Rules": [

"413957f5-c3ab-4765-9322-33d2983dabfe"

],

"Justification": "This document is authorized for sharing!"

Rule Priority
Each rule in a policy receives a priority number in the order in which it is created. You cannot change the priority order of
a rule except by removing it and then recreating it (this process might involve the removal and recreation of several rules
to get them in your preferred order). When Office 365 evaluates content against a policy, it processes the rules in priority
order; if content matches multiple rules, Office 365 applies the most restrictive action. For example, if rule 1 displays a
policy tip to users and rule 6 restricts access to the content, Office 365 applies rule 6. In addition, the policy tip for the
most restrictive rule is displayed to the user.

Protecting Sensitive Email with Encryption

If DLP detects outbound email with sensitive content, you can have Exchange apply encryption to the messages before
they leave your tenant. You can choose to apply any of the Azure Information Protection templates available in the tenant
to messages, including the default Encrypt Only and Do Not Forward templates. See Chapter 24 for more information
about how to create and use Azure Information Protection rights management templates to protect documents and email.

If you choose to apply encryption to messages in a DLP policy, that policy can only cover Exchange. You can’t apply
encryption to documents through a DLP policy.

GDPR Support
Tenants that come with the scope of the European Union’s General Data Protection Regulation (GDPR) need to meet
specific requirements for the protection and use of personal data. To help, Office 365 includes six common sensitive data
types that can be included in DLP policies or used to classify sensitive data. The data types are:

EU Debit Card Number.

EU Driver’s License Number.
 EU National Identification Number.
EU Passport Number.
EU Social Security Number (SSN) or equivalent ID.
EU Tax Identification Number (TIN).

In addition, Office 365 includes a General Data Protection Regulation DLP template. If you create a policy based on this
template, the six data types described are included in the policy. You can edit the policy to include other sensitive data
types or include the EU data types in other policies.

The difficulties involved in detecting sensitive data types were illustrated when emails imported into SharePoint Online
caused Office 365 to flag DLP violations for the new EU TIN because eight- and nine-digit numbers in the email headers
resembled tax identification numbers. Microsoft resolved the problem by adjusting the data definition to look for more
proof that a number was a TIN before flagging a violation.

Sample Test Data

When you start working with DLP, the issue of how to generate good test data to use to check policies always arises. The
DLPtest site offers several different sets of sample data, including social security numbers and credit card numbers, that
you can use for testing.

PowerShell support for Office 365 DLP policies

Office 365 DLP polices uses several cmdlets (included in the Compliance Center module) to work with DLP policies and
rules. The cmdlets include:

New/Get/Set/Remove-DlpCompliancePolicy : Create, access, manipulate, and delete Office 365 DLP policies. For example:

[PS] C:\> Get-DlpCompliancePolicy –Identity 'Confidential Patent Information DLP Policy'

New/Get/Set/Remove-DlpComplianceRule : Create, access, manipulate, and remove the rules used in Office 365 DLP
policies. For example, here is how to check the settings for a rule (edited output shown):

[PS] C:\> Get-DlpComplianceRule –Policy 'Check for SSN Data' | Format-List

ParentPolicyName : Credit Card data check

ContentContainsSensitiveInformation: {System.Collections.Hashtable, System.Collections.Hashtable,

System.Collections.Hashtable}

AccessScope : NotInOrganization

ContentPropertyContainsWords : {}

Workload : Exchange, SharePoint, OneDriveForBusiness

BlockAccess : True

GenerateIncidentReport : {DLPIncidents@ Office365ITPros.com }

IncidentReportContent All

NotifyUser : {SiteAdmin, LastModifier, Owner, Tony.Redmond@ office365itpros.com}

NotifyAllowOverride : WithJustification

NotifyEmailCustomText : You can't keep this kind of information in documents!

NotifyPolicyTipCustomText : Whoops - bad SSN data found here

ReadOnly : False

Priority : 0

Comment : A rule to detect SSN data in documents

Disabled : False

CreatedBy : Tony Redmond

We can tell that this rule:

Is associated with the DLP policy “Credit Card data check”.

Looks for three types of sensitive data (the hashtable references in the ContentContainsSensitiveInformation property
– the names are not shown).
Checks information shared with users outside the tenant (scope is "NotInOrganization ").
Scans for all three supported workloads (Exchange, SharePoint Online, OneDrive for Business)
Blocks access to content deemed to contain the sensitive data.
 Generates a DLP incident report.
Generates an email notification to the site administrator, last user who modified the content, its owner, and a
nominated other user.
Allows the policy tip to be overridden with a business justification.
Is active (Disabled is False).

Given that DLP policies can be reasonably complex to set up, it is best to manage Office 365 DLP policies and rules
through the Security and Compliance Center and only use PowerShell to check individual objects as needed.

Exchange DLP policies

Like Office 365 DLP policies, Exchange transport-rule based DLP policies (ETRs) are composed of user education (policy
tips) and content recognition to detect sensitive data. Users are perfectly capable of ignoring policy tips and sending
messages anyway and many messages are sent using clients that know nothing about DLP, such as mobile clients or older
versions of Outlook. These messages all must transit though the transport system, which is the real chokepoint for
blocking, should that be defined by policy.

To detect problems in content, Exchange uses a deep content analysis engine that can check for sensitive data types in
message bodies and attachments (to some 30 levels deep). The engine runs within the transport system and a modified
version is available inside Outlook. The content analysis occurs as the user enters text into the Outlook compose message
window and is enough to detect whether a problem might exist. A much deeper content analysis occurs when messages
are processed by transport rules. The analysis combines several analytical methods to find content including keyword
searches, pattern matches, searches against dictionaries, and specific tests for certain items such as credit card numbers.

An Exchange DLP policy is essentially a container holding a set of specialized transport rules to check messages as they
pass through the transport pipeline. Transport rules incorporate DLP policy checking through the condition "if the
message contains sensitive data," which tells the rule to check for a certain type of sensitive data together with what to do
if the rule encounters sensitive data.

Using transport rules means that tenants have assurance that DLP checking will process any message sent from any client
for compliance with company policy and blocked if whenever a rule finds a violation. In addition, details of messages with
sensitive content can be captured in incident reports and analyzed later to ensure that the policy is working and that users
are not including an inappropriate amount of sensitive information in email. Other transport rule actions such as copying
messages to a user’s manager can also be used to increase awareness of the compliance policy in both employees and
management. The mixture of soft education through visual clues and hard enforcement through transport rules is effective
at telling users why blocks are in place and ensuring that sensitive information does not leave the organization.

As we will explore in the example described later, you can implement a DLP policy in a single rule to check for a single
sensitive data type. However, it is more common to find that organizations deploy more complicated DLP policies
composed of multiple rules, each of which checks for a specific sensitive data type under different conditions. For
example, one rule might capture audit data for messages that have credit card information sent to internal recipients
while a second rule blocks messages having the same credit card information but covering the condition when messages
go external recipients. The logic here is that it might be OK to circulate sensitive information internally but not so good to
allow it to go outside. Another common rule found in DLP policies is one that blocks messages with unreadable
attachments. Exchange Online can decipher most file formats in widespread use, so if a strange format or password
protected attachment is presented it could be a sign that someone is trying to hide sensitive information. DLP is capable of
checking attachments many levels deep in a message and can detect attempts to hide information contained in a zip file in
a zip file in a zip file and so on.

Designing an Exchange DLP policy

When you create a new ETR-based DLP policy, you can choose a template and Exchange Online will populate the new
policy from the settings in that template. You can alter the policy afterward by removing unwanted rules or adding new
rules as your needs dictate.

The steps in defining a new DLP policy are:

1. Decide the types of sensitive data to protect.

2. Decide the set of rules needed to protect sensitive data:

1. Conditions (recipients inside or outside the organization, what kind of sensitive data is involved).
2. Actions (block or pass, allow override, generate incident report).
3. Exceptions (are all users subject to the same policy).

3. Create a new DLP policy:

1. Based on a Microsoft template.

2. Created from scratch.

4. Test and refine the DLP rules.

5. Ensure that the DLP rules do not interfere with other transport rules.
6. Enforce the DLP policy.
7. Audit the DLP policy by checking incident reports and DLP reports.

The starting point for defining any policy is to understand what the policy is designed to do. DLP policies protect the
company against inadvertent disclosure of sensitive data, so the first step is to decide what types of sensitive data need to
be protected. You then must think about the conditions that you want to check for, whether any exceptions exist, and what
action to take if a policy violation is detected. Because the implementation of a DLP policy might affect users, it is sensible
to get buy-in from the HR and legal departments, and to inform users about the new policy when implementation time
comes around.

Blocking Actions
The question that you need to answer is what should happen when sensitive data is detected in messages. The set of
available actions that can be specified in a DLP rule is described in Table 22-1 . The NotifySender value is used with
PowerShell to create a new DLP rule (New-TransportRule ) or to amend the action specified in an existing rule (Set-
TransportRule ).

Action Meaning NotifySender value

Notify Notify the user that a problem exists because sensitive data has NotifyOnly
been found in a message, but allow the message to be delivered
as normal.

Reject and block Transport will block the message and notify the user with a non- RejectMessage
delivery notification.

Reject unless false Transport will block the message unless the user marks it as a RejectUnlessFalsePositiveOverride
positive false positive.

Reject but allow Transport will block the message unless the user overrides the RejectUnlessSilentOverride
override policy and decides to send the message even though some
sensitive data is detected.

Reject but allow Transport will block the message unless the user overrides the RejectUnlessExplicitOverride
override with policy and gives a business justification that can be logged for
justification auditing.

Table 22-1: Available DLP actions

Policy Modes
Exchange Online supports three modes for DLP policies. The modes are:

Test without policy tips : This is the default mode when a new policy is created. It allows the transport system to
check messages for the sensitive data types dictated by the policy and to track the actions that would have been
taken if the policy was enforced. This data can be viewed through DLP reports.
Test with policy tips : This mode exposes policy tips to Outlook clients to allow you to know whether the policy tips
have the desired effect of informing users about the need to protect sensitive data. Actions associated with the rules
are not applied. Some functionality does not work in test mode. For example, you cannot override a rule by giving a
business justification.
Enforce : The actions in DLP rules are active and enforced as messages pass through the transport system.

If you change the mode for a policy, all the rules associated with the policy are updated to reflect the new mode. However,
you can selectively set a different mode for a rule by editing its properties. The processing of an individual rule can also be
limited by a start and end date.

DLP policies also have an enable state. When enabled, the policy is in force according to its mode as described above.
When disabled, the rules associated with the policy are disabled within the transport pipeline.

Starting with a DLP Template

Although you can create a DLP policy from scratch to reflect the customized needs of a company, it is often easier to start
with a template, especially when your policy is going to be based on the need to satisfy a certain set of regulations.
Microsoft offers Office 365 tenants a set of DLP templates to help them build a DLP policy for common-use scenarios. Each
template is designed to meet the requirements set out in well-known scenarios describing the protection of sensitive data.
For example, the Australia Financial Data template “Helps detect the presence of information commonly considered to be
financial data in Australia, including credit cards, and SWIFT codes, ” while the U.S. Health Insurance Act (HIPAA)
template “Helps detect the presence of information subject to United States Health Insurance Portability and
Accountability Act ​(HIPAA)​, including data like social security numbers and health information.“ Each template contains a
set of DLP-enabled transport rules to check for the data types covered by the chosen national regulations. Third parties
can create suitable DLP templates designed for use in certain industries or circumstances. These templates can then be
given to customers in an XML format and imported to create a new DLP policy. To illustrate the rules that are created
when a template is used to instantiate a DLP policy, Table 22-2 describes the five rules created by using the standard “U.S.
Patriot Act” template.

Rule Condition Actions

order

1 Message has the word “Override” Set the special message header X-Ms-Exchange-Organization-Dlp-
in the subject. SenderOverrideJustification to “TransportRule override ” to make the override
known to rules that process the message later in the pipeline.

2 The message Is sent to an Set the audit severity level to “Medium” and notify the user that the message
external recipient and has a low violates a DLP policy but allow the message to be sent.
(less than 4) count of these
sensitive data types:

Credit card number

U.S. Bank Account Number

U.S. Individual Taxpayer

Identification Number (ITIN)

U.S. Social Security Number

(SSN)

3 The message Is sent to an Set the audit severity level to “High” and notify the user that the message
external recipient and has a high cannot be sent because of the sensitive data in its content. Allow the user to
(4 or more) count of these give an override. If an override is not provided, reject the message with a DSN
sensitive data types: status code 5.7.1.

Credit card number

U.S. Bank Account Number

U.S. Individual Taxpayer

Identification Number (ITIN)

U.S. Social Security Number

(SSN)

4 The message has an attachment Set the audit severity level to “High” but allow the message to be delivered.
that cannot be fully processed for
some reason.

5 The message has an attachment Set the audit severity level to “medium” but allow the message to be delivered.
that cannot be opened.

Table 22-2: Rules created by the U.S. Patriot Act DLP template

You might decide that the rules created from a template are insufficient or unsuitable for your DLP policy. If so, you can
edit rules to add, remove, or change conditions and actions, remove rules, and add your own rules to the base set
inherited from the template.

DLP rules are executed alongside other transport rules as messages pass through the transport pipeline. Although an
organization can use multiple DLP policies, this can create confusion and the possibility that rules might interfere with
each other. For this reason, it is recommended to use a single DLP policy that encompasses all the sensitive data types of
concern to the organization and to make sure that the DLP execute without causing problems for other transport rules.

Creating a New DLP policy for Exchange

DLP policies for Exchange Online are created and managed through the compliance management section of the EAC. Here
you can create and edit DLP policies and the transport rules associated with those policies. Underpinning the EAC is a set
of Exchange PowerShell cmdlets such as New-DLPPolicy that you can use to programmatically create and control DLP
policies should you prefer. The normal *-TransportRule cmdlet set is used to update the transport rules linked to DLP
policies.

Before we get into the details of how to create comprehensive DLP policies, we can examine what happens when a very
simple policy consisting of just one rule is put in place. We can create this policy with two PowerShell commands. The first
creates the DLP policy, the second creates a new transport rule to check for credit card data and associates the rule with
the policy. The rule blocks any message found to have credit card numbers unless the user gives an override reason. We
enable the policy by setting its mode to “Enforce”. The scope of the rule means that it only fires when messages have
external recipients.

[PS] C:\> New-DLPPolicy -Name "Irish Personal Documents"

-Description "Protect against the transmission of Irish personal documents in email"

-State Enabled -Mode Enforce

[PS] C:\> New-TransportRule –Name "Irish Personal Documents: Credit Card Data"

–DLPPolicy "Irish Personal Documents" –Mode Enforce

–MessageContainsDataClassifications @{"Name" = "Credit Card Number"}

–SetAuditSeverity High –NotifySender RejectUnlessExplicitOverride

–SentToScope NotInOrganization

Soon after the new policy and rule are created, they become active within the transport pipeline. To explore what happens
then, we can use Outlook and OWA to create and send some messages holding some credit card information.

Outlook and DLP

The Professional Plus editions and the click-to-run version of Outlook both support DLP policies. When these clients
initialize and connect to Exchange Online, they check for the presence of a DLP policy for the organization. If found,
Outlook downloads the policy details for use in the current session. Outlook is also capable of operating offline, so to make
sure that messages can checked when offline, Outlook stores DLP policy information in two local XML files in
%USERPROFILE%\Appdata\Local\Microsoft\Outlook. The files are:

PolicyNudgeRules_guid .XML: a small file containing the text of customized policy tips used by the rules in the policy.
PolicyNudgeClassificationDefinitions_guid .XML: a much larger file holding the set of sensitive data classifications
including local language translations of the name of the sensitive data type. The extract shown below is some of the
international names for the French National Identity Card.

The guid part of these names is a unique identifier for the policy. Outlook refreshes these files every 24 hours. You can
force Outlook to refresh the DLP policy information by editing the system registry and removing the
HKCU\Software\Microsoft\Office\16.0\Outlook\PolicyNudges key. Substitute “15.0” for “xx.x” if you use Outlook 2013.

As the user types text into a message body, Outlook can scan the text for sensitive data types, including custom sensitive
data types created by document fingerprinting (more on this later). The text recognition engine takes a comprehensive
approach, as it associates different pieces of text to decide with a high degree of confidence that the text pieces, when
taken together, make up sensitive data. For example, if a 16-digit number matches the Luhn algorithm , it could well be a
credit card number. If a date that could be a credit card expiry date (e.g., 11/17) is also present, it is more evidence that
the two pieces of text are credit card data. Finally, if someone's name is also present close to the other pieces of text, then
a high degree of confidence exists that enough information is there for a recipient to charge something to a credit card.

<Resource idRef="f741ac74-1bc0-4665-b69b-f0c7f927c0c4">

<Name langcode="am-et" default="false"> (CNI)</Name>

<Name langcode="ar" default="false"> ‫( ﺑﻄﺎﻗﺔ اﻟﻬﻮﻳﺔ ﻓﻲ ﻓﺮﻧﺴﺎ‬CNI)</Name>

<Name langcode="ar-sa" default="false"> ‫( ﺑﻄﺎﻗﺔ اﻟﻬﻮﻳﺔ ﻓﻲ ﻓﺮﻧﺴﺎ‬CNI)</Name>

<Name langcode="bg-bg" default="false">Френска национална идентификационна карта (CNI)</Name>

<Name langcode="bn-bd" default="false"> (CNI)</Name>

<Name langcode="bn-in" default="false"> (CNI)</Name>

<Name langcode="ca-es" default="false">Targeta d'identificació nacional (CNI) de França</Name>

<Name langcode="cs-cz" default="false">Národní identifikační karta (Francie) (CNI)</Name>

<Name langcode="cy-gb" default="false">Cerdyn ID Cenedlaethol Ffrainc (CNI)</Name>

<Name langcode="da-dk" default="false">Nationalt fransk id-kort (CNI)</Name>

<Name langcode="de" default="false">Französischer Personalausweis (CNI)</Name>

 <Name langcode="de-de" default="false">Französischer Personalausweis (CNI)</Name>

<Name langcode="el-gr" default="false"> Εθνική ταυτότητα Γαλλίας (CNI)</Name>

<Name default="true" langcode="en-us">France National ID Card (CNI)</Name>

When sensitive data is detected, Outlook consults the policy to decide how to tell users that such data is present and what
they should do about it. The Policy Tip might display a message saying the email cannot be sent because of the presence of
some sensitive data, or the Policy Tip might just be advisory (e.g., "you shouldn't really be sending that kind of data).” It is
also possible to allow users to override a policy by asking them to give a justification for the override. That justification
can then be sent along with information about the message to a compliance manager, who can decide on what action to
take after they examine the full context of the message. For example, an HR representative might need to send a message
with a social security number from time to time. An override received from a HR representative is OK, while an override
from an administrator working in another department who circulated some HR data might not be.

Figure 22-8 shows what happens when some credit card information of the type covered by the one-rule DLP policy we
created is found in an Outlook message. The cheery policy tip shown above the recipients is a customized policy tip. We
will discuss how you can customize the standard policy tips shortly. Because our rule allows users to override that choice
is available. Clicking the override link in the policy tip invokes the dialog shown in the center. You can see that the user
has input what they believe to be a valid business justification to override the restriction on sending credit card data
through email. Click override to complete the process. Outlook then displays a different policy tip to inform the user that
even though they have opted to override and send the message, that decision might be later reviewed by the organization.
In other words, an audit report – called a DLP incident report - is created and might be reviewed by a compliance
manager. The user can choose to go ahead and send the message and it will be delivered.

Figure 22-8: How Outlook displays DLP policy prompts

A simple number is never enough : A sixteen-digit number is not by itself enough evidence to prove that it belongs to
a credit card. DLP applies several tests to ensure that data contained in message bodies or attachments is in fact
sensitive. In the case of credit cards, the first validation is done by checking the number with the Luhn algorithm to
ensure that it matches the rules set down for credit card numbers. Further evidence is sought such as the presence of an
expiry date (like 11/15), a name, or a CVR (card verification number). If these elements are found near the credit card
number, it lends weight to the argument that this really is sensitive data of the kind that DLP is designed to protect and
enough confidence is then achieved to allow the policy to be invoked. The same is true of nine-digit U.S. social security
numbers that do not have a checksum. If DLP applied a test of just looking for nine-digit numbers, the likely result is that
many false positives would be signaled. In this case, the extra evidence that is needed is the presence of the word “SSN”
in conjunction with nine-digit numbers. Different validation rules are employed to process the various sensitive data types
so the thing to remember is that DLP must be reasonably confident that a violation exists before it will signal a policy tip
to the user or block a message when it is processed by transport.

OWA and DLP

Because OWA is an online client, it does not keep a cache of DLP policy information like Outlook does. In addition, OWA
does not include the deep content analysis capability incorporated into Outlook. Exchange Online performs DLP checking
each time OWA saves a draft of a message (roughly every minute) or when the user adds a new recipient to the message.
It is entirely possible to send a message with sensitive data if the user composes and sends the content before Exchange
Online has an opportunity to check the message. However, in this case, the DLP validation performed in transport rules
will intercept the message and take whatever action is defined by policy.

Figure 22-9 is a composite image showing how OWA displays DLP policy tips and allows a user to override a DLP check. In
this instance, Exchange Online has detected that the message has some credit card information and the recipients include
an external addressee (our travel agent). The external recipient is the problem because the DLP policy does not allow the
inclusion of credit card information in messages to external people. The original policy tip informed the user that the
message cannot be sent because it appears to hold sensitive data. More information, shown at the top of the message
header, is revealed when the user clicks Show details in the policy tip to discover that the travel agent is the problem.
Clicking Learn more will tell the user what kind of sensitive data OWA has detected, even if this is clear from the content.
Note that OWA does not use customized text for policy tips.

Figure 22-9: How OWA displays DLP policy prompts

Clicking Override exposes the input form shown in Figure 22-9 to allow the user to input a justification for the override.
The options are to give a free-form business justification or to say that the message is a false positive and does not hold
any sensitive information. The rule allows the user to override in this manner and once an override reason is provided, the
message will be delivered as normal. The override reason and message details will be kept for auditing purposes.

Supporting mobile clients : Remember that DLP checking is implemented as a predicate for transport rules and that
all the other actions and options available in transport rules can be combined in the rules you create for DLP. For
instance, because mobile client interfaces do not support policy tips (the same is true for the Outlook for Mac client), you
could create an exception in the transport rule to allow mobile clients to override DLP checking when needed by
including a code word in the message subject. The same override will work for all clients, including Outlook and OWA.
This is the approach taken by the “allow override rule” found in many of the rule sets created when a DLP policy is
created from a template (see rule 1 in Table 22-2 ). Including an exception is a quick and effective way to support mobile
clients and to avoid support calls that come in after messages are rejected and users call up to find out why this
happened. You can decide whether to communicate up-front to explain how users can override DLP checking or to accept
that the help desk will receive calls from users. The latter approach is often better because it reduces the number of
overrides and gives the chance to explain why DLP policies are in place and why a message was bounced. Incident
reports can be generated by the rule to allow overrides to be checked and, if necessary, for further user education to
occur.

Customizing Standard Policy Tips

By default, a set of four policy tips are used by Outlook and OWA when messages holding sensitive data controlled by a
DLP policy are detected. The policy tips are invariably polite and pertinent in terms of informing users when they have
erred and included some sensitive information in a message, but sometimes you want to emphasize the point by using
different language. You can do this by customizing the policy tip.

The four DLP policy tips are:

Notify the sender; “This message appears to contain sensitive data. Make sure all recipients are authorized to receive
 it .”
Allow the sender to override: “This message can’t be sent because it appears to contain sensitive data. ”
Block the message: “This message can’t be sent because it appears to contain sensitive data.”
Provide a link to compliance URL: Used to configure a URL to a web page that hopefully explains the organization’s
compliance policy and how sensitive data types should be handled in email. This link shows up in OWA when the user
clicks Learn More in response to a policy tip “View details about the information that appears sensitive .”

Office 365 provides local language translations for the default policy tips. You can see the default text for a language with
the Get-PolicyTipConfig cmdlet. For example, to show the original (exclude any custom text) values for the English
language (or locale), use the command:

[PS] C:\> Get-PolicyTipConfig –Original –Locale en

You cannot change the default text for policy tips, but you can input custom text for Exchange Online to use instead. The
EAC offers the choice to customize policy tips through an icon that resembles a document with check marks and a
superimposed gear (Figure 22-10 ). The navigation from this point is straightforward and leads you through adding a new
custom policy tip, selecting the language, and finally inputting the custom text. As can be the case, it might be faster to do
the job with PowerShell. Here is how to change the text shown when some problematic data is detected in a message and
to set up a URL for users to learn more about our compliance policy. Note that Exchange Online is picky about the
capitalization of “Url”.

[PS] C:\> New-PolicyTipConfig –Name en\NotifyOnly –

Value "Oh dear! We have a problem. Some of this text is sensitive data. What shall we do next?"

[PS] C:\> New-PolicyTipConfig Url –Value "http://www.Office365ITPros.com/Compliance.html"

Quite logically, you cannot have multiple custom text entries for a policy tip, so if custom text already exists for a policy
tip, we must use the Set-PolicyTipConfig cmdlet to overwrite it with new text.

[PS] C:\> Set-PolicyTipConfig –Identity en\NotifyOnly

–Value "Oh dear! We have a problem. Some of this text is sensitive data. What shall we do next?"

To remove a custom policy tip:

[PS] C:\> Remove-PolicyTipConfig –Identity en\NotifyOnly

Updates to policy tips take a little while to become effective because caches must be refreshed before clients see the new
text.

Although you can change the text shown in policy tips, you cannot change the text shown under the tips to inform users
about actions they can take such as removing an external recipient or overriding the policy.

Figure 22-10: Customizing a DLP policy tip

DLP Incident Reports

It is often difficult to understand how effective a data loss prevention policy is in action. The Security and Compliance
section of the Reports tab in the Office 365 Admin console includes a number of DLP reports that provide an overview of
the number of incidents that have been detected. The following reports are available:

Top DLP policy matches for mail

Top DLP rule matches for mail
DLP policy matches by severity for mail
DLP policy matches, overrides, and false positives for mail

Information for matches against Office 365 DLP policies is available in the Reports section of the Security and Compliance
Center.

DLP incident reports give the necessary information to fine-tune DLP policies and improve user education on the subject.
An incident report is emailed to an incident manager whenever a DLP transport rule has the optional “generate incident
report” action. Figure 22-11 shows a rule being changed to add this action. As you can see, you can specify the
information that should be included in the incident report, including a copy of the original message.

Incident reports can be sent to any user mailbox. However, using a “normal” mailbox to hold incident reports is bad
practice and should only be done during testing. Any instance when oversight is applied to user messages needs to be
done under controlled circumstances and incident reports can hold confidential information. It is therefore best if a special
incident mailbox is created for this purpose and access to the mailbox is restricted on a need-to basis. To avoid paying for
a license, you might think about using a shared mailbox to hold incident reports. Note that you can nominate a distribution
group to receive incident reports, so it is possible to ensure that people who need to know about the reports receive them
along with a copy going to the dedicated mailbox.

Figure 22-12 shows an incident report generated because of the rule change made in Figure 22-11 . You can see that a
complete copy of the problem message is attached to the report. The contents of the report tell us what rules were fired as
the message was processed in the transport pipeline. As shown here, you might see both DLP and regular transport rules.
Rules are noted in the priority order that they fired. The DLP rule that we created is noted last, so we know that it is last
in the priority order, which is something that might be worth checking out to ensure that the rule is in the proper position.
Note that if another DLP rule is enforced later in the pipeline that also generates an incident report, the incident mailbox
will end up receiving multiple reports for the same message.

Figure 22-11: Amending a rule to add the generation of DLP incident reports

The value of an incident report is that a human being can review the content to decide whether this is a real violation of
policy or something more benign. Perhaps the policy is detecting too many potential problems and needs to be updated,
possibly by increasing the threshold for sensitive data types that can be included in messages. On the other hand, incident
reports might show that rules are not firing as you would expect, so they are a useful way to confirm that rules work
properly.
From an auditing perspective, incident reports hold the justifications given by users when rules allow for an override. The
provided reason needs to be considered in the light of the recipients for the message, its content, and the business
rationale that can be construed by reading the content. It would seem reasonable to send some credit card information to
an external address when a credit card is needed to complete a justifiable business purchase. It might even be acceptable
when someone sends email with a credit card number for personal purposes, like to pay for a child’s after-school activity.
It is completely a different matter if someone is circulating batches of credit card numbers for no good reason to people
outside the company. Standard DLP reports tell you that incidents occur; the incident reports tell you why they occurred.

Figure 22-12: A DLP incident report

Building out the DLP policy

Our single-rule policy works and is effective at preventing the transmission of credit card information to external
recipients. However, it is a very specific policy at this point and only covers a single scenario when sensitive data might be
sent through email. As such, the policy probably does not have the depth necessary to achieve the kind of comprehensive
protection needed in most organizations. As we learned by examining the set of rules in Table 22-2 that is generated by
Exchange Online when a template is used to create a DLP policy, it is common to need a broad collection of rules to handle
different circumstances and conditions that might occur when sensitive data are detected in email. Let’s explore how to
add a couple of additional rules to expand and enhance the policy.

An obvious rule we might add is one to deal with credit card information circulated internally. Generally, we are OK when
users do this, but we would still like to keep an eye on what is happening. An example of a suitable rule is shown in Figure
22-13 . This rule generates incident reports to keep our compliance team aware of what is happening in mail traffic but
does not block messages. An exception is present for the Accounting Department group because these users will probably
deal with credit card numbers in their normal activities.
Figure 22-13: Amending a rule for a sensitive data type

Note the right-hand screen showing details of the sensitive information types processed by the rule. To get here we click
on the sensitive data type selected within the rule. The screen allows us to edit the lowest number of occurrences of a
sensitive data type that must be present in a message before a rule fires. In this case, we can see that just one instance of
a credit card number is enough for the rule to fire. If we wanted a less sensitive rule, we could adjust the threshold and set
it to 2 or more instances. You cannot change the confidence levels as these conditions depend on other processing decided
by the provider of the sensitive data type definition.

Like the rule described above, the process of building out a DLP policy involves understanding the conditions, actions, and
exceptions for each scenario in which we need to give protection. Common scenarios that might be considered include:

Allow clients to override by including a code word in the subject. An incident report should always be gathered to
ensure that this facility is not abused.
Allow some level of sensitive data to be circulated within the organization with different thresholds used for external
recipients.
Block messages that cannot be scanned for some reason. These are usually file formats that cannot be accessed by
the set of filters supported by the Search Foundation, such as the files generated by an unsupported application.

You might also decide that protection is needed for a sensitive data type that Microsoft has never heard of because it is
unique to your company. A custom DLP document type can help here, but it is often easier to create a document
fingerprint from a sample of a specific document.

Document Fingerprinting
Microsoft supports a comprehensive set of standard sensitive data types for use in DLP transport rules, but they cannot be
aware of data that is uniquely sensitive to a company. Documents like HR forms, capital acquisition requests, employee
review forms, and so on. Most companies would not like this information to be circulated externally and probably have
security and privacy policies to govern external disclosure. DLP can help to educate users that confidential company
documents should not be circulating in email unless good business reasons exist.

Like the recognizable patterns that can be used to detect instances of sensitive data like social security numbers,
documents have characteristics in terms of layouts, fields, and text blocks. You can use a process called fingerprinting to
capture the characteristics of a document and create a digital signature in the form of a hash value. That signature, or
fingerprint, then becomes a valid sensitive data type that Exchange Online can use in DLP processing.

Let’s take an example. A digital fingerprint can be created from a document in any of the formats supported by the Search
Foundation. This includes any of the Microsoft Office formats plus Adobe PDF, so the clear majority of documents used in
corporations are candidates to serve as the basis of a digital fingerprint. It is always best to select a blank form rather
than one that is filled in so that the resulting fingerprint is not affected by text or other markings that will not appear in
other copies of the form. Sample documents should not be password-protected or be graphic-heavy (text-based documents
generate the best results).
Figure 22-14: Creating a DLP fingerprint from a PDF

To begin, go to the data loss prevention section of compliance management in the EAC and click Manage Document
Fingerprints , then click the plus sign to begin adding a new fingerprint. In Figure 22-14 we see that a PDF of the U.S.
1040EZ tax form is being processed. You can use a single input file or multiple files, as in the case when each file might
represent a single page of a multi-page form. You can also include different forms into a single fingerprint so that all the
forms are checked together. You can combine forms that originate in different formats (for instance, Word and PDF) into a
single fingerprint.

When you save the new fingerprint, the EAC processes the input file to create its hash value. Exchange Online stores the
new fingerprint and its hash value as part of a "fingerprint classification collection" in the tenant configuration data stored
in Azure Active Directory. The file that you use to create the digital fingerprint is not uploaded or stored by Exchange
Online as only the hash value is kept. Once the document is processed, you can use the new sensitive data type that it
represents in DLP rules in the same way as any of the out-of-the-box sensitive data types defined by Microsoft. The hash
value can be used to compare against documents that flow through the transport pipeline and any rule that uses the
sensitive data type associated with the hash value will "fire."
Figure 22-15: A collection of DLP document fingerprints

New document fingerprints can also be created with PowerShell. In this example, we use:

The Get-Content cmdlet to read in a PDF file for a U.S. tax form.
The New-FingerPrint cmdlet to create a new document fingerprint with a hash value from the data read in from the
PDF file.
The New-DataClassification cmdlet to create a new data classification rule based on the document fingerprint.

[PS] C:\> $InputFile = Get-Content "C:\Temp\US Form W8BEN.pdf" -Encoding byte

[PS] C:\> $W8BEN = New-Fingerprint -FileData $InputFile -Description "US Form W-8BEN"

[PS] C:\> New-DataClassification -Name "U.S. Tax Form W-8BEN"

-Fingerprints $W8BEN

-Description "U.S. IRS Form W8-BEN Certificate of Foreign Status of Beneficial Owner for United States Tax Withholding and Reporting
(Individuals)"

–Locale ‘en-US’

As you can see from Figure 22-15 , the new document fingerprint for the tax form is listed in the set available for DLP.

In international organizations where clients run in multiple languages, you can assign translated values for the new data
classification. If a suitable language value is available, it will be used by a client running in that language. For example,
this command gives a Spanish translation of the description for the tax form created above. Note that the locale is set to
the value for “Spanish-Spanish”:

[PS] C:\> Set-DataClassification –Identity "U.S. Tax Form W-8BEN"

–Description "EE.UU. IRS formulario W8-BEN Certificado de Estado de Asuntos Exteriores del propietario beneficiario para Estados Unidos de
retención de impuestos y presentación de informes (individuos)"
–Name " EE.UU. Fiscal Formulario W-8BEN " –Locale es-ES

You can use the Get-DataClassification cmdlet to check that the new language value is listed in the
AllLocalizedDescriptions and AllLocalizedNames property. You can also change the default language by passing the
IsDefault switch.

Generating a digital fingerprint from the U.S. Tax Form W-8BEN creates a data classification based on one document. This
is a good approach if you have a distinct form that you will use to check in a DLP rule. However, it is also possible to
create a data classification from several documents and use the collection of fingerprints from those documents to perform
a one-pass check against all documents in a DLP rule. For instance, you might use this method to check against a
collection of HR forms that are used to capture confidential personal information.

Let’s assume that you have created document fingerprints for three HR forms and stored them in variables as described
above. You can create a new data collection from the three digital fingerprints as follows:

[PS] C:\> New-DataClassification -Name "Human Resources Confidential Information"

-Fingerprints $HRdoc1, $HRdoc2, $HRdoc3 -Description "Set of HR documents that might contain company-confidential or sensitive personal
information"

Combining several documents into a data collection works well when different forms of documents exist, such as the case
in a multi-national company where the same basic document is translated into several languages. It also works well when
different variants of a document are used to collect much the same sensitive information. You can add new fingerprints to
the data classification too. Here is how:

[PS] C:\> $HRdoc4 = New-Fingerprint –FileData (Get-Content "C:\HRFiles\New-Employee-Form.doc" –Encoding Byte) –Description "HR New
Employee Form"

[PS] C:\> $NewSet = (Get-DataClassification –Identity "Human Resources Confidential Information").FingerPrints + $HRdoc4

[PS] C:\> Set-DataClassification –Identity "Human Resources Confidential Information" –Fingerprints $NewSet

Hybrid Exchange DLP

Transport rules are specific to a platform. If you run a hybrid environment, you will have one set of transport rules,
including those that for DLP, for on-premises Exchange and one for Exchange Online. To ensure that policies are
implemented in a consistent manner across both platforms, you must make sure that the same transport rules are enabled
everywhere. Exchange includes the Export-DlpPolicyCollection cmdlet to export DLP policy information from an on-
premises or cloud deployment to an XML file. The XML data can then be imported using the Import-DlpPolicyCollection
cmdlet. This is a manual process and you should be aware that importing a set of DLP rules using the Import-
DlpPolicyCollection cmdlet will overwrite any DLP policies and associated transport rules that exist in the tenant. This
example shows how to export a DLP policy collection from an on-premises organization):

[PS] C:\> Set-Content -Path '.\DLPPolicies.xml' -Value (Export-DlpPolicyCollection).FileData

-Encoding Byte

To import (in this example, to Office 365):

[PS[ C:\> Import-DlpPolicyCollection -FileData ([Byte[]]$(Get-Content -Path '.\DLPPolicies.xml'

-Encoding Byte -ReadCount 0))

Flow
Data Loss Prevention policies do an excellent job of blocking the spread of confidential information through email and file
sharing. But sometimes we need to share information automatically. Within Office 365, Flow is the tool of choice for
automating sharing as well as many other operations. Let’s go to Chapter 23 to explore the possibilities of Flow.

Chapter 23: Automating Office 365 with

Flow
Jussi Roine
This chapter discusses how power users can use Microsoft Flow to create automated processes. The topics covered
include:

The need for automation and integration solutions.

Understanding Microsoft Flow.
The difference between older workflow solutions in Office 365 and Flow.
Enabling and configuring Flow.
Building solutions with Flow.
Managing Flow and setting boundaries for usage and data access.
Using Flow with Office 365.

We often see users repetitively performing tasks that are candidates for automation with a script or other process.
Similarly, we see a growing demand for retrieving, filtering, accessing, manipulating and reusing data from Office 365,
external systems and on-premises data sources without the need for complex custom solutions that can be error-prone and
time consuming to create. The aim of Microsoft Flow is to provide a user-friendly but powerful engine for automating tasks
and integration data between systems.

The Need for Automation

Back in the late 1980s and early 1990s, it was common for IT administrators and professionals to create powerful scripts
and automation solutions to perform common administrative tasks. These would include cleaning up old log files on a
server’s hard drive, sending email-based alerts should a process stop running and transferring files between different
systems with such utilities as Xcopy and FTP.

In the early 2000s, Windows-based systems lacked a unified and shared scripting engine capable of accessing server-side
software, APIs, and files across multiple servers or environments. Thus, when Microsoft first introduced PowerShell in late
2006, IT Pros finally had a documented and supported way to cohesively manipulate systems and automate tasks.

Today, with cloud services such as Office 365 and Microsoft Azure commonly employed in organizations, many power
users struggle with how to automate common tasks involving data and file access, integration between multiple systems
and streamlining tasks that otherwise would take hours to complete manually.

While automation comes in many forms and shapes, the need to automate tasks has existed for decades. It doesn’t mean
that all information workers and power users can automate their daily tasks with ease, but rather that many repetitive and
mundane tasks can be built as self-contained solutions. Time that is freed can hopefully then be directed at more useful
tasks and activities.

Microsoft Flow
Microsoft Flow is a fully cloud-based solution that runs in Office 365. An equivalent on-premises version does not exist,
although there was a time when the System Center family had a similar offering called the Orchestrator that was intended
for IT Pros and administrators to automate and integrate their systems. If you’ve ever used a task automation tool such as
System Center Orchestrator, you’ll find Flow is a more modern and more capable member of the automation solution
family.

What is Microsoft Flow?

Microsoft Flow allows users to build self-contained workflows that execute when triggered (by request, such as when a
specific event occurs in a remote system) or as scheduled tasks. Users can build as many Flows as they need. Flow is a
highly available solution in the sense that users do not have to worry whether a certain server is available for running a
Flow, or if a user is logged in or not.

Administrators might initially feel that Flow is a modern take on the classic Windows Server Task Scheduler, but with a
more polished interface. While that is not an incorrect assumption because both schedule tasks, the two are not alike. Task
Scheduler is typically used to run backend tasks that users rarely, if ever, need to see, while Flow is more geared towards
delivering value to users – not just administrators or backend services.

Flow was initially introduced to Office 365 in 2016, together with PowerApps. The two share certain capabilities, but they
do not depend on each other. PowerApps is a design and implementation tool for building line of business apps that run in
the cloud. The main differences and capabilities between Flow and PowerApps are outlined in Table 23-1 .

Feature Microsoft Flow PowerApps

Execute tasks as processes X

Provide a UI for users to interact with X

Run in mobile devices X X

Run in browser X X

Design solution using a browser X X

Part of Office 365 license plans X X

Extended capabilities with Dynamics 365 license plans X X

Use custom APIs X X

Table 23-1: Differences between Flow and PowerApps

Over its first few years, Flow has matured enormously and now has wide support for integration with many data sources
through its use of connectors. It is also one of the few Microsoft services that has remained tightly focused on doing
process automation and integration since its inception.

With Flow, we can build automation solutions that do not need the person designing it to be a developer. In fact, Flow does
not allow you to call external code, unless that code is accessible through an API that is offered to Flow through an
OpenAPI specified interface. As such, Flow is very easy for end users to start building small solutions and ramp up to more
advanced integration solutions as their skills mature.

Typical solutions power users and administrators might build with Flow include:

Monitor a blog site through its RSS feed, and once a new post is made (and the RSS feed updated), send a notification
to a mobile phone with link to the new content.
Free up Office 365 licenses for disabled accounts.
Save all attachments sent to an unmonitored email address as file to an Office 365 group.
Start an approval process when a new request for purchase is made through a SharePoint site.
Post a weekly status update to a Teams channel detailing changes made to one or more documents.
Send an email alert to IT administrators when or more services fail in Microsoft Azure.
Track changes to an Excel file and create approval requests for certain changes.
Create to-do items in Microsoft To-Do or Planner for incoming helpdesk requests.

As you can see, the possibilities with Flow are almost limitless as the service is very flexible and easily enhanced with
added functionality.

Older Office 365 Workflow Solutions

Some readers might wonder whatever happened to other older workflow solutions available in Office 365 (and before that,
in equivalent on-premises server software). These solutions are still available, meaning that workflows based on Windows
Azure Workflow engine exposed to SharePoint Online and Project Online are still usable. In fact, many times we see
organizations build advanced solutions around SharePoint lists and document libraries with the help of workflows using
Windows Azure Workflow. SharePoint Designer 2013 is often used in these workflows. If you’ve never worked with
SharePoint projects, you might have not heard of SharePoint Designer 2013 before. You might, however, have a vague
recollection of FrontPage, which SharePoint Designer 2013 is the spiritual successor of.

The challenge in building workflows for SharePoint Online and Project Online with SharePoint Designer 2013 is that often
the engine simply lacked capabilities users expect. Custom connectors, while possible, were hard to implement and were
often based on SOAP-based interfaces with complex XML-based contracts (as opposed to lighter weight REST APIs with
simple HTTP-schemas). In addition, these workflows lacked a proper model for moving between development, testing and
production environments. That is, if you built a fantastic but a complex workflow, it was time consuming and error-prone
to move between multiple Office 365 tenants, or between multiple SharePoint Online site collections without breaking
something.

When users building workflows with SharePoint Designer 2013 run into a brick wall and simply cannot get their required
functionality to work, the only resolution was to change the implementation model from declarative (XML-based, as used
by SharePoint Designer 2013) to an authoritative model. The change in model means that they must use a different
toolset, and users only familiar with SharePoint Designer 2013 would have to learn Visual Studio and become adept at
developing custom workflows. Obviously, this created a need for developers who can create workflows that run in
SharePoint Online and Project Online but are based on a rudimentary process model that was created by someone without
development skills. Developers would then resort to researching the declarative XML files exported with SharePoint
Designer 2013 for the workflow and try to reverse engineer the actual process and desired outcomes.
Thus, the world was ready for something more modern. Ideally, something that would not have this sort of duality in the
outcome. With Flow, Microsoft aimed to create a simple tool that would grow to handle the most complex workflows and
automation needs together with making it easy for users to get work done. In many ways, they’ve succeeded. Yet, it’s
crucial to understand that Flow does not aim to replace SharePoint Designer 2013. There isn’t an upgrade or migration
path from existing workflows built with SharePoint Designer to Flow. Also, Flow does not have parity in terms of
connectivity, supported data types or capabilities with Windows Azure Workflow. As such, we’ve seen many times that
users struggle to port their legacy workflow solutions to Flow, as a one-to-one mapping does not exist and is not possible.
A more coherent approach is to re-design the process considering the feature set that Flow provides. This also allows the
older workflow processes to be phased out when a newer implementation is built with Flow. These two technologies also
run side-by-side in the same Office 365 tenant, so if you have a working workflow already running in SharePoint Online or
Project Online, starting to use Flow does not mean that you must deprecate those solutions.

Enabling Flow
As expected with modern Office 365 services, Microsoft enables Flow by default for tenants. Many organizations simply
leave Flow enabled without paying much attention to how or when Flow is used. Some organizations disable Flow upfront
for all users (typically together with PowerApps, as they are often mentioned in the same sentence in official Microsoft
documentation) if they do not see an immediate need for these tools. It’s worth reiterating that Flow is primarily a power
user and administrator tool, therefore it is wise to consider how an organization can best utilize Flow before introducing it
to users.

Licensing
Flow is licensed in three plans, which are:

Flow for Office 365 and Flow for Dynamics 365

Flow Plan 1
Flow Plan 2

The difference between these three is that the standalone Flow plans (1 and 2) include more capabilities and resources,
while Flow for Office 365 and Dynamics 365 are built-in Flow plans that include enough functionality for most
organizations.

In addition, both Flow Plans are available for trial purposes for a period of 90 days.

You should start with the Flow for Office 365 plan, as it is included in the following Office 365 plans:

Office 365 Business Essentials.

Office 365 Business Premium.
Office 365 Education.
Office 365 Education Plus.
Office 365 Enterprise E1.
Office 365 Enterprise E3.
Office 365 Enterprise E5.

In addition, Flow is also included in the following Dynamics 365 plans:

Dynamics 365 Enterprise Sales.

Dynamics 365 Enterprise Sales.
Dynamics 365 Enterprise Field Service.
Dynamics 365 Enterprise Marketing.
Dynamics 365 Enterprise Customer Service.
Dynamics 365 Enterprise Project Service Automation.
Dynamics 365 Enterprise Operations.
Dynamics 365 Business Edition Financials.

Flow Plan 1 allows for 4,500 flow runs per month (per user), Flow Plan 2 allows for 15,000 flow runs per month (per user).
The regular Office 365 plan allows for 2,000 flow runs per month (per user).

A main difference between the standalone plans and the regular Office 365 Flow plan is that the latter allows only one
custom connector to your own systems, per user. Plan 1 and 2 allow for an unlimited number of custom connectors. This
might prove to be a problem should your users with a regular Office 365 plan need to use more custom connectors than
the one supported by the Office 365 plan.

If your users exhaust the allocation of flow runs included in the regular Office 365 plan, you can buy a Flow add-on
subscription for an added 50,000 runs. At the time of writing, the add-on subscriptions are not available, so your only
choice is to buy some extra monthly user licenses to grant more runs in the overall quota. Table 23-2 lists some other
differences for Flow Plan 1 and 2, and the regular Office 365 plans.

Feature Flow for Office Flow Plan Flow Plan

365 1 2

Unlimited flows X X X


Maximum number of flow runs per month (per user) 2,000 4,500 15,000

Maximum Flow execution frequently 5 minutes 3 minutes 1 minute

Mobile app support X X X

Connect to Office 365, Dynamics 365, Azure SQL and common Microsoft X X X
services

Connect to third-party services using standard connectors X X X

Connect to third-party services using premium connectors X X

Access corporate data using on-premises data gateway X (some X X

limitations)

Create and user custom connectors 1 Unlimited Unlimited

Create separate environments for flows 2

Monitor flow usage across organization X

Implement policies to govern usage of connections and flows X

Table 23-2: Flow functionality in different Office 365 plans

Enabling Flow for Users

Verifying and enabling Flow access for users is done by setting license options. In the Office 365 Admin Center, navigate to
Users > Active users and select a user. In the user detail page, select Edit next to Product licenses . As you’ll now get a
list of all product licenses for the user, expand any of the Office 365 plans the user might have – such as Office 365
Enterprise E5 or Office 365 Enterprise E3. Search for Microsoft Flow (Figure 23-1 ) and verify that it is enabled.
Figure 23-1: Checking that a user has a license enabled for Flow

If the user has multiple different plans, you only need to enable Flow in one of the plans.

Using Flow Admin Center

Next, it’s time to view and verify Flow settings in Flow Admin Center. This is another administration portal that is
accessible through Office 365 Admin Center. Click Admin Centers in the left navigation and select Flow to open the Flow
Admin Center in a new browser tab.

As you’ll see, the functionality available in the Flow Admin Center is still sparse and offers very little to configure. First,
click Tenant and then select User licenses . This allows us to download a .CSV file with all user licenses (Figure 23-2 ).

Figure 23-2: Generating license information about Flow and PowerApps

Generating the CSV file takes about a minute, and you can then download and view it in a text editor or Excel (Figure 23-3
).

Figure 23-3: Using Excel to view Flow license data

It’s a rudimentary but powerful and working way to quickly capture the status of all user licenses for Flow, and perhaps
use this data for future automation or license management. As explained in Chapter4, you can also use PowerShell to fetch
details of Office 365 license data, but this way is easier.

Back in Flow Admin Center, click Quotas (under Tenant ) to see the current monthly flow run rate (Figure 23-4 ).
Depending on the amount and type of Flow licenses, you might have plenty of resources available each month.
Figure 23-4: Viewing the current Flow run rate

You can also download this data as .CSV, to later make comparisons or simply tracking general Flow usage across the
organization in terms of how many Flows are executing each month.

PowerShell for Flow

Often administrators want to use PowerShell to manage Office 365 services and automate and script administrative tasks.
For Flow, PowerShell support is a new addition and is still in preview.

To get started with PowerShell for Flow, download the zipped scripts from Microsoft at https://go.microsoft.com/fwlink/?
linkid=872358 . Once downloaded, unzip the package to a local directory and perform the following tasks:

1. Open PowerShell using an administrator account and navigate to the folder where you unzipped the files
2. Run the following command to allow execution of signed scripts that were downloaded from the Internet:

[PS] C:> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force

3. Typically, files downloaded from the Internet get automatically blocked for execution, so run the following command
to unblock these files:

[PS] C:\> dir . | Unblock-File

4. Next, import the Flow modules:

[PS] C:\> Import-Module .\Microsoft.PowerApps.Administration.PowerShell.psm1 -Force

[PS] C:\> Import-Module .\Microsoft.PowerApps.PowerShell.psm1 -Force

5. Finally, run the Add-PowerAppsAccount cmdlet to add an account for accessing Flow. Note that this account must be
a tenant admin to have permissions to execute the tasks listed below.

[PS] C:\> Add-PowerAppsAccount

Upon successfully authenticating, you are now ready to access Flow through PowerShell.
Run the Get-AdminFlow cmdlet (Figure 23-5 ) to get a list of all flows currently known in the
tenant:

Get-AdminFlow
 Figure 23-5: The output from the Get-AdminFlow cmdlet

6. You can now select a single flow and get a list of flow runs. Copy any FlowName ID (a GUID) and run the Get-
FlowRun cmdlet passing the identifier for the flow (Figure 23-6 ):

[PS] C:\> Get-FlowRun <ID>

Figure 23-6: Output from the Get-FlowRun cmdlet

Typically, you use PowerShell to monitor Flow connector usage, verify what and how many Flows are created and run, and
deleting obsolete Flows. You can, of course do many of these tasks from Flow using the administration portal, but you can
automate some of the more tedious tasks with PowerShell.

Building Automated Solutions with Flow

Now that we’ve performed the initial configuration for Flow, it is time to start building some Flows. We’ll start with an
easy Flow and move on to more advanced features once we have the basics in place.

To access Flow to build flows, navigate to https://flow.microsoft.com or simply click the Flow icon in any Office 365
interface from the app menu. We can create a Flow from a pre-defined template, or from scratch. To better understand
how flows are built, we’ll start from scratch. From the Flow front page, click My flows in the top navigation bar. This page
lists all your existing flows, with tools to enable or disable, share, and edit each one individually. From here, click Create
from blank in the toolbar.

Building Your First Flow

To build our flow, follows these steps:

1. To schedule a flow, akin to a scheduled task in Windows, you select Schedule as the first connector (this then defines
it as a trigger). For our first flow, select Schedule (Figure 23-7 ).
 Figure 23-7: Flow triggers and connectors

2. Flow searches for all triggers that match your choice and proposes the only one it finds – the Schedule –
Recurrence (Figure 23-8 ) trigger.

Figure 23-8: The Schedule trigger for Flow

You can now define the interval for your flow. The default is to run the flow every 5 minutes.
If you’re working with the trial version of Flow, keep in mind that you have 750 flow runs
each month. If you run your flow every 5 minutes, you will consume your monthly flow run
quota in just a few days. Luckily you can always disable any flows you might not be using,
and that’s a good practice for the future. Leave the 5-minute interval in place and continue
building the flow.

3. Next, we’ll start building out the actual logic for the flow. We’ll add actions, conditions, and other logical building
blocks in the view by clicking on clicking on + New step (Figure 23-9 ).

Figure 23-9: Adding a new step for a Flow

4. We now have the choice to add an action, a condition or more complex logic such as an apply to each loop. Select Add
an action (Figure 23-10) to see the available actions.

When you scroll down the list of available actions (which number in the hundreds), you’ll see
the vast scope of functionality that you can include in a flow. You can easily connect to
external databases, to services in Azure, and to data stored in on-premises servers. In this
 case, we select Office 365 Outlook – Create event (V2) for the first action.

5. Your first action is now transformed into a box with the Outlook icon and some mandatory fields to fill out. Flow might
prompt you to authenticate, as it might not have that authentication token for your account. Perform authentication
to an account that has access to the Outlook calendar you wish to work with. Once you’re done, you can verify the
available accounts by clicking the three little dots in the corner of your action (Figure 23-11 ):

Figure 23-11: Starting to create a calendar event in a Flow

You can see other options here, such as adding a comment for this action (for providing
better readability for other editors of this flow), and under Settings some specific settings for
this action. You can also remove this action should you not need it later. Verify that you are
using the desired credentials to work with the calendar.

6. Next, select the drop-down menu for Calendar id and select your calendar. Typically, it is named Calendar in
English-based interfaces. The name of your calendar might vary if you’re using Flow with a localized interface. You
might have other calendars visible here as well, such as those belonging to other users who have granted permission
for you to access their calendars. Flow also includes a powerful expression language that allows you to specify
dynamic values and combine or retrieve data from other actions within the same flow. This way you could dynamically
choose a calendar based on data you received earlier in the flow or based on actions the flow is instructed to do.
7. For Subject, we can add a fixed string, or use the expression language. This would be ideal if you needed to have a
date and time, or some other dynamic value as part of the event’s subject. To keep this example simple type in the
following text in the Subject field:

[Flow] Test event

8. For Start time , type a date and a time in the future – perhaps the next day. Note the time format, which needs to be
in ISO 8601 format, so the ‘T’ must appear literally to show the beginning of the time element. An example value is:

2018-10-10T09:00:00

For End time , add a similar date and time but a bit later than the start time:

2018-10-10T10:00:00

In our example, the event starts on 10 th of October 2018 at 9 am and ends on 10 th of

October 2018 at 10 am.

You might be wondering what benefits a Flow brings if it only supports fixed dates and times
for new calendar events. Rest assured, Flow has a powerful expression language that allows
us to substitute, generate and dynamically insert desired values – such as a dynamic date
that adds an event 1 day from today. You can access these expressions on the side menu
under Expression . This provides you with different collections, logical functions, string
functions and similar programming concepts you might need later.

9. Under Show advanced options you have many more ways to fine-tune your calendar event, but for our purposes
only the 4 mandatory fields (calendar id, subject, start and end time) are needed. Your event creation action should
now look like this (Figure 23-12 ):
 Figure 23-12: Creating the calendar event through Flow

10. As we need to see how our process works, we next need to name the flow. You can set a name in the top left, where it
says Untitled . Simply replace the default name with something more descriptive, such as Create test event in
calendar . That’s it! Save the flow by clicking Save in top right corner.
11. Once you’ve saved your flow, you can test it immediately. Click on Test and then select I’ll perform the trigger
action and click Save & Test (Figure 23-13 ).

Figure 23-13: Testing a Flow

12. Confirm in the dialog that you want to run your flow. This shouldn’t take more than a few seconds, and you can then
click See flow run activity . This opens a new browser tab and goes directly to the flow runs view to verify that your
flow ran successfully. Click on the latest entry in the list (Figure 23-14 ):

Figure 23-14: Viewing flow run activity

13. You should have both the trigger and the action item visible with a small green checkmark in each corner to tell you
that your flow worked. You can expand any actions to verify the input and output data and troubleshoot for any errors
(Figure 23-15 ):
Figure 23-15: Verifying flow run activity

14. Finally, open your calendar (where you made the new event with your flow) and verify that a new calendar entry is
present (Figure 23-16 ):

Figure 23-16: New calendar event created by Flow

15. Note, that the event was successfully created but the timestamp seems incorrect. This is because Flow uses UTC
timestamps internally while your calendar might use a different timezone.
16. Click next on My flows within the Flow designer browser tab, and click on the slider of your flow that says On to
disable the flow (Figure 23-17 ):

Figure 23-17: Flow is disabled

17. This is to prevent the flow running every 5 minutes and adding multiple overlapping calendar events.

You’ve now successfully completed creating your very first flow!

Sharing Flows
If you want to share your flow with a colleague in the same Office 365 tenant, click on the people icon next to your flow
(Figure 23-18 )

Figure 23-18: Sharing a flow

Anyone you choose to share your flow with becomes an owner of the flow. They can edit and modify the flow and will also
receive access to any connections within your flow. If you chose to authenticate with an Exchange Online mailbox to send
emails or create events, those connections become embedded connections when you choose to share your flow with others
(Figure 23-19 ):
Figure 23-19: Choosing what to share when sharing a flow

Building and running Flows in Excel

Flow is also now supported within Excel, allowing power users to create new flows within their spreadsheet. This brings in
the capability to trigger automations from data in Excel, and thus automate data processing.

To use Flow in Excel, use the Get add-ins button in Excel to add the Flow Add-in. Once the add-in has been installed, a
new Flow button allows selecting data on a spreadsheet such as a row.

In order for Flow to be able to interact with data in an Excel file, it has to be accessible – such as stored in OneDrive for
Business or a SharePoint Online document library.

Using Data Loss Prevention Policies

Data Loss Prevention policies allow administrators to set restriction on usage for Flow data handling and data capture.
Typically, organizations do not wish to capture or store sensitive information or privacy data, unless it is properly handled
and secured. As users build their solutions with plenty of freedom, a reasonable need exists for administrators to set
certain boundaries through Data Loss Prevention policies.

Note: DLP policies require a Flow Plan 2 license

To start defining a policy for Data Loss Prevention, in Flow Admin Center click Data policies and then click New policy
(Figure 23-20 ).

Figure 23-20: Defining a DLP flow

The first selection defines which environments this policy applies to. An environment is simply an isolation boundary for
DLP and data locality. We can create one or more environments and define if those environments will be created in
Europe, in the U.S. or some other region based on our data locality needs. If you haven’t created any environments yet,
don’t worry as you can create a DLP policy that applies to all environments (Figure 23-21 ) regardless of how many you
have in place or plan on creating in the future.
Figure 23-21: Defining the scope for a DLP flow

As you can see in Figure 23-21 , if your current Flow license does not include Plan 2, you’ll need to upgrade your plan.

Click Data groups (Figure 23-22 ) to define the policy restrictions. In this view, you can define which Flow connectors can
access business data and the flow connectors that are blocked from business data. You can also set which of these is the
default – can access business data only, or no business data allowed. This model then limits any Flows that users create to
share data between either group. Consider a Flow that triggers based on a tweet sent to Twitter to automatically reply
with a tweet. We’d like to limit users from creating Flows that might potentially access sensitive business data and then
send that data out in an insecure manner, such as in a tweet.

You can freely add and remove connectors from either group. In the example above, we’ve defined that SharePoint,
OneDrive for Business and Dynamics 365 allow access to business data, and Salesforce, OneDrive (consumer) and SQL
Server do not allow access to business data.

After you name and save your policy, it is automatically activated for the environment(s) you chose.

Users building Flows that contradict this DLP policy can create their flows, but Flow will put the flows into a suspended
state. If a user creates a Flow that retrieves data from SharePoint and pushes it to Salesforce (as per the example above),
Flow will automatically suspend the flow.
Figure 23-22: Defining data groups

Accessing On-premises Data with the

Data Gateway
Often the need to build solutions using Flow includes a need to integrate data from on-premises data repositories. These
are traditionally SQL Server databases, documents and other data stored in SharePoint document libraries and possible
other data stores.

To provide connectivity with data stores within a corporate network, the Data Gateway can be used (see this document for
software and hardware requirements for the Data Gateway). The gateway creates a HTTPS proxy connection that can be
accessed securely from Flow. It is the same component that Microsoft’s Power BI service uses. The supported data sources
for Data Gateway are:

SQL Server.
SharePoint.
Oracle.
Informix.
Filesystem (files and file shares).
DB2.

The gateway uses outbound connections, meaning that connections are always initiated from the internal network towards
the cloud – not the opposite way. Traffic is pushed through Azure Service Bus, which is hidden within Data Gateway’s
overall architecture.

A good practice is to download the Data Gateway executable and deploy it on at least two servers that can access your
required data sources. This way Data Gateway provides a highly available set up, as load balancing between the two
machines is done automatically.

Note that you cannot install Data Gateway on an AD Domain Controller. The machines that host Data Gateway can have
shared services, but they should be servers and not workstations.
To install Data Gateway, simply download the executable and run through its setup. When you configure Data Gateway,
make sure to store the recovery key in a safe place, as you will need it for recovery and when you expand your setup with
multiple Data Gateway installations.

You also can specify your region, which should be the one where your Flow’s are being executed to avoid excess latency.

If your outbound network is restricted by a proxy or a firewall, make certain that the ports and destinations listed in Table
23-3 are allowed for Data Gateway to operate correctly.

Destination addresses Port(s)

*.analysis.windows.net 443

*.login.windows.net 443

*.servicebus.windows.net 5671-5672

*.servicebus.windows.net 443, 9350-9354

*.frontend.clouddatahub.net 443

*.core.windows.net 443

login.microsoftonline.com 443

*.msftncsi.com 443

Table 23-3: Required destination addresses and ports for the Data Gateway

After installing the Data Gateway, on-premises data sources are accessible when building your Flow.

Building Advanced Solutions with Flow

As you've seen by now, building solutions with Flow is easy and straightforward. Things get more complex when your flow
needs to call other flows, and when your flow must make multiple decisions during its run. It’s always best to start
building a new flow by planning it first:

How will the flow trigger?

How many runs do you anticipate for the flow each month?
Is your flow self-contained, or will it call upon other services or third-party APIs?
What will happen if the flow fails?
How do you log flow activities?

You can always modify your flow definitions later. It’s also possible to export a flow in a .zip file, to keep it as a
timestamped copy. This way you can rest assured that you have a copy of your flow before making extensive changes.

For troubleshooting flows you have a few tools at your disposal. The run history for each flow expands to show each
connector input and output data and provides possible exceptions or raw errors your flow encountered. This is the best
way to understand why your flow fails, or why a single run failed among multiple successful runs. You can also test flows
by editing a single flow and clicking on the Test button in the upper right corner. This puts your flow in a test run view, and
shows a single flow execution in real-time to make debugging easier (Figure 23-23 ):
Figure 23-23: Raw output from Exchange Online connector in a flow

Flow checker is a small tool that can be activated in the upper right corner, while editing your flow. It performs an analysis
of your flow and provides recommendations should your flow have any detected issues.

Certain connectors and triggers also expose settings to allow you to further fine-tune how a flow activates and uses these
building blocks. For example, when you use the Send Email connector to send email through Exchange Online, you have
the option to specify a timeout value, and a retry policy.

Some connectors also expose a Run after trigger, that can be configured to run if the previous task failed, succeeded,
timed out or skipped. This gives you a handy way to provide further debugging, by sending out data to another API or
system if a certain condition was met. If, for example, you make a call to a third-party API but it times out, you can alert
another system that the API is not answering to avoid further issues in subsequent flow runs.

Over time you’ll end up with some flows you don’t actively need or use. Instead of deleting them, disabling allows you to
retain their run histories and enable them at will should you have a need.

Flow is Here to Stay

Microsoft Flow has quickly proven to be a valuable tool for administrators, power users and even developers who build
automation and integration solutions for Office 365. As a lighter version of Azure’s Logic Apps, Flow is both capable and
powerful for creating solutions without writing any code.

In just a few years we’ve seen Flow expand from a few connectors and triggers to an ever-growing platform that supports
several hundred connectors and allows custom APIs to be accessed. With the Data Gateway, data stored in on-premises
systems can be accessed securely and easily.

With the release of PowerShell support for Flow (and PowerApps), we’re also seeing some initial steps for managing and
monitoring Flow runs and settings in a modern way.

It’s now obvious that Flow is here to stay. It’s a core tool for administrators who are accustomed to building automations
since the old days of scheduling tasks on a Windows Server, to power users who need to build automated solutions without
spending days and weeks to integrating systems securely.

Protection
With all the data tenants store inside Office 365, it is natural to consider how best to protect content, especially when the
data is confidential or sensitive. Office 365 Sensitivity Labels and the underlying rights management technology is a good
way to protect email and documents, and that’s where we go next.
Chapter 24: Protecting Office 365
Content
Tony Redmond

This chapter discusses how rights management is used to protect Office 365 email, documents, and other files. The topics
covered include:

The use of rights management within Office 365.

The rights management service.
How to protect Office 365 documents and messages with sensitivity labels.
How to protect SharePoint Online document libraries and lists and OneDrive for Business sites.
How rights management templates work and how to create new templates with sensitivity labels.
Interoperability between Office 365 sensitivity labels and Azure Information Protection labels.
How to protect external communications.
Using PowerShell with protected content.

In some quarters, rights management enjoys a reputation of being a technology difficult to understand, implement, and
manage. It is certainly true that deploying this kind of protection for on-premises infrastructures can take significant
effort. Hopefully, the information discussed here proves that although the implementation of rights management within
Office 365 uses the same technology, the way that Microsoft takes care of setting up and managing the necessary
infrastructure makes it easier to take advantage of protection.

The Need to Protect Data

We live in a world where encryption is pervasive for both businesses and consumers. It is unthinkable to consider
conducting a banking transaction through a web site not protected with encryption. Even so, an awful lot of email traffic
still moves over unencrypted SMTP links (Office 365 uses TLS to secure mail traffic in transit), perhaps because setting
up reliable encryption for email has always run into the difficulties of agreeing on a common standard for external
transmission and the cost of deploying and supporting an infrastructure to assure secure email internally. The earliest
versions of Exchange Server included the Key Management Server (KMS) to manage the storage and distribution of keys
for message encryption. At the time, the great hope was that the availability of KMS would encourage many organizations
to adopt encryption to protect email. That hope never fully materialized.

Rights management delivers the capability to protect confidential messages or documents to control what recipients can
do with the content. Without protection, an email recipient can send on messages they receive, print the messages, cut
and paste from the messages, and so on. Protection templates allow organization to define sets of rights that users apply
to messages and documents to restrict what a recipient can do. The functionality protects companies by allowing them to
circulate confidential or sensitive information under control and avoid situations such as “information leakage” which
happens when, for instance, a disaffected employee forwards messages to journalists or other interested parties. Another
example of similar functionality in a different context is the use of DRM to protect music downloaded from various sites.
In addition, Template settings also control how long users can access content after which the information becomes
inaccessible. Microsoft offers several ways to deploy rights management to protect content:

On-premises customers can deploy their own Rights Management (RMS) server to manage the encryption keys used
to protect content by both on-premises and hybrid users. Hold your own key (HYOK), which also uses keys managed
by on-premises servers, extends coverage to Office 365. Because of the complexities involved in using HYOK to
protect data differ from customer to customer, we do not cover it in this chapter.
Office 365 tenants can generate their own encryption keys and import the keys into Microsoft’s datacenters, where
the keys are under the control of the tenants. This implementation is known as Bring your own key (BYOK).
The most common method to implement protection within Office 365 is when Microsoft manages the encryption keys
for tenants. As with BYOK and HYOK, the keys issued to tenants protect Office 365 information including Exchange
Online messages and document libraries and lists managed by SharePoint Online and OneDrive for Business.

Although the concept of rights management extends back to the early 1990s, Microsoft started to implement the
technology for products like Exchange and Outlook in the middle of the decade. The technology proved capable of
protecting information, but customers did not take to the technology for a few reasons. Part of this was because of the
culture shift needed inside companies to take the protection of information seriously, including strong executive
leadership to champion the case for to deploy the technology. Part of it was due to the extra infrastructure plus the time
needed to deploy and run Active Directory Rights Management Services (ADRMS), the service underpinning protection
within an on-premises environment. Because Microsoft takes care of the required infrastructure and integration, tenants
need to dedicate less effort to implement protection within Office 365.

The work to deliver and expand protection across all Office 365 workloads is a journey and continues as Microsoft
delivers new functionality. The latest example is the transition from Azure Information Protection (AIP) labels, originally
introduced as a separate product in 2016, to Office 365 sensitivity labels, which can apply both visual marking and
protection defined in rights management templates to content via Office desktop and online applications. As we’ll
discover later, tenants manage sensitivity labels through the Security and Compliance Center and do not have to worry
about the underlying templates because these are hidden behind the scenes.
Office 365 and Rights Management
Originally, the protection capabilities within Office 365 mimicked those available for on-premises servers. As Office 365
matured, the capabilities available to cloud and on-premises customers diverged. Office 365 uses the Azure Rights
Management protection service as the foundation for rights management and encryption to deliver features including:

The Office 365 Message Encryption (OME) portal, where recipients of encrypted email whose mail server cannot
decrypt the messages automatically can access the content using a browser. When Microsoft talks about OME, they
also mean:

Encryption built into Outlook and OWA, which can replace the need to deploy S/MIME and other third-party tools. Of
course, if you’ve invested in an S/MIME infrastructure, you can continue to use S/MIME to digitally sign and encrypt
messages.
Protection applied by email transport rules. For example, to ensure that any message sent to a partner domain is
encrypted.

Office 365 sensitivity labels assigned to email and documents. While some labels only apply visual markings to
content, others can invoke protection.
Information Rights Management protection for documents downloaded from SharePoint document libraries.

Figure 24-1 illustrates how Office 365 consumes the Azure Information Protection service and rights management
templates to deliver protection to email and documents. Rights management is part of the Office 365 E3 and E5 plans and
includes the ability to create and use sensitivity labels to protect email and documents in Exchange Online, SharePoint
Online, and OneDrive for Business. Users with other Office 365 plans can buy the Azure Information Protection P1 add-in
to license rights management. Even if you have Office 365 E3 or E5, if you need to extend protection to files stored
outside Office 365, you need Azure Information Protection licenses, which are sold standalone or bundled in Enterprise
Mobility and Security or Microsoft 365.

Figure 24-1: Office 365 and Rights Management

Some differences might exist in the capabilities depending on the Office 365 datacenter region used by your tenant. For
example, the sovereign cloud Office 365 datacenter regions in China and Germany do not support the full suite of
protection capabilities and users within these clouds have limited options to protect email and documents. Check with
Microsoft to learn the current situation.

The Flow of Protection

Once you enable rights management for a tenant, clients like Outlook authenticate (an automatic process) to receive
certificates from the rights management service to allow the user to access protected content even when they work
offline. The service keeps a copy of the user certificate so that it can issue it to another workstation if the user connects
to the service from there. Once authenticated, users can send and receive 256-bit AES-encrypted protected messages.
When you protect a message or document, the client embeds a unique key (the content key) in the item’s header. To
protect an item, the client encrypts it (and any attachments that supports protection) using the AES symmetric
encryption algorithm. The content key persists with the item even if the author edits it to create an updated version.
Rights management protects the content key with another key (the tenant key), which is common across all protected
content. The tenant key can be managed by Microsoft or by the tenant (BYOK).

When it encrypts content, the client also generates a certificate that includes a policy with the usage rights for recipients
(individual users or groups) as well as any other restriction, such as an expiry date. These settings come from the
template selected by the user to protect the content. The client signs the policy with a user certificate. The client also
encrypts the policy and the content key using the tenant key. Finally, the client combines the encrypted content with the
signed and encrypted policy to create the protected item. This ensures that the policy always stays with the encrypted
content.
When a recipient reads protected content, the client extracts the policy from the protected item and sends it and the user
certificates to the rights management service, which decrypts and evaluates the policy to build a set of rights for the
item. The service then extracts the content key from the policy and uses it to create an encrypted use license that holds
the set of rights and returns the license to the client. The client decrypts the license and uses it to decrypt the content
and display it to the recipient. The client also handles the enforcement of rights given to the user by policy, including
limiting access if the license is valid for a certain period. When a license expires, the user must reauthenticate with Azure
Active Directory to get another license.

Mobile clients like Outlook for iOS use a simpler transactional flow. When mobile clients protect content, they send the
selected policy to the server and receive a publishing license and symmetric key to protect the item. To consume content,
the client sends the policy to the service and request a use license. The service responds with the necessary keys and
policy information to allow the client to open and display the content.

Protection Templates
As soon as you enable rights management for a tenant, you can start to protect content using three default templates.
Later, you can customize the default templates or create customized templates to reflect the way you want to protect
confidential information within your tenant. Protection templates are managed through the Azure Information Protection
blade of the Azure portal . Templates can be defined as standalone objects or they can be associated with Azure
Information Protection labels, which work in similar ways to the Office 365 Sensitivity Labels discussed later in this
chapter.

Figure 24-2: A rights management template as viewed in the Azure portal

A template defines the access rights granted to a recipient to work with content. Logically, the author or originator of
content always has full control over the content. You can compare applying a template to a message as being analogous
to registering a postal letter. The recipient is only able to open and access the content if Office 365 recognizes their
access – the recipient gains access by having a known account that has access rights defined by the template that the
application can apply to the item. If protected content ends up in the hands of an unknown user, they will not be able to
access the content because it will stay within an encrypted “wrapper” that only intended users can open. The template
shown in Figure 24-2 grants co-author rights to members of a distribution list called Project Condor and viewer rights to
anyone from the Redmond & Associates tenant.

You can continue to manage rights management templates through the Azure portal, but in most cases, Office 365
tenants can manage templates through Sensitivity Labels, managed through the Security and Compliance Center. If you
create a new label that needs encryption, you also create a new rights management template that shows up in the Azure
portal.

In addition to allowing authors to decide what level of protection to assign to their work by adding a template to a
message or document, administrators can create transport rules to apply a template to messages that meet certain
criteria as the messages travel through the transport pipeline. This is an excellent way to automatically protect
confidential messages if you can create criteria to find those messages. For example, all messages that mention the word
“Confidential” in the message subject or all messages sent to the “Corporate Planning” distribution group. Transport
rules protect messages without user intervention and users cannot override what they do. Best of all, this approach works
for messages sent from any client.

Protecting messages by applying templates through transport rules allows great control over information circulated by
users in email. It also means that the content of any message sent externally is inaccessible because external recipients
cannot retrieve the use licenses necessary for decryption. Fortunately, you can use encryption to protect messages sent to
external recipients in a way that they can access the content in a safe and secure manner.

Protection can also be applied by Office 365 data loss prevention polices if sensitive data such as passport numbers or
credit cards are detected inside documents stored in SharePoint Online or OneDrive for Business sites.

While more difficult to manage, it is possible to use an on-premises Active Directory RMS server to deliver a Rights
Management service to Office 365 users. That setup is outside the boundaries of the discussion presented here.

External Users
Protection works on the basis that a recipient can authenticate themselves using an Azure Active Directory account or a
Microsoft Service Account (MSA) associated with an email address. Therefore, you can send protected items to people in
other Office 365 tenants and other email domains, providing that the policy used allows those recipients rights over the
item. See this document for more information about how to use protection to secure documents shared with people
outside your tenant.

Azure Active Directory federates with some other directories, such as Gmail and Yahoo, to allow users of those services to
authenticate by signing into that service. Otherwise the external user will have to sign in using their MSA account before
they can access the document.

Contacting the Azure Information Protection Team

If you have a technical issue with Azure Information Protection or wish to discuss a requirement your organization might
have, you can join the conversation with the AIP team in Yammer .

Enabling Rights Management

Before you can protect content, the rights management service must be enabled. Go to the Office 365 Admin Center and
go to the Settings section. Select Services & Add-ins, then Microsoft Azure Information Protection, and then click
Manage Microsoft Azure Information Protection Settings . You then have the choice to activate Rights Management
for your tenant, or as we see in Figure 24-3 , confirm that the service is already activated. The advanced features button
brings you to the Azure Active Directory console where you can check other configuration details. However, the first step
to making Rights Management work is to enable it for the tenant.

Figure 24-3: Viewing the protection status for a tenant

Starting August 1. 2018, Microsoft will enable rights management automatically for eligible Office 365 tenants. In other
words, if you have an eligible plan like Office 365 E3 or E5 or buy the Azure Information Protection add-on for other
plans, you will not have to enable Rights Management.

PowerShell for Rights Management

If you want to manage the protection service with PowerShell, you need to download and install the AADRM PowerShell
Module . Afterwards, you can connect to the service on an ad-hoc basis or include a connection to the service in your
PowerShell profile. When you connect to the service, the Get-Aadrm cmdlet tells you if protection is active for your
tenant. It must be before you can do any useful work!

[PS] C:\> Import-Module Aadrm

[PS] C:\> Connect-AadrmService

[PS] C:\> Get-Aadrm

Enabled

Configuring Rights Management for Exchange Online

Before Exchange Online can protect messages, you must follow these steps to configure the necessary settings.

Configure Exchange Online

Connect a PowerShell session to Exchange Online and the protection service and then run the following commands. The
overall process should take less than a minute.

[PS] C:\> Enable-Aadrm

#Get the configuration information needed for message protection.

$rmsConfig = Get-AadrmConfiguration

$licenseUri = $rmsConfig.LicensingIntranetDistributionPointUrl

#Collect IRM configuration for Office 365.

$irmConfig = Get-IRMConfiguration

$list = $irmConfig.LicensingLocation

if (!$list) { $list = @() }

if (!$list.Contains($licenseUri)) { $list += $licenseUri }

#Enable message protection for Office 365.

Set-IRMConfiguration -LicensingLocation $list

Set-IRMConfiguration -AzureRMSLicensingEnabled $True -InternalLicensingEnabled $True

#Enable new Protect button in Outlook on the Web

Set-IRMConfiguration -SimplifiedClientAccessEnabled $true

From March 1, 2018, Microsoft sets the AzureRMSLicensingEnabled property for tenant IRM configurations to $True .
The commands are shown here for completeness.

Test the configuration

To test the rights management configuration, we run the Test-IRMConfiguration cmdlet. The templates listed in the
results are the default templates created in every tenant to set up a base for protection. These templates are Confidential
and Confidential View Only (prefixed with the tenant name), Encrypt, and Do Not Forward. Of course, you want to see
“PASS” as the overall test result.

PS] C:\> Test-IRMConfiguration -Sender Tony.Redmond@Office365itpros.com

Results : Acquiring RMS Templates ...

- PASS: RMS Templates acquired. Templates available: Redmond & Associates - Confidential View Only, Redmond & Associates -
Confidential, Encrypt, Do Not Forward.

Verifying encryption ...

- PASS: Encryption verified successfully.

Verifying decryption ...

 - PASS: Decryption verified successfully.

Verifying IRM is enabled ...

- PASS: IRM verified successfully.

OVERALL RESULT: PASS

If you do not see similar output to that shown above, it is probably because your tenant configuration uses the earlier
IRM stack. Repeat the steps to configure IRM and then retest to confirm that the new configuration is in place.

Check the Exchange Online IRM configuration

Run the Get-IRMConfiguration cmdlet to check the rights management configuration for Exchange Online. If any changes
are necessary, you can make them with the Set-IRMConfiguration cmdlet. For example:

[PS] C\> Set-IRMConfiguration –InternalLicensingEnabled $True

–ClientAccessServerEnabled $True

In most cases, you do not have to amend the default configuration to use start protecting email. If you want to, you can
investigate settings such as:

ClientAccessServerEnabled : Default is $True . Controls whether OWA and ActiveSync clients (including Outlook
for iOS and Android) can use IRM to protect and decrypt messages. When true, Exchange Online can fetch use
licenses from the rights management service on behalf of these clients (see below).
DecryptAttachmentForEncryptOnly : Controls whether Exchange Online decrypts attachments protected with the
Encrypt feature when a recipient with an Azure Active Directory account (for instance, a recipient in another Office
365 domain or Outlook.com) downloads a message attachment. The default is $False , meaning that attachments
remain encrypted when downloaded. If you change this control to $True , attachments are decrypted, and recipients
have full control over the files.
DecryptAttachmentFromPortal : Controls whether Office 365 decrypts attachments for messages protected with
the Encrypt feature when a recipient who does not have an Azure Active Directory account downloads a message
attachment through the OME portal. The default is $False , meaning that attachments remain encrypted after
download. Change the setting to $True to force Exchange Online to decrypt attachments when downloaded from the
OME portal.
EDiscoverySuperUserEnabled : Default is True, which allows members of the Discovery Management RBAC
management role group to access protected information found through Exchange Online eDiscovery searches and
copied to a discovery mailbox. Members of the Discovery Management role group can access protected items in the
discovery mailbox using OWA, including being able to preview items found by eDiscovery searches, as the client will
obtain the necessary use license to decrypt the information. Unless their accounts have rights to view the items,
members of the Discovery Management role group cannot export protected items from a discovery mailbox to a PST
and access them using Outlook.
InternalLicensingEnabled . Default is $True . Controls whether licenses are automatically granted to internal
recipients to allow them to access protected content.
JournalReportDecryptionEnabled : Default is True. Controls whether journaling attaches a decrypted copy of
protected messages to journal reports.
SearchEnabled : Default is True. Controls whether OWA can search protected items in a mailbox.
SimplifiedClientAccessEnabled : This setting controls whether the Protect button appears in OWA to allow users
to apply protection templates to messages. (the new version of OWA replaces Protect with Encrypt) The default is
$True , which exposes the Protect button. Set this to $False if you don’t want users to apply protection to messages.
TransportDecryptionSetting : Default is Optional. Controls the access to protected messages for the transport
service as they pass through the transport pipeline. Optional means that the transport service tries to decrypt
protected messages so that the content is available for checking by transport and DLP rules or by anti-virus agents.
However, transport will continue to process and deliver the message even if decryption is not possible. You can
disable any attempt to decrypt messages by setting this value to Disabled . Setting it to Mandatory means that
transport will reject any messages that it cannot decrypt and return a non-delivery report to the sender. The
transport service automatically re-encrypts decrypted messages when they reach the end of the transport pipeline.

To make things easier for OWA users, if the ClientAccessServerEnabled setting in the IRM configuration is $True ,
Exchange Online imports keys from the Rights Management Trusted Publishing Domain (TPD) and is thereafter able to
decrypt content locally on behalf of clients so that OWA can display protected content inline within message windows.
Some Exchange ActiveSync clients use the same approach, which is known as prelicensing.

Bring Your Own Key (BYOK)

Office 365 allows customers who need the highest possible degree of control over all aspects of security the ability to
have full control over their tenant key, the element that serves as the root of trust for protection. The key is “pinned” or
imported to a FIPS140-2 hardware security module (HSM), usually kept under tight control at a customer’s premises. In
collaboration with Thales E-Security, Microsoft uses a secure process to transfer the key from a customer and import it
into the Office 365 datacenters into Azure KeyVault to make the key available to serve as the basis for protection. The
process is free, but obviously needs a good deal of planning and coordination. Both SharePoint Online and Exchange
Online (from October 2017) support Azure KeyVault, so once a tenant imports its own key to become the tenant key, that
key becomes the base for encryption for SharePoint and Exchange content, except when encryption is applied through
sensitivity labels as the templates used by these labels are cloud-based. See this link for more information about BYOK.
Protecting Email
Office 365 supports several methods to protect email:

Users can apply protection templates using Outlook 2013 or 2016 desktop, Outlook 2016 for Mac, or OWA. The Click
to run version of Outlook is needed to use the Encrypt-only feature. The same clients can also read protected
messages, as can the Outlook for iOS and Android clients. These clients allow inline viewing of the content of
protected messages, just like normal email. Permissions are visible for attachments that support protection, like
Word and Excel files. Other clients can read protected email using an embedded link in a notification holding
information telling the recipient how to access the content
Tenants can ensure protection for all messages sent to specific domains or sets of recipients by configuring transport
rules.

Protected messages can be scanned and discovered by Office 365 content searches. Only the metadata of protected
documents are scanned by searches. The difference between the two implementations is because Exchange Online can
invoke super-user capability to decrypt protected messages when needed. Exchange Online Protection can scan protected
email to detect malware.

All successful implementations of protection for common Office content such as email and documents involve
communication with the user community to explain the need for the technology, how to protect information (including
some examples of best practice based on common scenarios), and the consequences (as laid down in HR policies) that
might ensue if someone ignores the protection applied to data. No need exists to hit users across the head with a 2-by-4
to enforce policies; instead, all that anyone needs is a common-sense approach from all concerned.

Enlightened and Unenlightened Clients

Office desktop applications like Outlook, Word, PowerPoint, and Excel are “enlightened” applications. In other words,
these applications include the necessary software to use the rights management APIs to retrieve and consume rights
policies and licenses, including when they run offline. Browser-based applications, like OWA or OneDrive for Business,
run online and use a connection to the service to obtain use licenses and apply and check protection for documents.
Outlook for iOS and Android also support protection when connected to Exchange Online. See here for more information
about applications that support rights management.

In many Office 365 deployments, the Apple iOS mail app is popular with users. This client does not incorporate the rights
management APIs and is therefore unable to read protected messages, so it is “unenlightened,“ meaning that users who
receive protected messages must go to the Office 365 encryption portal to access the content. This is obviously a sub-
optimal experience, so if you want these users to be able to read protected email on their devices, you can configure
server-side decryption. The downside of this approach is that decrypted copies of the message then exist on user devices.
To enable server-side decryption, run this command:

[PS] C:\> Set-ActiveSyncOrganizationSettings –AllowRMSSupportForUnenlightenedApps $True

When people use the mail app to view protected messages decrypted on the server, they see a header to tell them that
the sender applied protection to the message. The mail app does not apply the permissions given to the user, so they can
copy or print the message. However, Exchange Online knows that the original message is protected and if the user
attempts to do something without permission that involves the server, like forwarding the message, the action is blocked
when the server processes that message.

In most cases, if you want to protect email within a tenant, it is a better idea to encourage people to use the Outlook
clients rather than disabling encryption.

How Rights Management Templates Protect Email

The process to enable protection for a tenant creates a set of default rights management templates to allow users to
begin protecting confidential messages. We will discuss how to add a custom template when we discuss Office 365
sensitivity labels. The names of the default templates are:

Tenant Name - Confidential : Only recipients inside the company (users with an Office 365 user account belonging
to the tenant) can access the content, make edits, and share the content with others in the company.
Tenant Name - Confidential View Only : Recipients inside the company can view the content but they are not able
to edit or otherwise update it. Recipients can print the information and share it with other internal recipients.
Do Not Forward : Recipients can read the message but cannot print, forward, or copy its content.
Encrypt : The message is encrypted, but the recipient has full control over the content after it is decrypted.

If you use Azure Information Protection without Office 365, you see default templates named Confidential\All Employees
and Highly Confidential\All Employees .

During the enablement process for a tenant, Office 365 copies the Confidential and Confidential View Only templates into
the tenant's Azure Active Directory instance and renames them as Tenant Name – Confidential and Tenant Name –
Confidential View Only . For example, if your tenant name is Contoso, the two templates copied for your use are Contoso –
Confidential and Contoso – Confidential View Only . Normally, template settings define who can use it to protect content.
To make sure that anyone in the tenant can use the default templates, Office 365 binds them to a hidden group. You can
also use the default templates to apply protection through transport rules.

Templates Created for Office 365 Tenants

The Do Not Forward and Encrypt-Only templates are special because they are created automatically for every Office 365
tenant, meaning that so you can use the templates to protect messages sent to any user within Office 365, Outlook.com,
or users of other email systems. Outlook and OWA clients connected to Office 365 tenants and Outlook.com know how to
process messages protected with these templates, so their contents can be read as normal. Recipients in other mail
systems must go to the Office 365 Message Encryption portal to read the messages.

The Do Not Forward and Encrypt-Only templates grant viewer permission to all recipients. If you use another template to
protect messages sent to other domains, the recipients cannot read the content unless the template includes their email
address or their domain in the list of users and groups who can access content. You cannot edit the default templates to
change the permissions assigned to recipients.

You can edit the Confidential and Confidential View templates to change how they work (for example, to increase or
decrease the seven-day period the templates allow access to content when offline) or to restrict the groups who can
access the template. You cannot remove a default template, including the Do Not Forward and Encrypt templates, which
do not appear the list of templates shown in the Azure portal. If you want to stop people using the default templates, you
can archive the templates to stop users applying them to items. Table 24-1 lists the full set of access rights granted
through the default templates.

Access Right Confidential Confidential View Only Do Not Forward

View Yes Yes Yes

Edit Yes No Yes

Copy No No No

Print No No No

Save Yes No Yes

Export No No No

Access the item programmatically Yes No Yes

Full Control No No No

Reply Yes No Yes

Reply All Yes No Yes

Forward Yes No No

Table 24-1: Access rights granted by default protection templates

You can include a mixture of internal and external recipients, including those who do not use Office 365, in the recipient
list for a message. You can also send protected messages to Office 365 Groups, where users either access the message
through the conversations held in the group mailbox or as an individual copy delivered to their mailbox. However, Teams
does not have the ability to process protected message and protected messages are rejected if they are sent to a Teams
channel. If you send protected email to a Yammer group, Yammer responds via email to ask whether you really want to
post the message. If you do, Yammer decrypts the message and posts the content as text or a PDF file.

Messages carry the template for their entire lifetime. Any replies to messages inherit the same template to protect the
complete conversation.


Outlook for Mac and Protection : Outlook for Mac 2014 or later versions can access protected messages and apply
templates to messages. Earlier versions of Outlook for Mac do not support Office 365 protection.

Protecting Email with Outlook

To apply a template when composing a new message with Outlook, click Permissions in the Options tab to reveal the
set of available templates (Figure 24-4 ). Much the same approach occurs with Outlook for Mac where you select and add
permissions to a message through the Options tab.

Figure 24-4: Selecting protection for a message with Outlook 2016 for Windows

If attachments support protection, like all the Office document formats, they receive the same protection as applied to the
message. For example, if you attach a Word document to a message protected with the Confidential View Only template,
the recipients will be able to view its content but will not be able to edit it. For example, Figure 24-5 shows a Word
document open on an iPhone. This is an attachment to a protected message opened by Outlook for iOS and the user can
discover what rights they have over the content by clicking Permissions .

Recipients can open and edit attachments in file formats that do not support protection, like text files or bitmap images,
unless you protect the files with the Azure Information Protection app (covered later).

Any replies and forwards (if allowed) created in response to a protected message inherit the same level of protection.
Because the author always has full control over the content, they can decide to forward the message to another person,
reply to the original message, or apply a different template. Outlook does not support the recall of a protected message.
This is not as big a problem as it might seem because the speed of email servers today means that it is almost impossible
for a message recall to work, especially when a message travels to another domain.

A delegate can send protected content on behalf of a shared mailbox if they have full access and send as permissions for
the mailbox. In this scenario, the user granting permission is the shared mailbox. Delegates can also read protected
messages in a shared mailbox. Things are a little different with user mailboxes. Delegates can send protected messages
on behalf of someone else’s mailbox, but they cannot read protected messages sent to another user with Outlook desktop.
It is currently possible for a delegate to read protected messages with OWA due to the way that OWA fetches use
licenses. Microsoft acknowledges this as a problem and has committed to fix the loophole.
Figure 24-5: A protected Word attachment viewed with Word for iOS

Refreshing Templates for Outlook

When Outlook desktop (2013 or 2016) first protects a message, the client contacts the rights management service to
download the set of published templates and caches copies of the templates as XML documents in
%localappdata%\Microsoft\MSIPC\Templates . If the Azure Information Protection client runs on the workstation, it
refreshes policy settings every 24 hours. Policy settings that affect Outlook include the custom setting controlling the
display of the Do Not Forward button (explained later).

By default, Outlook checks with the service to refresh its settings every seven days, but you can change this by creating
either of two DWORD values at HKLU\Software\Classes\Local Settings\Software\Microsoft\MSIPC .

TemplateUpdateFrequency sets the update frequency in days. For example, set this value to 3 if you want Outlook to
update templates every three days.
TemplateUpdateFrequencyInSeconds sets the update frequency in seconds, if you want to get that precise.

If both values exist, Outlook uses TemplateUpdateFrequencyInSeconds . Templates and policy settings do not change all
that often and the default weekly refresh is enough for most purposes.

If you need to refresh templates without waiting for the frequency interval to elapse, you can force this to happen with
these steps:

1. Connect to the protection service and run the Get-AadrmConfiguration cmdlet. Note the value of the
RightsManagementServiceId . This is the identifier assigned to your tenant by the service.

[PS] C:\> (Get-AadrmConfiguration).RightsManagementServiceId

Guid

----

04dac1c9-6661-42f4-b94-c68551262cff

2. Open the system registry and move to HKLU\Software\Classes\LocalSettings\Software\Microsoft\MSIPC\

<RightsManagementServiceId>\Template\<User Principal Name> . The value of the RightsManagementServiceId is
the GUID retrieved in the first step plus something like rms.eu.aadrm.com. Locate the LastUpdatedTime value,
which tells you the last time Outlook refreshed templates, and remove this value.
3. Exit and restart Outlook. The client will connect to the protection service and refresh its set of templates.
This technique is useful when testing templates as you will probably go through multiple cycles of updating templates,
testing the effect of changes, and adjusting until you reach the desired state. Another thing you might want to do during
testing is sign-out of one account and into another. Usually, you sign into the protection service using the account you use
to sign into Office 365. The account information is in a file named TokenCache in the %localappdata%\Microsoft\MSIP
folder .

If the Azure Information Protection client is on your workstation (used to apply AIP labels to files), you can check the
account information by clicking Protect in any Office desktop application and then select Help and Feedback . Figure
24-6 shows an example of the information shown.

To clear the account information, exit all Office applications and then remove the file. You should then see a prompt to
sign-into the protection service when you restart any Office application. If you use single sign-on to connect to Windows
and Office 365, you must exit Windows and sign in again. Clicking Reset Settings in the Azure Information Protection
client also signs you out of the service and removes the current policy, but it will not allow you to sign in as another user
afterwards.

Figure 24-6: Viewing Azure Information Protection account information

Viewing Rights
When you open a protected item, you see the name of the template and some information about its intended use. Outlook
users can click a message header to gain more insight into what they can and cannot do with a protected item (Figure 24-
7 ). Outlook for iOS and Android support similar ways for users to see the full permission set. It is obvious from the list of
rights supported for email that you cannot block every conceivable action that a recipient can take with a message. For
instance, although you can remove the Copy right to stop someone copying text from a message or attached document,
including blocking screen captures on Windows devices, you cannot stop them taking a screen snapshot with a
smartphone and circulating that image to others. The point here is that protection cannot prevent every kind of
unacceptable user interaction with content. Users must accept some responsibility for their actions. Applying a protection
template makes users aware that the author protected the information for a reason and that they should therefore deal
with that information appropriately. What they do is entirely up to the user.

Figure 24-7: Outlook reveals the rights available for a message

Protecting Email with OWA

Because OWA runs in online mode, it does not need to download templates and always uses the latest published set. OWA
supports a slightly different method of applying a protection template. In this case, if the rights management
configuration for the tenant specifies $True for the SimplifiedClientAccessEnabled setting, the Protect button appears in
the menu bar for the new message window (Figure 24-8 ). Clicking Protect (or Encrypt in the preview version of OWA)
applies the Do Not Forward template to the message. If you want to use a different template, click Change Permissions
and select the template from the list.

Figure 24-8: Applying a protection template with OWA

Both Outlook and OWA use a stop sign icon (red circle with white bar) in folder lists to mark protected items.

Inbuilt Message Encryption

The Encrypt-only feature protects email with encryption without placing any restrictions on the recipients. The feature is
accessible when creating a new message in the following Outlook clients:

OWA: Select Protect , then Change permissions , and then choose Encrypt from the drop-down list.
New OWA: Select Encrypt .
Outlook for Windows (Click to run only, build 1804 and higher): In the create message window, click Options , then
Permissions , and select Encrypt-Only from the drop-down menu.
Outlook for Mac (build 16.19.18110915 and higher): In the create message window, click Options , select Encrypt ,
and then choose Encrypt-Only . In earlier versions, the Encrypt-Only template can be applied through the
Permissions menu.

Encryption happens within client code. In other words, messages are encrypted during the sending process and before
the client submits email to Exchange for onward processing. The only time Exchange applies encryption to email is when
a transport rule includes the application of a protection template as an action.

Like the Do Not Forward template, the “Encrypt-Only” protection policy is an out-of-the-box template available to any
Office 365 tenant after they configure IRM (all E3 and E5 tenants can automatically read encrypted email from other
tenants; the IRM configuration must be in place to allow users initiate new encrypted conversations). Unlike a protection
template, which restricts what a recipient can do after they receive a message, when you encrypt a message, recipients
have full rights over a message. In fact, protection through encryption is a form of right like the other rights found in
protection templates. The sender grants the recipient the right to decrypt and view the content. Put another way, the
sender implicitly trusts the recipient to do the right thing with the content when they receive it. This isn’t the case with
other protection templates, where the assumption is that the sender removes rights from the recipient so that they don’t
do the wrong thing. The important thing is that both approaches assure the sender that only authorized recipients can
access a message and its attachments.

The idea behind the Encrypt-Only policy is to encourage users to protect confidential messages. Unlike third-party
solutions such as S/MIME (also supported by Office 365) or PGP, users do not have to configure and manage certificates
or install plug-ins. To make the idea even more appealing, encryption works for messages sent to any email address. A
recipient in an Office 365 tenant can read encrypted content inline within a client, while users of other email systems can
read encrypted messages through the Office 365 Message Encryption portal. Recipients of encrypted messages
forwarded (by rules or when a forwarding address is set for a mailbox) cannot read the content because their addresses
are not in the original recipient list. However, if a user forwards an encrypted message with OWA or Outlook, the
recipient’s address is added to the message header and so they can read the encrypted content.

The Outlook mobile clients can read encrypted messages, and if you use the Outlook mobile client to reply to an
encrypted message, the reply is also encrypted.

Reporting message encryption . The Security and Compliance Center includes a preview report for message
encryption activity. Although in preview at the time of writing, the report gives a useful insight into how people use
encryption to protect email. To access the report, enter the URL https://protection.office.com/?
flight=enableencryptionreport#/reportv2?id=encryptionreport

Combining S/MIME and Protection

It is technically possible to combine S/MIME with rights management protection, but only if you use S/MIME to encrypt a
message and then apply protection afterwards. This can only be done using Outlook desktop (for Windows) as the client is
intelligent enough to resolve the inherent conflict between the two encryption schemes. Of course, technically possible
and feasible are two different concepts, and combining the two schemes might result in a message that the recipient
cannot process. A more practical approach is to select one scheme and use it everywhere.

Dealing with Encrypted Attachments

Attachments to messages protected with the Encrypt Only feature are also encrypted. This isn’t a problem for recipients
like Office 365 users who can decrypt attachments automatically because they obtain get the necessary use licenses.
However, if you send encrypted email to recipients of other email systems like Gmail, they can read the encrypted email
through the Office 365 Message Encryption portal, but they cannot download and access the attachments because they
cannot decrypt the files. You can force Office 365 to decrypt attachments and remove their protection on download by
setting the DecryptAttachmentFromPortal setting in your IRM configuration from the default $False to $True :

[PS] C:\> Set-IRMConfiguration –DecryptAttachmentFromPortal $True

You can also change the IRM configuration so that attachments downloaded by recipients with Azure Active Directory
accounts, such as those in other Office 365 tenants, have their protection removed on download. The effect is to give
recipients full control over the downloaded files. To change the configuration, run this command:

[PS] C:\> Set-IRMConfiguration -DecryptAttachmentForEncryptOnly $True

These settings only apply to messages protected with the Encrypt template.

Revoking Encrypted Messages

Recipients of protected messages fall into two categories: recipients in other Office 365 domains and Outlook.com who
see messages inline within email clients (in other words, no obvious decryption action is necessary), and those of other
email systems such as Gmail or Yahoo! who receive a link to allow them to view decrypted content through the OME
portal. Revocation, which means removing the right of a recipient to view encrypted content, is only possible for
messages sent to link recipients where control can be exercised through the portal.

To revoke an encrypted message, you need to know its message identifier. This can be found using the Message Trace
feature in the Security and Compliance Center. Execute a search to find the message, select it in the set of results, and
look at its properties. The message identifier is a long string ending in something like “prod.outlook.com.” When you have
the identifier, you can run the Set-OMEMessageRevocation (from the Exchange Online module) cmdlet to revoke
permission. For example:

[PS] C:\> $MessageId = "AM6PR0402MB3462A8DF67AF7AF9C9F0B20A8BE50@AM6PR0402MB3462.eurprd04.prod.outlook.com"

[PS] C:\> Set-OMEMessageRevocation -Revoke $True -MessageId $MessageId

The encrypted email with subject “ Plans for 2019 ” and Message ID “
AM6PR0402MB3462A8DF67AF7AF9C9F0B20A8BE50@AM6PR0402MB3462.eurprd04.prod.outlook.com ” was successfully revoked.

To check the status of the message, you can run the Get-OMEMessageStatus cmdlet. The results returned by the cmdlet
confirm that the message is revoked:

[PS] C:\> Get-OMEMessageStatus -MessageId $MessageId

RunspaceId : 4039a474-e2ab-498f-a92c-460154537c8f

Identity : 04e78939-85a9-4bf8-8d65-9f533648bb37

IsValid : True

ObjectState : New

Container : SystemMailbox{D0E409A0-AF9B-4720-92FE-AAC869B0D201}@office365itpros.onmicrosoft.com

Subject : Tony Encrypt

ReceivedTime : 7 Oct 2018 14:03:55

Revoked : True

When the recipient goes to the OME portal and tries to read the protected message, they see “the message has been
revoked by the sender” and are told to contact the original sender if they want to see the sender. An administrator rather
than the sender revoked the message and there’s no way for the original sender to cancel the revocation, so if a mistake
is made and the recipient should have access to the content, the original sender will have to resend the message.

Encrypt or Protect
Given the choice to encrypt or protect messages, what should you do? Here’s a simple rule of thumb.

Encrypt messages to protect confidential or sensitive data sent to recipients outside your organization. This
includes messages sent to Office 365 Groups (which can decrypt and show encrypted content in conversations to
group members). The situation for Teams is different because Teams uses “special” email addresses for channels
that don’t belong to a regular Office 365 tenant. When you send an encrypted message to a channel, it is rejected
because the Exchange transport service cannot decrypt and re-encrypt the message for the channel email address.
Protect messages with confidential or sensitive data sent to internal recipients or to recipients in domains you know
respect the rights expressed in protection templates.

This rule of thumb is based on the simple fact that encryption works for all email addresses, so it is the catch-all solution
when a need exists to protect content sent outside the company. Not every destination might be able to understand the
limitations set by rights templates, but if a template is configured to support recipients in an external domain, it is an
excellent way to protect information for the lifetime of the content.

Protecting Email Sent to Shared Mailboxes

You can send protected messages to shared mailboxes, but people who can access the shared mailbox might not be able
to read the content. The reason why is that messages are protected on a per-user basis. In other words, the user’s
identity establishes whether they have the right to access protected content. When you have permission to access a
shared mailbox (even full access), you connect to the mailbox using your account. Therefore, unless the template grants
your account rights to access the content, you cannot open protected messages.

The Encrypt and Do Not Forward default templates behave differently in OWA and Outlook. If you open a shared mailbox
(as a shared folder) with OWA, messages protected with the Encrypt or Do Not Forward templates can be read inline as
normal. However, if you open the shared mailbox with Outlook, the client displays the protected messages as a rpmsg
attachment with an instruction for the user to go to the OME portal to read the content. When the user clicks the link to
the portal, OME rejects the request because it cannot find the message.

The reason for the different behavior is that Exchange Online decrypts messages on behalf of OWA and ActiveSync clients
(including Outlook for iOS) if the ClientAccessServerEnabled setting of the IRM configuration is set to $True . Decryption
happens for any account that accesses a mailbox with full access permission, so delegates with full access to a shared
mailbox or another user’s mailbox can see the content of protected messages sent to that mailbox. Because Outlook can
run offline, the client does not depend on Exchange to decrypt protected messages and obtains the necessary use
licenses to access protected content from the rights management service. When a delegate tries to open a protected
message in a shared mailbox, Outlook asks for permission to access the content. Because the account is different to the
shared mailbox, the license is refused, and the user sees the default “go to the portal” text. However, accessing the portal
to see the content fails because the portal does not recognize the right of the user to access the message.

If you do not want delegates to be able to access protected content in shared mailboxes (or user mailboxes where a
delegate is granted full access by the mailbox owner), you can run the Set-IRMConfiguration cmdlet to set
ClientAccessServerEnabled to $False . The downside of this solution is that OWA and ActiveSync clients will no longer be
able to display protected content.

Office 365 Sensitivity Labels

The original implementation of rights management within Office 365 followed the approach taken on-premises and
focused on the application of templates to protect content. Over time, Microsoft developed Azure Information Protection
to become the overarching scheme for protection of content across Microsoft 365, including the ability for users to
classify content according to its sensitivity as that content is created. Users who generate content best understand its
nature and if documents and messages receive the correct label from the start, they carry that classification for their
entire lifecycle.
Azure Information Protection (AIP) labels were introduced in 2016. Applying a label to a document or email marked it as
holding content of a certain type as shown by the label. The higher the sensitivity of the content, the more stringent the
protection invoked by the label. Some labels were purely visual indicators for a certain kind of content, like “Public.”
Others imposed markers inside documents and email to help users understand that the information was more sensitive; a
label named “Confidential” might insert a footer and watermark in documents when applied. The final form of labels
invoked rights management templates to protect the most sensitive content. For instance, we might have a “Secret”
template that assigned a template of the same name to documents to grant viewer access to anyone in the tenant but
restrict access to anyone external. Documents protected with rights management can be tracked when they go outside
the tenant and access revoked at any time should documents fall into the hands of people who should not have them.

Table 24-2 summarizes the differences between the two types of labels. Although there are some important differences in
functionality, the major dividing line between these labels is their target market. AIP is a good solution for companies who
want to protect files stored outside Office 365 while Office 365 sensitivity labels focus on content stored in SharePoint
Online, OneDrive for Business, and Exchange Online. Behind the scenes, both types of labels are interoperable as they
use the same metadata and share a common set of rights management templates.

Feature Azure Information Protection Sensitivity Labels

Target content Any file available to Windows. SharePoint Online, OneDrive for Business, and
Exchange Online.

Licensing Separate Azure Information Protection (P1 or Included in Office 365 E3 and E5.
P2) needed.

Visual marking Included (includes per app) with support for Included (the same marking is used for all
custom front and color. apps)

Clients Azure Information Protection client for Integrated in Office desktop and online apps
Windows, iOS, and Android. (2019). Mac and mobile support due.

Encryption Uses Azure Information Protection (and Uses Azure Information Protection with cloud
S/MIME for Outlook) with cloud or HYOK. keys.

User-defined Supported. Not supported.

permissions for Office
apps

Define conditions to Based on matching sensitive data types or text Not yet available.
apply automatic labels patterns. Needs AIP P2 license.

Custom text Supported. Not supported.

Remove protection Supported Not supported

Table 24-2: Differences between AIP Labels and Office 365 Sensitivity Labels

Sensitivity Labels
Sensitivity labels are the implementation of protection labels within Office 365 and are managed through the
Classifications section of the Security and Compliance Center. The labels and policies found under Classifications are
divided into retention and sensitivity. Retention labels dictate how long content assigned a label is kept within Office 365
and what happens when its retention period elapses. See Chapter 19 for more information on retention labels. Sensitivity
labels dictate what level of sensitivity content has and how the tenant wishes to protect that content. At the highest level
of sensitivity, labels can invoke rights management templates to encrypt and protect documents and messages. Items can
be assigned a single sensitivity label and a single retention label, but not multiple labels of either type.
If you used Azure Information Protection labels in the past, you can migrate those labels to create sensitivity labels . After
the migration finishes, two sets of labels exist, both of which depend on the same rights management templates. Behind
the scenes, AIP and the Security and Compliance Center use the same code to interact with templates. Because they
share a common base, the two sets of labels insert the same metadata or tags in documents and messages, so the
transition from Azure Information Protection labels to sensitivity labels does not affect content. You do not need to
reapply or reclassify previously-labeled items.

Although you can use AIP labels to protect documents stored in SharePoint Online and OneDrive for Business, from an
Office 365 perspective, the long-term focus is on sensitivity labels, which are designed to make it easy for tenants to use
and manage encrypted content. The future focus for any Office 365 tenant who started out with rights management
should be on exploiting sensitivity labels rather than using AIP labels. This doesn’t mean that you can’t continue using
AIP labels because they are still valuable when used to protect files stored outside Office 365, such on file servers.

At the time of writing, client updates are needed to support sensitivity labels. Table 24-2 lists the current support for
sensitivity labels in different client platforms. Full integration with the Office desktop applications for Windows and Mac,
Office online apps, and Office mobile apps (iOS and Android) will come in early 2019.

Client Notes

Office Desktop Apps Install the preview version of the Azure Information Protection client to expose the Sensitivity
(Outlook, Word, button and apply sensitivity labels in released versions of the Office Click-to-Run applications. A
PowerPoint, Excel) preview of integrated support for sensitivity labels is available on limited release. The MSI version
for Windows. of Office applications don’t support sensitivity labels.

Office Desktop Apps Not yet available.

for Mac

Office online apps Not yet available.

Office mobile apps Not yet available. When an update for Outlook mobile is available, the connection must use the new
client architecture.

Other non-Office Use the Azure Information Protection client to apply labels to these files.
files

Table 24-2: Client support for Office 365 sensitivity labels.

Sensitivity labels are also supported by InTune and can be used with an app protection policy to ensure that content is
not copied to a third-party app like DropBox or removeable storage like a USB drive

Planning Labels
Before creating any sensitivity labels, it is sensible to chart out a plan for protection within the tenant. The plan should
include discussion of points such as:

What levels of protection are needed? Some labels can be text-only so that they serve as purely visual indicators of
an item’s sensitivity without taking any protective action. Others will take actions such as marking and/or
encryption.
How to translate the levels into labels and what names to give the labels. Names should be concise and help users
understand how to assign labels to content based on the sensitivity and confidentiality of the content.
If a label needs to apply marking (watermarks, headers and footers) to content and what that marking should be.
If a label needs to apply protection. If so, what rights are to be assigned to different user groups.

The goal is to create a practical set of general-purpose labels that make sense to the business without over-complicating
matters. We do not want to have labels that are never used, and we need labels that give enough granularity in terms of
confidentiality.

The plan might be documented in something like that shown in Table 24-3 , ordering labels in order of sensitivity from
least sensitive to most confidential. The Security and Compliance Center organizes sensitivity labels in this order and
Office 365 uses the order to know whether a user has replaced a label with one of higher or lower sensitivity.

Label Description Marking Protection Extended

Name Protection


Public Content that can be shared outside the Footer None None
company.

Internal Content that should remain internal but can Footer None None
be shared.

Confidential Content that should remain inside the Footer Yes None
company

Secret Content that should never go outside the Footer and Yes 7-day expiry
company watermark

Ultra “Eyes-only” content for restricted Footer and Yes, restricted to certain
circulation watermark groups

Table 24-3: Planning Sensitivity Labels

In addition to the set of general-purpose labels, the plan might include labels for use by certain projects or departments.
For instance, you could create a label to mark documents associated with a patent application.

Figure 24-9: A set of Sensitivity Labels in the Security and Compliance Center

Figure 24-9 shows a set of sensitivity labels displayed in the Security and Compliance Center. The labels are ordered as
described above, with some of the labels created for use by projects and departments coming after the general-purpose
set. You can also see the choices available in the ellipsis menu to reorder labels or to create a sublabel for a label.

Sublabels
Sensitivity labels can have sublabels. This means that you have a main or group label that isn’t used for labeling. Instead,
the main label is a container that serves as a logical collection of other related labels, each of which can have different
marking and encryption settings. Although main labels appear in applications, you can’t assign them to content. Instead,
users can select from one of the sublabels, which is then assigned to the content.
Creating a New Sensitivity Label
Creating a new sensitivity label goes through four steps:

1. Name : This is the name that users see in Office applications. You can also add a tooltip that the apps display when
users hover over the label name, and an administrative description that can be whatever makes sense for your
organization.
2. Encryption : Decide if the label invokes rights management. If yes, Office 365 creates a new template in Azure with
the same name as the label. You can assign rights to specific users and groups at this point.
3. Marking : Decide if the label should insert text in the header or footer of documents and messages, or as a
watermark (documents only).
4. Endpoint data loss prevention : Files with the label are checked by InTune app protection policies to ensure that
they are not moved to unauthorized devices or storage.

After going through the steps, you can review and change settings before finally saving the new label. At this point, Office
365 and Azure create the objects needed to instantiate the new sensitivity label.

In the following example, we create a new sensitivity label called “Black Matter” that gives protection and marking to
items to which it is applied. We then create a label policy to publish the new label so that it shows up in applications and
can be assigned to items by users.

Naming

Figure 24-10: Naming a new sensitivity label

Naming a label populates three properties (Figure 24-10 ). The name is the only value you must give because this is how
Office 365 refers to a label. The display name is set to the same value as a name when a label is created, and it is this
property that appears to users in Office applications. Although you can’t change the name of a label, you can change its
display name at any time through the Security and Compliance Center or with PowerShell.

The optional properties are a tooltip and description. The tooltip is intended as text that can be shown to users to help
them understand how to use a label. The Office apps display the tooltip when a user hovers over a label in the list shown
by the Sensitivity button. The description is only for administrator use and is never revealed to users, so it can store
anything that the organization considers necessary to note why a label exists and its general history.

Encryption
Encryption is either on or off for a label. A label without encryption does not give any protection to items to which it is
assigned. When encryption is on, Office 365 uses a rights management template to define the protection given to items.
The settings are:

Apply to : The choice is to restrict the label to Email only (use with Exchange) or email and documents (both
Exchange and SharePoint).
User access to content expires : If never, the user can continue to access labeled content without hindrance. In
some cases, like a draft for a plan, you will set a defined expiry date. In others, you know that content is only
valuable for a certain period, so you can set access to expire a specific number of days after a user applies the label
to an item.
Allow offline access : When offline access is allowed, the use license obtained by the user and downloaded along
with the content is used to gain access to the content. The use license is a certificate that attests to the user’s right
to access content and the encryption key used to decrypt the content. The normal validity of a use license is 30 days,
during which the user is not forced to reauthenticate and prove their access. However, you can block offline access
completely (for very sensitive information) or allow access for limited periods when offline, which means that no
 internet connection is available to check a use license. You can then further limit access by requiring the application
to check with the rights management service when online after a set period to ensure that access is still valid and
hasn’t been revoked. When a check is performed, group membership is also revaluated to ensure that someone who
gains their access by being in a group is still a member.
Permissions : The heart of rights management is the permissions given to recipients of an item. Permissions are
granted by the author or some other person with full access to the item to individual accounts, groups, or collections
of people (such as every authenticated user in a tenant). The granting of permissions is defined in terms of a set of
rights that the grantor authorizes. In the case shown in Figure 24-11 , three sets of permissions exist. The Viewer
permission is granted to all users in the tenant while the Co-author permission is granted to two distribution groups.
Any member of those groups can access the item using the permission granted to the group.

Figure 24-11: Encryption settings for a sensitivity label

Viewer and co-author are two of the predefined permissions defined in Office 365. The other permissions are co-owner
and reviewer. Each permission is built from a set of individual rights.

View content
View rights.
Edit content.
Save.
Print.
Copy and extract content.
Reply.
Reply all.
Forward.
Edit rights.
Export content.
Allow macros.
Full control.

Assigning a predefined permission to someone is a convenient way of giving them all the rights defined in the permission.
However, if the set permissions don’t meet your needs, you can choose to assign a custom permission made up of any or
all the available rights.

When someone receives protected content, Office 365 uses their signed-in account to check against the access granted to
the users and groups specified in the template. If the account cannot be found in the list of permissions, they won’t be
able to open the content. Whenever possible, it is best to grant rights to groups rather than individuals as this makes
permissions much easier to manage. You can’t grant permissions to the guest accounts used by Office 365 applications
such as Teams and Office 365 Groups. A template used to protect information that isn’t too confidential might also
include a catch-all assignment of the viewer permission to tenant owners. This ensures that content can at least be
viewed if you forget to add someone to a more highly permissioned group.

Collaborative Permissions
It has always been possible to define precisely who can share protected documents by adding permissions in a template
for certain users and groups. However, those permissions only work within the boundary of a single tenant. It is also
possible to add permissions to allow collaboration with people belonging to other domains by specifying their email
address or the name of their domain. You can grant access to people outside the organization by including their email
address. If you add a domain, any user belonging to that domain gets access.

The widest possible permission you can grant is “Any Authenticated Users.” This permission allows any user who can
authenticate with a Microsoft directory (for example, any account in any other Office 365 tenant or anyone with an
Outlook.com account) to access a document or message and have whatever rights you assign to them.

Warning: The migration of AIP labels to Sensitivity Labels is still in preview. A background process to keep AIP labels
and sensitivity labels synchronized exists and should make sure that changes made in either portal are reflected in the
other. Because the process is in preview, any time you make an important change to a label, like adding or removing
permissions, check that the changes replicate properly.

Owners
When someone applies a sensitivity label with encryption to a message or document, they are regarded as the owner or
issuer of rights for that content. The owner always has rights to access content, even if the policy sets an expiry date.
Likewise, the owner can always access content offline or open it after it has been revoked.

Link to Azure Information Protection Labels

When you create a new sensitivity label, Office 365 also creates a matching Azure Information Protection (AIP) label. If
the new label includes encryption, Office 365 also creates a rights management template. You can see and manage these
objects through the Azure Information Protection blade of the Azure portal and if you make changes in the Azure portal,
the changes are replicated to Office 365 if the updated property is supported by sensitivity labels (for instance, you can
set a color for an AIP label but not for a sensitivity label). However, changes made to sensitivity labels inside Office 365
are not replicated back to Azure.

One interesting aspect that is unique to AIP labels is the ability to allow Outlook to apply protection to email using
S/MIME instead of using rights management. This capability is intended for tenants who have invested in an S/MIME
infrastructure. Rights management based protection is more flexible and remains the default method used within Office
365. For more information about how to set up AIP to apply S/MIME with Outlook, see the online instructions for the
custom configuration .

Problems with Encrypted SharePoint Files

When you choose to protect files stored in SharePoint Online or OneDrive for Business with encryption, some Office 365
functionality suffers. Encrypted content cannot be indexed by SharePoint, so files can only be found through their
metadata (like their title or tags). Any encrypted files recovered by eDiscovery searches must be decrypted before
investigators can review their contents (we cover how to do this later). Co-authoring isn’t possible with encrypted content
and SharePoint can’t display a preview of an encrypted file. Finally, because Office 365 data loss prevention policies
depend on the indexed content of files stored in SharePoint and OneDrive for Business, those policies can only check the
metadata of the files and so could miss potential sensitive data stored in the file bodies. These issues usually pail into
insignificance against the reasons why encryption is used to protect confidential information, but they might come as a
surprise to the unwary.

Exchange transport rules do not have the same issue because Exchange uses super-user permission to examine protected
email (and attachments) as messages pass through the transport pipeline.

Content Marking
The content marking settings control if the application inserts visual indicators into content after a user applies a label.
Word, Excel, and PowerPoint support headers, footers, and watermarks and insert the markings as soon as a label is
assigned to a file, while Outlook only supports footers and inserts the text when a message is saved.
Figure 24-12: Content marking settings for a sensitivity label

Headers and footers are limited to 1024 characters, except for Excel, which limits these markings to 255 characters.
Watermarks are limited to 255 characters.

Publishing Sensitivity Labels

Before sensitivity labels show up in applications and can be applied by users, they must be published in a label policy. A
policy consists of:

One or more sensitivity labels.

A target audience. The default audience is everyone in the tenant. You can specify an audience by selecting Office
365 Group or distribution lists or individual users.
Settings to define whether one of the labels in the policy is mandatory and is therefore added by default to new
messages and documents, and whether users are forced to give a justification if they remove a label or replace a
label with a lower classification. A setting is also available to define a custom help page for users to consult to learn
about the proper use of labels.

A tenant can have multiple label policies, each of which has different sensitivity labels and target audiences. For instance,
you might create a general-purpose policy to publish a default set of sensitivity labels to everyone in the tenant and then
have a set of specific policies to publish certain labels to specific groups.

Select Labels and Target Audiences

To create a policy for a single label, select the label from the list and then click Publish label . Office 365 invokes the
policy publication wizard with the selected label already added (Figure 24-13 ). You can add other labels to the policy
with the Edit link. You can also create a policy from scratch by going to Label policies under Classifications in the
Security and Compliance Center and clicking the Publish labels button there.

Figure 24-13: Beginning the publication process with a selected label

The next step is to select the target audience who will be able to use the labels published in the policy. You can select
individual users, Office 365 groups, or distribution lists. Members of the groups keep access to the labels defined in the
policy for as long as they are members. In Figure 24-14 , we see that five users or groups are selected to receive the
policy. You can add more or refine the selected set by clicking the Choose users or groups link.
Figure 24-14: Specifying a target audience for the label policy

When you define a target audience for a policy, make sure that the users in that audience have rights to access any
encrypted content protected by a label included in the policy.

Publication Policy Settings

The policy settings allow administrators to select a label for automatic application to new messages and documents
created by the target audience, if created on a client that supports rights management (for example, sending email when
connected to Exchange Online via IMAP4 with the Thunderbird client won’t do anything). You can also set whether users
are forced to provide a free-text justification if they change the assigned label to another label with a lower classification.
The justification is logged in the Azure Information Protection logs rather than the Office 365 audit log. Finally, you can
define a web page for users to view if they want added information about how a label or labels should be used.

Figure 24-15: Defining settings for the label policy

The ultimate step in the publication process is to add a name and description for the policy (only the name is needed),
preview the policy, and then publish it. You can’t change the name of a label policy, but you can update its description at
any time. Users pick up the new policy when applications next refresh their cached copy of the policy, usually within a
couple of hours. At this point, the applications combine the labels published through all the policies that cover the user
and present the labels in a single list. Users cannot see details of the policies through which they access labels.

No Auto-Label Policies
Office 365 retention labels support auto-label policies to find and apply labels based on certain conditions such as the
presence of a sensitive data type or keywords. Sensitivity labels don’t have auto-label policies. Applying a label that might
encrypt content and affect the access users have to files is very different to assigning a retention period to items.

Sensitivity labels are not yet supported by Office 365 data loss prevention policies. When they are, you will be able to use
those policies to apply protection to any file detected to hold content of a certain sensitive data type or types.

Removing Sensitivity Labels

If you remove a sensitivity label from Office 365, the action does not affect access to content. The label metadata stored
in files stays intact, including the pointer to the rights management template which can be used by the rights
management service to confirm access to a file. If the template was removed, it would make the content inaccessible.
Without the template, the rights management service cannot grant permission to anyone to decrypt and access any file to
which the label is assigned. However, while the template stays in place, the removal of a sensitivity label results in the
removal of the associated AIP label from Azure.

In some cases, it might be a better idea to remove the sensitivity label from any policies it is part of so that users can no
longer apply the label. In this case, the label is still visible and works as before, but it cannot be applied to new content.

If you delete an AIP label, the label is also removed from Office 365. As in the case of a sensitivity label, the rights
management template stays intact.

Using Sensitivity Labels with Auto-Signature Products

Many ISV products exist that insert autosignature text into outbound messages. The autosignatures usually include
personal details about the sender plus some organizational information and a company logo. If you apply a sensitivity
label that protects content with encryption to an email, autosignature products might not be able to process messages
because Outlook or OWA encrypts the content when sending the messages. Some products have client-side plug-ins that
work by inserting text during message creation. These usually work because the autosignature is inserted before
encryption.

If you want to send protected email with autosignatures, you should test the available products to find one that supports
sensitivity labels. Alternatively, you can use a transport rule to insert an autosignature as messages pass through the
transport pipeline (see the example in Chapter 17). A transport rule can insert autosignature text even when messages
are encrypted because Exchange Online uses super-user privilege to decrypt the content, apply the autosignature, and
then encrypt the message again.

Applying Protection with Transport

Rules
All email sent by clients connected to Exchange Online mailboxes must pass through the transport pipeline on the servers
that host the sending mailboxes. As email passes through the pipeline, the criteria defined by the tenant in transport
rules (also called mail flow rules) are used to examine messages. If an email matches the criteria set by a rule, Exchange
applies the action or actions defined in the rule. The rule set defined by a tenant can span hundreds of rules to account
for different circumstances, and Exchange examines each message against the criteria in each rule until it reaches the
end of the rule set or the actions set in a rule cause rule processing to halt.

Adding protection to messages by applying a template is one of the actions available for transport rules. This capability is
often used to ensure that protection is applied to messages even if they originate on clients that do not support rights
management, such as the default mail apps for iOS and Android. Organizations often use transport rules to apply
protection to ensure that confidential information cannot pass in an unencrypted form to recipients outside the tenant. A
rule can apply protection using criteria such as:

Any message sent to specific domains.

Any message sent to specific users.
Any message that contains a specific phrase in the message text or an attachment.
Any message that has a specific word in its subject.

When a transport rule protects outbound messages, the senders of those messages might not be aware that this
processing occurs because the copy of the item in their Sent Items folder has no sign that protection is applied. This is
because transport rules apply templates in the transport pipeline to protect the copies delivered to recipients. The copy
of the message in the sender’s mailbox has not been through the transport pipeline and is therefore unprotected. If users
want to see a protected copy, they must include their name as a recipient.

Configuring a Transport Rule for Protection

Transport rules are configured through the Mail flow section of the EAC (for more information on this topic, see the
discussion about transport rules in Chapter 17). In Figure 24-16 , we see a new transport rule being built to do the
following:

Monitor messages sent by members of the Executive Committee distribution group; and
Apply the “Confidential” template to these messages. This is an action in the modify message security set.

Transport rules do not apply templates to messages already protected with a template. This is logical because the author
has already explicitly applied protection to the message with what they believe is the right level of protection. Overriding
protection selected by a user with a rule runs the risk of interference with the recipient’s ability to process the message
in the way intended by the author.

Unless they belong to another Office 365 domain or use Outlook.com, recipients of protected messages outside the tenant
do not receive a copy of the message. Instead, they get a notification with a link to the Office 365 encryption portal
(Figure 24-18 illustrates an example notification). After they sign into the portal, they can read and responds to the
message using a special form of the OWA reading window. Users can use reply, reply all, or print encrypted messages. If
you want to limit their rights further, you can protect the message using a custom template. Any attachment sent with the
message also receives protection.
Figure 24-16: Applying protection with a transport rule

Transport Rule Priority : As is the case any time you create a transport rule, you must be careful about the priority a
rule that applies protection has within the rule set. Organizations use transport rules for many purposes, including Data
Loss Prevention checking, enforcing ethical firewalls, applying corporate disclaimers, and so on. Each rule has a priority
number (zero or 0 is the highest priority), and it is possible that a higher-priority rule might specify the action to stop
processing further rules after it completes. If this is the case, the rule will never apply protection because the rule will
never fire. Some caution is therefore necessary to understand exactly how rules interact with each other to process
messages as they pass through the transport pipeline. See Chapter 17 for more information about transport rules.

Exemptions
Often rules include an exception condition to allow users to override the rule in certain circumstances. For example, if the
subject of the message has a specific text pattern, the rule will not apply the template. When a rule allows a message to
pass without intervention due to an override, because the transport captures the fact that it processed the rule in the
message header, Exchange Online will not try to apply the rule again to replies and forwards that flow from the original
message.

Another common exception is not to apply protection when the recipient is a member of a specific group. The logic here
is that the members of the group might need to change or update the content circulated in email.

Although distribution groups are an excellent way to specify the users that come under the scope of a transport rule,
remember that Exchange Online caches group membership to avoid the performance penalty of going back to Azure
Active Directory to expand membership each time a group is in a message header. For this reason, do not expect a
change made to a group used in a transport rule to be effective at once. It can take between 30 minutes and an hour
before the transport service learns about the new group membership.

Interlocking protection : You can apply a mixture of Data Loss Prevention checking and protection templates to make
sure that sensitive data does not leave the organization unprotected. If the transport service detects sensitive data
protected by a DLP policy in a message, a transport rule can apply a template. This level of interlocking protection helps
organizations ensure that people do not misuse sensitive data.
Applying Protection in Transport Rules with PowerShell
Office 365 offers two ways to apply templates to outbound email. The traditional approach that we’ve just explored is to
use a transport rule; a newer approach is to specify protection as an action in a DLP policy. You can find information
about how to use templates in DLP policies in Chapter 22. Because of the complexities involved in many rules and
policies, it is often easier to build them through the relevant GUI (EAC for transport rules, Security and Compliance
Center for DLP policies), but PowerShell can also be used for the task.

The New-TransportRule cmdlet creates a new transport rule. In this example, we create a rule that applies the “Sensitive
Board Reports” template to email sent from one distribution group to another distribution group.

[PS] C:\> New-TransportRule -Name "Protect Board Meeting Information" -FromMemberOf "Board Reports"

-SentToMemberOf "Board Members" -ApplyRightsProtectionTemplate "Sensitive Board Reports"

-ExceptIfSubjectContainsWords @("Override") -StopRuleProcessing:$False -Mode Enforce

-Comments "Protect Board Information when circulated" -RuleErrorAction Ignore

-SenderAddressLocation Header

Another example is where you want to protect messages that contain sensitive data types such as credit card numbers.
Office 365 has a wide range of default sensitive data types created for data loss prevention (DLP) policies (see Chapter
22). You can define your own sensitive data types if necessary. Any sensitive data type can be used in a transport rule to
identify messages for protection by including it in the MessageContainsDataClassifications parameter. Here’s a simple
PowerShell example that looks for six different sensitive data types. If any are found in a message, Exchange Online
applies the Encrypt template.

[PS] C:\> New-TransportRule -Name "Encrypt external email with PII content" -SentToScope NotInOrganization -ApplyRightsProtectionTemplate
"Encrypt" -MessageContainsDataClassifications @(@{Name="ABA Routing Number"; minCount="1"},@{Name="Credit Card Number";
minCount="1"},@{Name="U.S. / U.K. Passport Number"; minCount="1"},@{Name="U.S. Bank Account Number"; minCount="1"},@{Name="U.S.
Individual Taxpayer Identification Number (ITIN)"; minCount="1"},@{Name="U.S. Social Security Number (SSN)"; minCount="1"}) -Mode Enforce

Alternatively, you can create a DLP policy that applies a template when messages are shared outside the organization.
Two steps are needed to do this in PowerShell. The first creates a DLP policy; the second creates the rule to encrypt
email with the same set of sensitive data types specified for the transport rule and attaches the rule to the policy.

[PS] C:\> New-DlpCompliancePolicy -Name "Encrypt external sensitive mail" -ExchangeLocation "All"

[PS] C:\> New-DlpComplianceRule -Name "Encrypt external email with PII content" -Policy "Encrypt external sensitive mail" -AccessScope
NotInOrganization -EncryptRMSTemplate "Encrypt" -NotifyUser "LastModifier" -NotifyPolicyTipCustomText "This email contains sensitive PII
information and will be encrypted when sent." -NotifyEmailCustomText "This email contains sensitive PII information and will be encrypted
when sent." -ContentContainsSensitiveInformation @(@{Name="ABA Routing Number"; minCount="1"},@{Name="Credit Card Number";
minCount="1"},@{Name="U.S. / U.K. Passport Number"; minCount="1"},@{Name="U.S. Bank Account Number"; minCount="1"},@{Name="U.S.
Individual Taxpayer Identification Number (ITIN)"; minCount="1"},@{Name="U.S. Social Security Number (SSN)"; minCount="1"})

As you can see, Office 365 offers several ways to apply encryption via policy to outbound email. It’s important that you
choose either transport rules or DLP policies to protect sensitive data as it is easy to cause confusion if protection is
applied for the same content using multiple methods.

Applying Sensitivity Labels with Transport Rules

As we know from the discussion earlier in this chapter, a sensitivity label can be associated with a protection template.
However, if you apply a template to a message using a transport rule, the message is not labelled. This is because a label
exists for a message through the presence of an x-header called msip_labels . The value of the header points to the GUID
for the sensitivity label. GUIDs are used instead of text values to allow clients display label names in local languages. If
the msip_labels x-header is not present in a message header, clients that understand labels cannot display a label. Thus, if
we want to apply a label to a message, we must do so by adding the label using a client or by using a transport rule as the
message passes through the transport pipeline.

In Figure 24-17 we see what needs to be done. This transport rule is very simple. It does the following:

If the message subject or body contains the word #Confidential, execute the rule. This is an easy way of allowing
people who use clients that don’t support Office 365 rights management (like Mac or iOS devices) to instruct
Exchange to protect messages. The rule applies to messages sent inside and outside the organization.
Apply the Encrypt template to protect the message.
Add MSIP_Label_1b070e6f-4b3c-4534-95c4-08335a5ca610_Enabled=True; as the value of the msip_labels x-header
in the message. The closing semi-colon is important as it terminates the x-header.
Figure 24-17: Setting a transport rule to add the msip_labels x-header to a message

The inevitable first question for anyone wanting to use a sensitivity label in a transport rule is how to find its GUID. It is
found at the bottom of the label properties in the Azure Information Protection console (or can be retrieved by running
the Get-Label PowerShell cmdlet as described later in this chapter). You can combine labels with templates. In other
words, even if a label includes protection, you can decide that a transport should apply an x-header for one label and
protect the message with a different template (that isn’t associated with a label). If you want to use a sub-label, make
sure you use its GUID and not that of the parent label.

Remember that the senders of messages that are processed by this rule will not see a label or protection on the copy of
the message in their Sent Items folder because protection is only applied to messages that go through the transport
pipeline. If you want to check that the rule works as expected, include your address as a recipient and examine the copy
of the message that arrives into your Inbox with a client that supports sensitivity labels. You should find the msip_labels x-
header in the message headers and that the message is both labeled and encrypted. It’s also true that the label
information will mean nothing to a different email system (or different Office 365 tenant) because those systems won’t be
able to translate the GUID into a label. However, the protection will still apply even if the label isn’t respected, and that is
probably the most important thing.

Customizing the OME Configuration

Exchange Online sends notifications to users when they receive protected email that their email service cannot process.
The notification directs the recipient to access the Office 365 encryption portal where they can sign in using an account
from one of the federated identity providers. If they cannot sign-in or prefer not to, the recipient can get a one-time code
to read the content. Once signed-in, the user sees a modified version of the OWA read message window, which enforces
the permissions they have over the content.

You can customize the OME configuration to add custom text in the notifications for protected messages or to deliver
one-time codes as well as some elements used in the portal. To start, use the Get-OMEConfiguration cmdlet to view the
details of the current OME configuration. In this case, the configuration has several customized settings, which we can
update further with the Set-OMEConfiguration cmdlet.

[PS] C:\> Get-OMEConfiguration

Image : {137, 80, 78, 71...}

EmailText : Thank you for communicating with our company. To protect our confidentiality, this message has been encrypted.

PortalText : Our Great Encrypted EMail Portal

DisclaimerText: This message is intended for the address only and should not be viewed by idle miscreants

OTPEnabled : True

Identity : OME Configuration

The settings are:

 EMailText : A string of up to 1024 characters to show the purpose of the encrypted message. The default text is
“You've received an encrypted message from ” plus the SMTP address of the sender. If you change the default
message, OME does not include the sender’s SMTP address (but obviously still appears in the message header).
PortalText : A string of up to 128 characters shown when users connect to the Office 365 Message Portal to
download a decrypted message.
DisclaimerText : A string of up to 1024 characters placed at the bottom of the encrypted message intended as a
disclaimer but can convey a different message. The default text is “This email message and its attachments are for
the sole use of the intended recipient or recipients and may contain confidential information. If you have received
this email in error, please notify the sender and delete this message .”
Image : An image file (.png, .jpg, .bmp, or .tiff) of less than 40 KB to show that messages originate from the sender’s
company. Ideally, you might use a 170x170 pixel version of the company logo. By default, OME does not display a
logo.
BackgroundColor : You can set the background color for OME messages and the portal by passing the hex code
color. Usually, people who know about corporate branding will know the right code to use.
OPTEnabled : If true (the default), recipients can opt to get a one-time code to access the Office 365 Encryption
portal. Set this to False if you want to force users to authenticate using an account from one of the supported
providers.

For example:

[PS] C:\> Set-OMEConfiguration –Identity "OME Configuration" -BackgroundColor "#5183cd"

-DisclaimerText "We take zero responsibility for anything that happens here." –EmailText "Yippe! You’ve got some encrypted super-secret
email"

A slightly different approach adds the graphic file for the logo used in notification messages and the portal.

[PS] C:\> Set-OMEConfiguration –Identity "OME Configuration"

–Image (Get-Content "C:\Temp\CompanyLogo.jpg" –Encoding Byte)

We can see the effect in Figure 24-18 . The logo appears at the top of the message, the email text appears under the link,
and the customized disclaimer text is in place.

Figure 24-18: Viewing a notification that a protected message is available for viewing

Moving from Office 365 Message Encryption V1

Microsoft introduced Office 365 Message Encryption (OME) in mid-2015 as a method to protect outbound messages.
OME V1 uses the Apply Office 365 Message Encryption action in transport rules to encrypt messages. Like with template-
protected messages, recipients get a notification telling them to access the OME portal to access the protected content.
However, recipients can only read the content and are unable to use the other functionality exposed through rights
granted by templates. In October 2017, Microsoft deprecated the functionality delivered through OME V1 in favor of
protecting all email through rights templates.

The older Apply Office 365 Message Encryption action is still available for transport rules but is now in maintenance
mode. Microsoft does not support the action for new deployments because they want tenants to use protection templates.
The recommendation is that tenants upgrade transport rules that use the older action to use protection templates instead
as soon as possible., In addition, if you used OME V1 protection in the past, you might have a rule to remove this
protection from messages received from other Office 365 tenants. As Office 365 tenants upgrade their rules to replace
OME V1, a rule to remove encryption becomes less valuable. Although there is no harm in leaving an unused rule in
place, you can remove or disable the rule once you are satisfied that no other tenant sends messages protected with the
older technology to your tenant.

Protecting SharePoint Online and

OneDrive for Business
Office 365 includes two ways to protect files stored in SharePoint and OneDrive for Business.

You can apply a rights management template to protect items downloaded from a library or list. This is the older
form of protection that is also available in SharePoint on-premises and is intended to limit the set of actions users
can take after they download files. It does not protect individual items held in the library or list.
You can assign Office 365 sensitivity labels to individual documents in a library. The label and associated protection
stay with their assigned documents even if they are moved from their original library.

The first method is less preferable because protection depends on files being in a certain library. Assigning protection to
individual files is better because the protection persists no matter where the file travels, including outside the
organization.

Enabling Rights Management for SharePoint Online and OneDrive for Business

Figure 24-19: Enabling Rights Management for SharePoint Online

Before you can apply a default template to a SharePoint Online document library or list, you must enable use of the rights
management (IRM) service for the tenant:

Open the SharePoint Online Admin Center and select Settings .

Scroll down to the Information Rights Management section and select “Use the IRM service specified in your
configuration ” instead of the default “Do not use IRM for this tenant.”
Click Refresh IRM Settings (Figure 24-19 ).
Note the response “we successfully refreshed your settings.” Rights management is now enabled for SharePoint
Online and OneDrive for Business.

Protecting Document Libraries and Lists

Figure 24-20: Protecting a SharePoint Online document library

With rights management enabled for SharePoint Online, you can go ahead and protect document libraries and lists. Go to
the library or list that you want to protect and open it.

Click the cogwheel icon and select Library settings for a library or List settings for a list.
Select Information Rights Management under Permissions and Management. If you do not see this option, rights
management is not enabled correctly for SharePoint.
You can then select the rights management template to apply to items downloaded from the library or list along with
other settings (Figure 24-20 ).

The permissions and rights defined in the IRM template assigned to the library or list sets the protection for files
downloaded from a document library or list. This does not prevent you protecting an individual document before
uploading it to a library, in which case the template applied to the document persists.

Among the available settings to control rights management for a library are:

Do not allow users to upload documents that do not support rights management : Microsoft Word, Excel,
and PowerPoint file formats support rights management but others might not, in which case SharePoint blocks any
attempts to upload files in those formats. You can upload protected Adobe PDF files to a document library, but you
cannot view them thereafter unless your PC has a PDF viewer available that supports Rights Management See this
page for more information.
Allow viewers to print : Because SharePoint Online is often the basis for joint document authoring, it is usually a
good idea to allow people to print off documents that they download from a library, but you can stop them printing if
needed.
Allow users to write on a copy of a downloaded document : This controls whether a user can apply digital ink to
write comments on downloaded documents.
After download, document access rights will expire after these number of days : One way to protect
confidential information is to give it a “best by” date by setting an expiry period for access rights. You can decide
that users have between 1 and 365 days to view downloaded content, or you can leave this setting blank to allow an
indeterminate period. Another is to set the Users must verify their credentials using this interval checkbox and
define a period (in days) after which users must reauthenticate to maintain access.
Allow group protection : You can allow users to share information from the library with other members of an Azure
Active Directory group or an Exchange Online distribution group.
PDF files that are protected by SharePoint when downloaded from a document library do not use native PDF encryption
and therefore cannot be read by Acrobat. Use the Azure Information Protection viewer to view the content of these files.
The intention is that SharePoint will support native PDF protection in the future.

Synchronizing Protected Libraries with OneDrive

To synchronize content from protected libraries with the OneDrive sync client, the following software must run on the
workstation:

Version 17.3.7294.0108 (or later) of the OneDrive sync client.

Rights Management Service client.

The OneDrive sync client only supports SharePoint Online protected libraries. It does not synchronize protected content
from on-premises libraries. When you change the IRM status for a library, OneDrive resynchronizes the complete library
to make sure that it respects the new status (document attributes rather than complete files are downloaded). For this
reason, don’t change the IRM status for a library containing thousands of files without thinking about what will happen.

OneDrive for Business Permissions

OneDrive for Business sites are personal and therefore need some individual attention before a site’s owner can protect
documents. You can read how to apply protection using steps for an individual user or how to use scripts to enable
protection for multiple sites at this link .

Protecting Individual Documents

You can change the permissions assigned to a Word, Excel, or PowerPoint file by clicking the Sensitivity icon in the menu
bar and selecting the sensitivity label to apply (Figure 24-21 ). If the sensitivity label includes encryption, Office 365
applies the rights management template the next time the file is saved.

Figure 24-21: Applying a sensitivity label to a Word document

When a document is protected individually, the following SharePoint Online features do not work:

Co-authoring.
Editing of the document with Office Online.
As discussed above, document Preview and thumbnail.
eDiscovery (the metadata of the document is indexed and is discoverable; the content is not).
Data Loss Prevention (because the content is unavailable).

For more information, see this support article .

Individually Protected Documents in Unprotected Libraries

If you upload a protected document to a SharePoint or OneDrive library that is not configured for protection, the
document keeps its protection. Likewise, if you edit a file from a SharePoint or OneDrive library and apply a sensitivity
label that includes encryption to it, the protection assigned by the label is used.
Functionality Lost for Protected Documents
The online versions of the Office apps can open protected documents as they use the user’s credentials to open
documents: However, the preview function that is usually available for documents stored in SharePoint and OneDrive for
Business document libraries is unavailable when a library is protected (Figure 24-22 ). This is because the preview mode
of the online versions of the Office apps cannot secure the necessary license to decrypt the content. The Preview feature
does not work when documents are viewed through Delve. Co-authoring is also unsupported for documents in a protected
library. Only one person can edit a document at a time.

Figure 24-22: Document preview is unavailable for protected content

Other functionality lost when you protect individual files in SharePoint libraries include co-authoring and the ability for
SharePoint to fully index document content. In turn, the inability to index means that Office 365 content searches or
SharePoint search cannot find protected documents based on their content (searches can use file metadata to find
protected documents), nor can Office 365 apply DLP policies to protected documents.

Indexing Labels with SharePoint

Office documents, spreadsheets, and presentations store label information in their file attributes. When you store a
labelled file in a SharePoint or OneDrive for Business document library, the label data is captured in crawled properties
(something extracted from a file when the SharePoint crawler processes a file). The properties include Sensitivity , which
stores the local language version of the label applied to the stored document (for example, “Confidential” or “Public”).
You can remap the Sensitivity crawled property to one of the RedefinableString preconfigured managed properties to
make the label searchable. The SharePoint schema includes a set of 200 RedefinableString managed properties (from
RedefinableString00 to RedefinableString199) to allow tenants to customize search behavior. Because Azure Information
Protection labels and sensitivity labels stamp the same metadata into files, the same indexing behavior works no matter
what label is applied.

To make label data searchable, use the Search section of the SharePoint Admin Center to remap the Sensitivity crawled
property to one of the RedefinableString properties (for example, RedefinableString01). It is also a good idea to give the
property an alias (for instance, SensitivityLabel ) to make it clear what the data is. Because the crawler must process a
file before the label data is searchable, you should reindex the sites that hold classified information to force the crawler
to find and index the label data. After the label data is added to the index, you can search for it using terms such as
“SensitivityLabel:Confidential .”

Hybrid Protection
In a hybrid deployment, the on-premises Active Directory RMS servers support on-premises mailboxes and Office 365
supports cloud mailboxes. Automatic synchronization of configuration data, including templates, does not occur. You
must:

Export the trusted publishing domain (TPD) data from your on-premises RMS servers .
 Allow external access to your on-premises servers.
Import the TPD data into the rights management service.
Enable any on-premises templates that are in use and make them available to cloud users.

It is obviously desirable to ensure that the same templates are available to both cloud and on-premises users and that
they run in the same manner. Because no automatic synchronization exists, any time an administrator updates the
configuration for either the on-premises or cloud platforms, you must replicate the change to the other platform.

Super-Users
Super-users are accounts that can decrypt any protected content. On-premises deployments often use super-user
permissions to decrypt content to allow examination as messages pass through the Exchange transport service or when
they review items following retrieval by eDiscovery searches. Assigning highly privileged access like super-user status is
something that needs control, restricted to as few people as possible, and audited to track the user of super-user
privileges.

However, Exchange Online functions in a very different environment. It is under Microsoft’s control and tenant
administrators do not have access to servers and other basic infrastructure elements that they could use to compromise
information. Because Office 365 is locked-down, Microsoft runs a different regime where Office 365 components decrypt
protected content automatically so that transport rules can access protected content, discovery searches uncover
protected content with ease, and journaling generates unencrypted reports. You do not need to nominate any account as
a super-user to make any standard functionality in Exchange Online or SharePoint Online work.

It is possible that a situation might exist where you need to assign super-user status to an account to allow the account to
access encrypted some content. For example, you might have a set of protected Word documents left by an ex-employee
that investigators need to review. Another example is where a set of protected documents is recovered by an eDiscovery
search. Searches often find protected content based on their metadata, but the content stays encrypted when copied by
the search. In both instances, a super-user can decrypt the protected documents. Super-users are even able to decrypt
documents after a tenant archives the protection templates used for protection or after a document expires. The only
time a super-user is unable to decrypt a document is after the deletion of a template. If the template does not exist,
decryption is impossible.

You must enable the super-user feature before you can nominate accounts to be super-users. To enable or disable the
super-user feature, first load the protection cmdlets into a PowerShell session and connect to the rights management
service. The complete set of commands to enable or disable the super-user feature are shown below:

[PS] C:\> Import-Module Aadrm

[PS] C:\> Connect-AadrmService

[PS] C:\> Enable-AadrmSuperUserFeature

[PS] C:\> Disable-AadrmSuperUserFeature

After enabling the super-user feature, you can add accounts to the super-user list. The email address that you pass must
be valid for a user account belonging to the tenant, including those synchronized from on-premises Active Directory. You
can add multiple users at one time, separating each email address with a comma.

[PS] C:\> Add-AadrmSuperUser –EmailAddress Kim.Akers@Office365ITPros.com

The other commands used to manage super-users are straightforward. Get-AadrmSuperUser returns a list of the current
accounts that have super-user privilege. Normally this list should be empty and the presence of any account on the list
should have justification for the status. To remove an account from the super-user list, run the Remove-AadrmSuperUser
command as shown below:

[PS] C:\> Remove-AadrmSuperUser –EmailAddress "Kim.Akers@Office365ITPros.com"

Azure Information Protection logs all additions and removals of super users. You can report this access using the Get-
AadrmAdminLog cmdlet. See this article for more information on how logging occurs.

Protecting Individual Files

From 2013 onward, all versions of the Office desktop applications support the ability to protect content natively. In other
words, you do not have to do anything specific to use Outlook, Word, Excel, and PowerPoint to protect email and
documents after you configure protection for a tenant. The choice to apply the same templates to documents that are
available to protect items in an Exchange Online mailbox is in the “backstage” area (click File to expose the back-stage
area), where selecting Restrict Access under the Protect Document drop-down menu lists the available templates.

Two additional methods exist to apply protection to individual files:

The original Rights Management sharing app is still available and supported by Microsoft until January 31, 2019.
You can download versions for Windows and Apple Macintosh (OS 10.9 and later). You only need this app if you want
to protect non-Office documents.
The unified Azure Information Protection client is available for Windows . This is the client to use going forward. The
Azure Information Protection user guide has some information to help administrators plan the transition to the new
 client. The Azure Information Protection client exposes the protection bar within Office applications.

Both clients integrate with Windows File Explorer. Users can select and apply protection to files in much the same way as
they perform other operations like printing. You can protect multiple files and folders in a single operation. To protect a
file, you select it in File Explorer and select Classify and Protect from the context-sensitive menu (Figure 24-23 ). The
options exist to merely classify a file or to apply full protection where you specify a list of users who can interact with the
file together with the permissions you want to give them. For convenience, roles like Viewer – View Only and Co-Owner
dictate the actions that recipients can take when they receive protected files. You can select recipients from the corporate
GAL or input the email addresses for individual recipients, distribution lists, or complete domains. As you can see in
Figure 24-24 , you can also add an expiration date after which recipients no longer have access to the content.

Figure 24-23: Selecting a file to protect with the Azure Information Protection app

After you protect and classify a file, the client updates its properties to apply protection. In some cases, like Office
documents, this is a matter of adjusting the built-in attributes that control access to the file. In others, like JPEG files, the
client converts the file into another format (in this case, protected JPEG). After protecting the file, you can share it with
anyone you like using whatever mechanism is best. For instance, you can attach the file to a message and send it to
someone. The protection placed on files works no matter how users share files, including uploading files to cloud sharing
services like DropBox.

If the recipient is on the list of the users specified to access the content, they will be able to open it and interact with the
content based on the rights granted by the author. Anyone else will be unable to access the content and told that they
need to contact the owner to receive a version of the file that includes them in the permissions list. If the file is in a
format supported by an application that supports rights management, the recipient can open and interact with the file in
that application. However, if the file is in a format that is unsupported by enlightened applications or such an application
is unavailable, you can download a lightweight viewer that understands how to interpret the protection placed on the file
and to open it for viewing.

As noted earlier, as part of the introduction of sensitivity labels, Microsoft’s plan of record is to incorporate protection
functionality into the Office applications to remove the need for a separate installation. When this happens, the client will
only be necessary to set labels and permissions on non-Office files.
Figure 24-24: Applying a classification and protection to a file

Protected PDFs
In December 2018, the joint Microsoft and Adobe project to deliver a “native” integration of rights management
protection for Adobe PDF documents reached general availability. Native means that the PDF files are protected using
the ISO standard for PDF encryption and can be opened and processed by applications which support the standard.
Microsoft’s list of PDF readers that support Azure Information Protection is available online .

Unlike earlier third-party implementations of rights-management protected PDFs, which use a PPDF file extension to
show the encrypted nature of the files, native-protected files keep their PDF file extension.

To apply native protection to PDF files, you need the latest version of the Azure Information Protection client . You can
then protect PDFs by selecting them in File Explorer and using Classify and Protect to choose a label or custom
permissions to apply. To read the files, use a supported reader that understands both the ISO standard and how to display
the rights assigned by Azure Information Protection, such as Adobe Acrobat Reader (third-party readers like Foxit are
also available). To use Acrobat, you need the latest version of Acrobat Reader DC and the Microsoft Information
Protection plug-in . Both the reader and the plug-in must be installed. Protected PDFs can be stored in SharePoint Online
or OneDrive for Business document libraries, but you must download the files to access their content as the online
viewers included in Office 365 can’t decrypt the files.

Using PowerShell to Protect Information

A set of PowerShell cmdlets is available to protect and apply Azure Information Protection labels to collections of files
through scripting. The download for the tool and documentation for the cmdlets are both available online. These cmdlets
do not process Office 365 sensitivity labels.

Remember that your account must have administrative rights for the rights management service before you can run the
Connect-AadrmService cmdlet to connect PowerShell to the rights management service. Accounts have administrative
rights if they are:

A global administrator for the Office 365 tenant.

A global administrator for the Azure tenant.
Granted administrator access with the Add- AadrmRoleBasedAdministrator cmdlet. For example, this command
grants administrator access to the Kim Akers account.

[PS] C:\> Add-AadrmRoleBasedAdministrator -EmailAddress Kim.Akers@Office365itpros.com

Retrieving Protection Information

The Get-AipFileStatus cmdlet returns protection properties for files, including the classification and templates applied to
files. The code in this example checks for files that have an Azure Information Protection label we know (from the label
settings) is applied to files automatically because they hold some confidential information. Note that the cmdlet cannot
return the status for a file if it is open in an application.

[ PS] C:\> Get-AipFileStatus -Path “C:\Office 365 for IT Pros - Fifth Edition Files” | ? {$_.LabelingMethod -eq “Automatic”} | Format-
Table FileName, MainLabelName, LabelDate

Another example shows how to find files in a folder protected by a specific template.

PS] C:\> Get-AipFileStatus -Path “C:\Office 365 for IT Pros - Fifth Edition Files” | ? {$_.RMSTemplateName -eq “Viewer – View Only”} |
Format-Table FileName, RMSTemplateName

Applying Protection to Files

To apply an Azure Information Protection label to files, we need to know the GUID used for the AIP label. The easiest way
to get this is to capture the label identifier from a file that we know has the desired classification. You can also find the
GUID in the Label ID property of a label when viewed through the Azure portal.

[PS] C:\> $LabelId = (Get-AipFileStatus “C:\Temp\Important Staff.XLSX”).MainLabelID

We can now use the label identifier to apply the same classification to one or more files. In this case, we look for all Word
documents in a folder and its subfolders that do not yet have a label and apply the label to the set of files.

[PS] C:\> $TargetLocation = “c:\Temp\”

$TargetFiles = “*.docx”

$Files = (Get-ChildItem ($TargetLocation + $TargetFiles) -File -Recurse)

ForEach ($F in $Files) {

$FileName = $TargetLocation + $F.Name

$FileStatus = (Get-AipFileStatus -Path $FileName)

If ($FileStatus.IsLabeled -eq $False) {

Set-AIPFileLabel -Path $FileName -Label $LabelId }

Applying a label to a file updates the file and it can take several seconds to process a file. Be careful when you apply
labels to a folder holding many files.

Reporting Labelled Files

We can use the same technique to generate a report about the labelled and protected files in a folder, including folders
holding synchronized files from a SharePoint Online or OneDrive for Business library. This code looks for any Excel and
Word documents in a folder and then checks each file for the presence of an AIP label. If a label is found, we extract
details of the label and any associated rights management template.

[PS] C:\> $Report = @()

$Files = (Get-ChildItem "c:\temp\" -Include *.docx, *.xlsx -Recurse)

ForEach ($F in $Files) {

$FileName = "c:\temp\"+ $F.Name

$TemplateName = $Null

$Status = (Get-AipFileStatus -Path $FileName)

If ($Status.IsLabeled -ne $False) {

If ($Status.RmsTemplateId -ne $Null) {

$TemplateId = [GUID]($Status.RMSTemplateId)

$TemplateName = (Get-RMSTemplate -Identity $TemplateId.Guid -ErrorAction SilentlyContinue).Name

If ($TemplateName -eq $Null) {$TemplateName = $Status.MainLabelName }

$ReportLine = [PSCustomObject][Ordered]@{

File = $F.Name

IsLabeled = $Status.IsLabeled
 LabelId = $Status.MainLabelId

Label = $Status.MainLabelName

Date = $Status.LabelDate

RMSGuid = $Status.RMSTemplateId

RMSTemplate = $TemplateName

Owner = $Status.RMSOwner }

$Report += $ReportLine

$Report | Export-CSV -NoTypeInformation c:\Temp\LabeledFiles.csv$Report | Export-CSV -NoTypeInformation c:\Temp\LabeledFiles.csv

The output for an individual file that has a sensitivity label with protection is:

File : ABPs and Teams.docx

IsLabeled : True

LabelId : 81955691-b8e8-4a81-b7b4-ab32b130bff5

Label : Secret

Date : 13 Nov 2018 12:29:42

RMSGuid : c7fc2174-097c-4123-9cad-15f1a32cb145

RMSTemplate : Secret

Owner : Tony.Redmond@office365itpros.com

After all files are processed, the script writes the information collected into a CSV file that can be opened and analyzed
with Excel or Power BI.

Removing Labels
If you want to remove a label, run the Set-AipFileLabel cmdlet and specify the RemoveLabel parameter and give a reason
why the classification is no longer needed. For example:

[PS] C:\> Set-AipFileLabel "C:\Temp\Important Stuff.docx" -RemoveLabel -JustificationMessage "Label no longer necessary"

Protecting Files with RMS Templates

The Protect-RMSFile cmdlet protects a file by applying a template without applying a label. You can do this by creating a
new protected copy of the file that leaves the original intact, or you can rewrite the file to create a protected version.
Once again, we must retrieve the GUID for the template we want to use.

[PS] C:\> $TemplateId = (Get-RMSTemplate -Identity "Do Not Forward").TemplateGuid

We can now apply the template to a file or files. The InPlace switch shows that we want to apply protection to the file
without creating a copy. The OwnerEmail parameter sets the owner of the file.

[PS] C:\> Protect-RMSFile -File "C:\Temp\ImportantInfo.docx" -InPlace -TemplateId $TemplateId

-OwnerEmail Kim.Akers@Office365ITPros.com

Processing Protected Documents Found in Content Searches

As an example of how to use the PowerShell cmdlets, let’s assume that your organization must respond to a GDPR Data
Subject Request (DSR). Office 365 includes the ability to create and process DSRs in the Security and Compliance Center.
In this context (see Chapter 20), a DSR is a special form of Office 365 eDiscovery case that depends on one or more
content searches to find information relating to a named individual (the data subject). To satisfy a DSR, we need to find
all content relating to a data subject. Exchange indexes the content of protected email, so a content search finds those
messages without difficulty. The problem lies in protected documents stored in SharePoint Online and OneDrive for
Business sites because Office 365 does not index this content.

Searches against SharePoint and OneDrive sites still turn up results but only if the search keywords occur in metadata
like a document subject. Even so, before those documents can be turned over to the data subject, the content must be
checked to understand whether the documents relate to that person. In addition, there’s no point in giving someone a set
of protected documents that they cannot read. All of which means that we need some way to decrypt protected
documents.

A content search can export protected documents. When you export the results, you nominate a target folder to receive
copies of the found files. Under the target folder, a folder (named after the date and time of the search) holds the search
results, including the export summary, manifest, and a folder called SharePoint. Inside the SharePoint folder are folders
for each site where the search found something, and folders navigating to the point in the site where the search found the
content. The path to such a folder might be something like this:

C:\Temp\Search for documents_Export\06.05.2018-1106AM\SharePoint\GDPR Planning Mark II\gdprplanningmarkii\Shared Documents\General

The best approach is to gather all the protected documents found by a search in a single folder and process them there.
The basic approach is to examine each file in the target folder and remove the protection if it exists. In this example, after
connecting to the rights management service with a superuser account that also has the administrator role, we can form
a collection of the files in the target folder and then loop through the set to find any that are protected. We then use the
Set-AipFileLabel cmdlet to remove the protection.

[PS] C:\> $TargetFolder = "C:\Temp\Search for documents_Export\06.05.2018-1106AM\SharePoint\GDPR Planning Mark

II\gdprplanningmarkii\Shared Documents\General"

$Documents = Get-ChildItem -File $TargetFolder

ForEach ($D in $Documents) {

$ProtectStatus = Get-AipFileStatus -Path $D.FullName

If ($ProtectStatus.RMSTemplateId -ne $Null) {

Write-Host $D.Name "is protected with" $ProtectStatus.RMSTemplateName

$Message = $ProtectStatus.RMSTemplateName + " removed for GDPR DSR"

Set-AipFileLabel -Path $D.FullName -RemoveLabel -JustificationMessage $Message }

APC123.docx is protected with Intellectual Property

SPO Protected Content Test.docx is protected with Patent Submission

FileName

--------

C:\Temp\Search for documents_Export\06.05.2018-1106AM\SharePoint\GDPR Planning Mark II\gdprplanningmarkii\Shared Documents\General...

C:\Temp\Search for documents_Export\06.05.2018-1106AM\SharePoint\GDPR Planning Mark II\gdprplanningmarkii\Shared Documents\General...

When all the protected files are unprotected, investigators can review the document content to decide whether to release
files as part of the response to the DSR. The same problem of how to deal with protected documents exists for all Office
365 content searches.

Surprisingly, Azure Information Protection does not capture audit records in its logs when a user removes protection from
a file. Instead, it writes event 104 into the Windows event log on the workstation where the removal command was
processed.

Sensitivity Labels and PowerShell

The Get-Label cmdlet returns the set of sensitivity labels defined in a tenant or the properties of an individual label. If
encryption is enabled for a sensitivity label, its link to the underlying protection template is found in the LabelActions
data, where the TemplateId field in the settings is the GUID for the template. In this case, we use the Get-Label cmdlet to
examine the properties of the Black Matter sensitivity label.

$Label = Get-Label -Identity "Black Matter"

$Data = ConvertFrom-Json ($Label.labelactions)[1]

$Data.Settings

Key Value

--- -----

protectiontype Template

disabled false

templateid 3fd634ba-2063-47a4-87d6-bef80ba50bdd
templatearchived True

linkedtemplateid 3fd634ba-2063-47a4-87d6-bef80ba50bdd

contentexpiredondateindaysornever Never

offlineaccessdays -1

rightsdefinitions [{"Identity":"office365itpros.onmicrosoft.com","Rights":"VIEW,VIEWRIGHTSDATA,OBJMODEL"},{"Iden...

Examining the rightsdefinitions value more closely, we see the individual rights assignments:

[PS] C:\ $Data.Settings[8].Value

[{"Identity":"office365itpros.onmicrosoft.com","Rights":"VIEW,VIEWRIGHTSDATA,OBJMODEL"},
{"Identity":"ProjectCondor@office365itpros.com","Rights":"VIEW,VIEWRIGHTSDATA,OBJMODEL,DOCEDIT,EDIT,PRINT,EXTRACT,REPLY,REPLYALL,FORWARD"}]

The service domain for the tenant is office365itpros.onmicrosoft.com. We can see that the rights assigned to the domain
are view and view rights, which equate to the precanned “Viewer” permission. The other entry in the list is for
ProjectCondor@office365itpros,.com, the email address of a group. Anyone who is a member of the group gets a much
larger set of rights including View, View Rights, Edit, Print, Extract, Reply, ReplyAll, and Forward. This set of rights
equates to the Co-Author permission.

The Set-Label cmdlet updates label properties. For example, here’s how to update the display name:

[PS] C:\> Set-Label -Identity Ultra -DisplayName "Ultra Confidential"

However, given the complexity of labels, especially those used with encryption, it best to manage them through the
Security and Compliance Center whenever possible.

Logging Protected Documents

Taking the necessary steps to make protection available to users is one step. Knowing that people use the technology to
protect important content is another. Azure Information Protection logs most protection activities, such as someone
applying protection to a document or a recipient reading a protected message. Records exist for activities performed
using the Azure Information Client (for example, to protect an individual file through Windows Explorer).

However, Microsoft has not enabled the usage reports previously available in the older Azure portal in the new version. A
version of a central reporting and analytics service for Azure Information Protection logs is in preview to report on the
use of AIP labels (but not sensitivity labels). Version 1.41.51.0 or later of the AIP client supports central reporting. While
the central reporting and analytics service is the preferred option for largescale reporting, you can also create your own
ad-hoc reports by extracting information from the usage logs.

The Rights Management service creates a new log daily. To retrieve the logs, you connect to the service with the Connect-
AadrmService cmdlet and then run the Get-AadrmUserLog cmdlet to download the files to a nominated directory.

In this example, we connect to the service to download logs from October 1, 2017 to the current date to the C:\Temp
folder. Specifying the Force parameter allows the cmdlet to overwrite logs if they already exist in the target folder.

[PS] C:\> Connect-AadrmService

[PS] C:\> Get-AadrmUserLog -Force -Path C:\Temp -FromDate 1-Oct-2017

The output is a set of logs named rmslog-YYYY-MM-DD.log. For example, the log with events for October 1, 2017 is
rmslog-2017-10-01.log . The data in the log is in W3C extended log format and holds information about the set of rights
management events that occurred on a single day. The fields for each event include:

The date and time.

The user account that interacted with rights management, the operation requested by the user (for example, the
AcquireLicense operation is recorded when a user gets a license to read some protected content.
The result of the operation (Success or Failure).
The GUID for the template used in an operation.
Information about the requesting client.

You can open and examine the logs with any text editor or an application like Excel that can read W3C format files. For
further analysis and reporting on exactly how the tenant uses rights management, you can load the logs into tools such as
Microsoft Log Parser or any other application able to parse the data. See this article for some interesting tips.
Alternatively, you can gather all the logs together in a CSV file and use Excel to examine the contents. This PowerShell
code depends on the log files being in the c:\temp\ folder. The output is a CSV file called AllRMSlogs.csv.

[PS] C:\> $Logfiles = Get-ChildItem "C:\Temp\" -Filter rmslog*.log

[PS] C:\> $Logfiles | % { Get-Content $_.FullName | Select -Skip 2 | % {$_ -replace "#Fields: ",""} | ConvertFrom-Csv -Delimiter `t |
Export-Csv -nti "C:\Temp\AllRMSLogs.csv" -Append }

Other cmdlets that you can use to interrogate document tracking information are:
 Get-AadrrmTrackingLog : Returns tracking information about protected documents (not messages), including who
protected documents and who accessed those protected documents.

PS C:\> Get-AadrmTrackingLog -UserEmail Tony.Redmond@Office365itpros.com -FromTime "10-May-2018 00:01" -ToTime "11-May-20181" | Format-
Table RequestTime, RequesterDisplayName, Contentid

RequestTime RequesterDisplayName ContentId

----------- -------------------- ---------

10/05/2018 11:18:57 Tony Redmond f3cbc13c-4ff0-4837-b88c-8dc29782f9bf

10/05/2018 11:18:33 Tony Redmond 374cc8a7-cf37-45c8-bf97-7e9bf9a09345

Get-AadrmDocumentLog : Returns information about documents protected by a specific user.

PS C:\> Get-AadrmDocumentLog -FromTime "1-May-2018" -ToTime "12-May-2018" -UserEmail Tony.Redmond@office365itpros.com | Format-Table

Owner, ContentName, CreatedTime, Recipients, PolicyExpires

Owner ContentName CreatedTime Recipients

----- ----------- ----------- ----------

tony.redmond@ Ch 1 - Introduction to Office 365.docx 10/05/2018 20:56:00 {PrimaryEmail: flayosc1@outlook.com...

See the Azure Information Protection documentation for more information, including how to use the tracking site to
retrieve tracking information for protected documents.

Track that document ! Protected documents usually hold some form of confidential information and it is good to know
who has access to that information and to be able to revoke access if needed. To meet this need, users can track the
progress of protected documents by using the rights management sharing application or by signing into the document
tracking site . The ability to track documents is an Azure Information Protection Premium feature that needs a separate
license (tracking of email messages is not supported). If they have a license, users can track documents from the Track
Usage feature available in the Office applications or by signing into the RMS document tracking site. From the site,
users have access different views of the access to protected information that they have circulated. If they discover a
problem, like an unauthorized access to a document by an unexpected person, document owners can revoke access no
matter where files are currently stored. Administrators can also use the site to track documents on behalf of users. In
June 2017, Microsoft announced that they have a “Do Not Track” feature in preview to give organizations the ability to
exclude some users from document tracking because of privacy or other concerns.

The End or The Beginning

Books like this don’t happen without a great deal of support. Quadrotech, our sponsor, has some interesting products to
help companies migrate content to Office 365 and manage the content once it’s safely stored in the cloud. On to Chapter
25 to hear what Quadrotech has to say.
Chapter 25: Office 365 Migration for
Enterprises
Sponsored Chapter by Quadrotech Solutions

A Short History of Quadrotech

As emphasized numerous times throughout this book, Office 365 is an ever-changing suite of services. Trying to
embrace Office 365 without re-assessing our own processes and practices is almost certainly a project doomed to fail.
This is a concern not only for IT departments in organizations moving to or already using Office 365, but also for
Independent Software Vendors (ISVs) building tools to work alongside Microsoft’s cloud platform.

Quadrotech is an example of such an ISV. In this chapter, we will walk you through the journey we undertook to
ensure that our solutions are able to fully address the challenges set by Office 365. Microsoft’s success in moving
customer workload to the cloud has driven a transformation throughout the industry, forcing companies to update
their products, strategy, and vision to adapt to the cloud-first reality, or be left in the past.

Quadrotech was founded back in 2011, the same year Office 365 launched. Our first products focused on helping
companies to migrate to Office 365, enabling them to bring data from multiple sources to the cloud. To date,
Quadrotech is the only enterprise email migration vendor with an integrated suite of products covering email archive
migration (Archive Shuttle), mailbox migration (Mailbox Shuttle), PST migration (PST Flight Deck), and public folder
analysis and migration (Public Folder Shuttle). The solutions are also available as ready-to-go cloud variants, with no
need to deploy extra infrastructure to make your enterprise migration project successful.

Archive Shuttle
Before Office 365, it was common for organizations to enforce a strict storage quota on user mailboxes, often lower
than 1 GB. Third-party archiving solutions (such as Enterprise Vault) were used to store old email data in a separate
repository to save precious mailbox space. The same products were also used to store data for legal and compliance
purposes.

With the move to Office 365, archive products were no longer necessary as all the data could now be stored in-place
in user mailboxes or archives. The big advantage of keeping data in mailboxes is that everything is exposed for
compliance. However, none of the Microsoft tools supported migration from third-party archiving solutions to
Exchange Online, presenting a problem for organizations looking to move to Office 365.

Enter Archive Shuttle. A fast, intelligent, and highly automated solution, Archive Shuttle migrates mail and journal
archives from legacy on-premises and cloud archiving systems to Office 365, while preserving envelope data.
Workflow-driven processes automate the extraction and ingestion of archive data, eliminating risk with backups and
data integrity checks throughout the entire process.

The proprietary Advanced Ingestion Protocol (AIP) used in Archive Shuttle to achieve migration speeds dramatically
faster than traditional methods and deliver a lower ingestion error rate. The leavers feature automates the creation of
terminated user archives in Exchange Online, enabling data owned by people who leave a tenant to remain
discoverable for compliance purposes without requiring Office 365 licenses. Our cloud-hosted deployment
ArchiveShuttle.cloud minimizes on-site footprint and hardware requirements, and dramatically accelerates migration
projects.

Key features of Archive Shuttle include:

Highly automated and workflow-based process.

Sync ’n’ Switch Technology - synchronize content in the background, then switch users and migrate within
minutes.
Record-breaking performance thanks to Quadrotech’s proprietary Advanced Ingestion Protocol.
Preserves Chain of Custody.
Schedulable reporting that can be sent to project stakeholders.
Selective migrations available to “migrate what you need.”
Simple, automated process for managing “Leavers” or “Orphaned” archives.
Journal explosion – the ability to deliver an instance of a message from a journal archive to all recipients of that
message.

Archive Shuttle supports the Sources and Targets (any source can be written to any target).

Source Target


Veritas Enterprise Vault Veritas Enterprise Vault

EMC SourceOne/EmailXtender Microsoft Exchange on-premises

HP Autonomy/Zantaz EAS Exchange Online

Dell/Quest Archive Manager Proofpoint

PST files PST files

Sherpa Archive Attender

Table 25-1: Archive Shuttle sources and targets

Learn more about Archive Shuttle by heading to the product page or request a demo here .

Mailbox Shuttle
Mailbox migration, as a standalone project, is relatively straightforward, and there are a number of solutions,
including Microsoft’s built-in mailbox migration tools that are able to help you move this data effectively. The problem
is that most organizations do not have just mailboxes to migrate, they have a highly complex email ecosystem, which
might include archives, PSTs, or legacy public folders. Each of these technologies have deep integration and
interdependencies with the mailbox, which needs to be kept intact during the transition. In this context, Mailbox
Shuttle can preserve interdependencies, and orchestrate a connected workflow, migrating email quickly and safely to
Exchange 2010, 2013, 2016, and Office 365. The solution provides maximum efficiency with minimum intervention.

Mailbox Shuttle’s proprietary Sync ’n’ Switch technology uncouples data synchronization from user migration,
guaranteeing that users are not interrupted while their email is migrated. This enables seamless migration across
Active Directory (AD) forests, helps consolidate email across organizational boundaries, and enables email integration
following mergers and acquisitions.

Mailbox Shuttle provides full checks, monitoring and reports, and is readily scalable. Chain of Custody and
compliance best practice is respected at all times, and a full audit trail following Microsoft’s best practices is
maintained at every step of the migration process.

Mailbox Shuttle cloud gets you migrating in minutes. Its dedicated control module manages migration tasks without
needing to deploy on-premises hardware. Use the custom script engine to run tasks before the migration starts, after
the initial sync completes, or after the migration. From litigation holds, to licensing, and mobile device tasks - if you
can execute the command in PowerShell, then you can put it into the work flow!

Key features of Mailbox Shuttle include:

Ensures continuous access to email.

Highly automated and workflow-based process.
Full reporting options to track progress.
Integrates with other Quadrotech products seamlessly.
Custom PowerShell script engine.
Licensed per user.

Learn more about Mailbox Shuttle by heading to the product page or request a demo here .

Public Folder Shuttle

Exchange public folders have been around for over 20 years, and for much of that time they were the preferred
solution for document sharing and collaboration. Far from a perfect solution, public folders have numerous functional
and administrative limitations which make them difficult to manage and control. Over the years, Public folders have
grown to span extremely large hierarchies, with millions of folders, and huge volumes of data. Even though there are
better alternatives available now, the process of confronting, consolidating, and migrating data from Public folders
can be very challenging.
Public Folder Shuttle is a powerful, intelligent solution for public folder analysis and migration, able to move Public
folder data into the most appropriate target in Office 365 using a highly automated approach. Public Folder Shuttle
analyzes characteristics within data hierarchies and repositories and classifies their contents. Once the analysis is
complete, the solution recommends where the data should be placed, provisions the relevant targets and migrates the
selected information to the chosen destination. The solution can also find obsolete data and is able to remove or
archive it as required.

Key features of Public Folder Shuttle include:

Public Folder Shuttle is a cloud-based service, requiring no hardware or deployment of infrastructure.

Public Folder Shuttle compares multiple characteristics of public folders to determine the most suitable
recommended target for those folders.
Administrators can determine value criteria and thresholds according to their organizations’ retention needs and
policies.
Automatically creates the relevant Office 365 Groups, shared mailboxes, and modern public folders, and
populates them with corresponding data.
Automation increases speed by removing the need for manual reconciliation of folder structures and hierarchies.

Free Public Folder Scanner Tool: Scan and analyze your public folder hierarchy for free and receive a detailed
report to help you understand what’s in your folders, and the migrations options available for your legacy data.

Learn more about Public Folder Shuttle by heading to the product page or request a demo here .

PST Flight Deck: eradicating the PST

pests
For years organizations have paid the hefty price of dealing with PST file creation and usage in their environments. At
a basic level, the use of PST files dates to the late 1990s, when email usage outgrew the storage that organizations
were able to afford. What started as a great solution for users that needed more space has now grown into a
compliance and risk nightmare. Let’s review some common PST problems:

PST files are everywhere (network drives, removable media, local drives).
It’s often hard to determine who the real owner is.
Duplicate PST files and backup copies can often be found in the environment.
Users oppose any disruption to the data contained in them.
Many PST files belong to terminated employees.
Files are password protected.
Files are corrupt or partially corrupt.

The compliance implications of using PST files make things even worse. Regardless of where you do business,
regulation changes have made it difficult to be compliant while using PSTs. The introduction of the European Union’s
General Data Protection Regulation (GDPR) caused many organizations to review how they store and manage data
that might include personal information, but similar regulations also exist in the US and other countries. From a
compliance standpoint, PSTs are a nightmare. In several notable occasions, companies have had lost PST files or had
them stolen, resulting in fines and negative brand issues.

Other PST problems:

They provide no protection and rely on host storage (including local PC disks that are prone to failure).
Passwords do not encrypt the file and are easily broken.
An organization has no idea what valuable information might exist in PSTs.
Users are likely to avoid retention policies and other compliance controls by using PST files, which can then
expose the organization to regulatory intervention and possible fines.
The PST file structure is prone to corruption and data loss.

To make matters worse, PST files can contain any information circulated in email since your organization first started
using Exchange. This can include Identifying Information like National Insurance details, SSNs, and any other
personal or sensitive data that violates multiple modern compliance policies but may have been acceptable (or at least
possible) to share many years ago. The files could also contain current company information that should not be public,
including intellectual property and insider information. In terms of GDPR, these emails could contain information that
would be subject to Subject Access Requests and “Right To Be Forgotten” articles of the regulations. It would be
nearly impossible to satisfy GDPR requests if PST files are in active use.

Compliance
So how does PST Flight Deck help? The first major part of any PST discovery, consolidation, and migration product is
the need to identify PST files and their owners. PST FlightDeck scans target locations for PST files and records what it
finds. Once all PST files are discovered, using this ownership information, PST Flight Deck ensures that the right file
goes to the right place. To do so, an eight-factor algorithm checks for:

NT File Owner.
 Most Common Sender.
Most Common Receiver.
Is the location in a user’s home drive?
Does the folder patch contain the users alias?
Was the file discovered by local computer scan by a user?
Does any user have it connected in Microsoft Outlook?
Custom path regex options.

User experience
Another major concern in a PST eradication project is the user experience. In some cases, data stored in PSTs is vital
to an employee’s ability to do their job. Organizations that have communicated the process as an upgrade to their PST
usage are often the most successful in convincing users to stop using PSTs. Why an upgrade? Because after the
migration, users will get many features “back”: they will be able to see all their email on their mobile devices, they do
not need to backup PSTs anymore, don’t need to split PSTs any more when files get too big, and so on. From an IT
perspective, all email is now in a central store and subject to Office 365 compliance policies, we reduce support costs,
reduce risk for data loss and corruption, simplify e-discovery, and more.

PST Flight Deck can provide this upgrade experience. Our method controls the user experience by providing the
following features:

User keeps their PST open and in read-only while it is being migrated.
Options to run silent without user interaction, or in an interactive mode allowing the user to select the PSTs they
need.
No need to provide passwords for their PST files (including the passwords the user has forgotten).
Remove data file entries from Outlook after the migration – no more error messages!
Automated deletion of migrated files.
User engagement screens can be changed to the user’s local language.
User Communication Process to keep everyone engaged and informed.
Processing is done on the server, not the workstation, to avoid impacting performance.

Organizations that have piloted other methods report heavy increases in support calls to help desks (as high as 80%!)
and unhappy users. PST Flight Deck enables a smooth transition that maintains PST data visibility for the end user,
including them in the process, where required.

Automation
The final topic we want to cover is the massive administrative headache most organizations face when undertaking a
PST eradication project. After finding PSTs and linking them to users, we move on to the process of collecting and
uploading PSTs to move them to Office 365.Two major problems exist: disrupted uploads due to network interruptions
and bandwidth consumption.

PST Flight Deck handles the first problem by uploading the files via an http or https stream, with a robust auto
resume process. The bandwidth issue is handled in two ways. PST Flight Deck has the option to control bandwidth
consumption on the user or site level. Our servers can run in a hub and spoke configuration, with the “spoke” placed
in Azure. The organization can then configure “split tunneling” to allow users to upload outside of the company
WAN/VPN infrastructure. This offloads the data volume from business-critical connections, and in many cases allows
the upload to run faster. Combining both options can drastically mitigate this problem and mean that PST data is
assembled in a central location faster

Once the files are gathered in a central location, the Import phase begins. The biggest issue we then face is to
automate the handling of the common problems encountered with PSTs. Our processing server automates many
issues that administrators otherwise must manually perform:

Password removal.
Final backup of files.
Item level deduplication.
Filtering out redundant data.
Fixing corruption (using industry best practice).
Keeping user’s folder structure.
Outlook profile clean up.

Learn more about PST Flight Deck by heading to the product page or request more information here .

Mastering Cloud Migration

Over several years, Quadrotech dedicated enormous effort to make sure that our products work together with end-to-
end workflow and integrated orchestration. The reason behind this is simple: organizations often use more than one
on-premises email system (archives, PSTs, public folders) and need to extract information quickly and easily from all
their systems to consolidate email in Office 365. If more than one email system is used, an organization might need to
deploy more than one tool and divide the work over multiple migration projects. Quadrotech’s orchestration helps
automate the interactions necessary to process data across a range of systems.
To understand the vital importance of automation for projects of enterprise scale, consider a breakdown of tasks for
migration of PST files for 10 users. Next, imagine the scenario when scaled up to 10,000 users, with no support for
automation:

All PST files need to be discovered and analyzed.

PST usage needs to be disabled for each user.
The user’s Exchange Online mailbox must be archive-enabled.
Mailbox archiving needs to be disabled in the legacy archive.
The email archive needs to be migrated from the legacy archive to the user’s Online archive.
Shortcuts need to be removed from Exchange Online mailboxes.
Shortcuts in any PST files referenced in user profiles need to be eradicated.
The user needs to be notified that a PST collection is occurring.
Ownerless or shared PSTs need to be resolved to the correct target.
The content of PST files needs to be migrated to the correct Exchange Online archive mailbox.
The live mailbox needs to be migrated from the legacy platform and moved to Exchange Online.

Even if the tools you use incorporate automation, each step typically must be initiated using a different interface or
tool – whether that be the Office 365 Administration Center, PowerShell scripts, or third-party migration tools.
Orchestration is about combining individual areas of the migration project into one over-arching program so that the
entire workflow is integrated and automated from end to end. The orchestration functionality in Quadrotech’s
MailboxShuttle.cloud is capable of driving the entire workflow. The built-in PowerShell engine means that workflows
can include custom scripts, which can call APIs, kick-off commands, and perform other actions related to each user or
group.

Figure 25-1 shows the first steps in a sample workflow, beginning by assigning the initial Office 365 licenses, then
creating the Office 365 archive mailbox and executing a manual script to notify the user’s manager that the mailbox
has started being processed. Finally, this workflow ends with the mapping and synchronizing of the user’s archive
data. Figure 25-2 gives a glimpse at the interface used to assign PowerShell scripts that can perform extra tasks as
part of each step of the workflow.

Figure 25-1: Drag-and-drop Workflow in MailboxShuttle.cloud

Centralized migrations involving mailboxes or public folders can usually be completed without user involvement. On
the other hand, some migration projects will never be successful unless the end users are involved. Examples include
tasks that need data classification, identification of owners, and mapping relevance of file contents - like PST file
eradication projects. Your end users know what data is most important to them, and where that data is kept.
Communicating this kind of information to the migration team can have a significant impact on the overall success of
the project.
Figure 25-2: Setting up Custom PowerShell Scripts in MailboxShuttle.cloud

While too much end user intervention can significantly delay a project, the right amount of input can deliver vital
support when scaling enterprise migration activities - if you manage end user involvement carefully. It’s crucial to get
the balance right - to know when you can rely on analytics and technology to give the answers, and when you should
involve your end users to understand the data that you collect. Figure 25-3 shows how this might be done. The
software is asking the user to review a set of discovered PST files before migration starts. Each file can then be
inspected by the user, and they can decide to keep or discard the file (assuming the administrator sets the relevant
permissions).

Figure 25-3: Migrating PST files

For large PST migration projects, it is extremely important to keep the end user informed throughout the migration
process. They should be notified about what’s about to happen, updated throughout the process, and offered training
or resources where required. Regular updates can be communicated via email, outlining any activity that might affect
users at each stage. PST Flight Deck incorporates the ability for organizations to communicate with users at every
step of the process, with detailed information, and topic guides to support a smooth, predictable transition.

Beyond Migration: Analysis, Reporting,

and Insights
Migration is about more than just moving data. It is an ideal opportunity to explore the data stored in different
repositories, to separate the essential, business critical information from old, legacy data that no longer needs to be
kept, and does not have to be migrated. This kind of transformation cannot be done manually. You need to be able to
rely on analytics to explore and understand data, and then intelligently decide the most appropriate migration target
within Office 365. To address such needs, a strategic decision was made that going forward our products should
incorporate proprietary technology that is used to understand the relations, groupings, and types of data within
repositories.

For example, it might seem natural to move data from legacy public folders to modern public folders in Exchange
Online, but there are other options available in Office 365. On closer analysis, you may see that some public folders
have characteristics that would actually benefit from being placed into an Office 365 Group. You can take a similar
approach to other data types too, such as PST files, journal data and even user mailboxes. Ask yourself - do you need
to move everything?
Of course, migration is just one of the steps an organization can take on their cloud journey, and certainly not the last
one. It is important to get all your relevant data to the cloud in a fast, automated manner that appears seamless to
your end users, but it’s equally as important to be able to analyze and report on the way users work with that data.
Not only can this help you get the full value of Office 365 and optimize your investment, but it can also protect you
from embarrassing data leakage situations and ensure you stay compliant with regulations such as GDPR.

To deliver these capabilities to new and existing customers, and to strengthen the analytics features of our migration
products, in 2016 Quadrotech acquired Cogmotive, a market leading provider of Office 365 reporting and analytics.

Radar Reporting
A dizzying array of services and workloads exists within Office 365, and as an IT administrator you are expected to
understand, manage, and maintain control of an ever-changing set of services. This is true throughout Office 365 but
is particularly apparent when attempting to report user and system activities. Your activity, growth and operational
metrics now sit within Office 365, and the tools you may have used on-premises are no longer an option. How do you
extract this information, and get the visibility you need to review activity and operations, and gain value from your
new IT environment?

Office 365 includes various forms of built-in reporting. However, the standard reports have functional limitations that
prohibit large or complex organizations from gaining the specific insight they need. More importantly, there is no way
to get a comprehensive view of your entire environment in a single interface. You will likely need to stitch together
reports exposed in the Office 365 Admin Center with the ones available in the Security and Compliance Center or via
the Power BI content pack. Each of these options only cover a small subset of areas and services. This means that to
get a full view that spans your tenant, reports need to be consolidated manually, and you will need to visit each place
at regular intervals to gather the data you need.

You can also use PowerShell for reporting, which is effective but time-consuming method. Not only is it resource
intensive, PowerShell can be temperamental, requiring precision and a specific skill set to run and manage reporting
from the command line.

Radar Reporting (formerly Cogmotive Reports) provides Office 365 reporting, auditing and analytics as a software-as-
a-service (SaaS) web application. The tool provides a clear, unified view of your entire Office 365 environment. In
addition, the application expands capabilities around accessibility and customization of the data. The export, send,
and schedule functionality enables you to share reporting information securely across the organization. You can
create as many user and admin accounts as you need and configure access on a fine-grained level, for example you
can make sure that a user from the Finance department is only able to see the ‘Licence reports’. Radar Reporting has
much longer data retention than built-in options; data is stored for as long as you continue to use the application,
enabling long term, historical trend analysis.

How does it work?

The application uses a read-only Service Account to connect to Office 365 and collects data on usage and activity.
Data collection for Radar Reporting happens once every 24 hours.

Radar Reporting provides more than 100 pre-configured reports plus a custom report builder (you can report on any
Azure AD attribute) and a custom pivot tool. New reports and features are added every month, and you get instant
access to all new features as soon as they become available. Figure 25-4 shows a sample report of Teams activity
within an Office 365 tenant. Find out more about Radar Reporting here . You can take a free 14-day trial or explore
the full-feature demo environment here .
Figure 25-4: The Teams Daily Activity report in Radar Reporting

Radar Security & Audit

Office 365 is flexible, scalable, and mobile, with clear productivity benefits that have brought millions of users on to
the service. However, it is these same advantages that pose the biggest challenges when it comes to identifying and
‘locking down’ the service from potential security threats or attacks.

In order to protect your environment, you need to know exactly what’s happening in it. However, simply keeping an
audit trail isn’t enough, you also need to be able to identify, interpret and conduct focused searches on these events,
often across an extended timeframe whether it’s weeks, months, or even years.

Radar Security & Audit ensures that all Office 365 activity data is tracked and stored in a searchable, highly
customizable interface. You can use advanced filters to include or exclude elements from the results: user, workload,
event type, location, the list goes on. The Audit timeline is the hub of this module, it gives you instant access to all the
events in a scrollable, timeline style view that can be quickly searched. Figure 25-5 shows the timeline view of audit
events flowing into Radar for an Office 365 tenant.

Key features include:

Audit and event data across 10 core Office 365 workloads.

Track the activities an employee is performing on Office 365.
Identify the source of data leaks and PII information breaches.
Configure on-event alerting for sensitive documents, services or areas of your environment.
Reduce the risk of insider threats by monitoring admin activities, and quickly identifying ‘elevation of privilege’
style attacks.
Audit third party mailbox access:

Administrators accessing user mailboxes.

Delegates accessing other mailboxes.
Users performing actions on their own mailboxes.

Keep track of actions on VIP or sensitive mailboxes.

Figure 25-5: Timeline view for the Office 365 audit log in Radar Security and Audit

Radar Security & Audit integrates tightly with Radar Reporting, meaning that all the information stored within your
reporting database is also at your fingertips. This allows the administrator to quickly drill down into all aspect of the
user. For example, if you see suspicious user activity, you could enhance your investigation by reviewing their mail
traffic trends or mobile device configuration to cross reference this with said activity, such as moving between
different locations and devices in quick succession. This integration enables IT security teams to have the context they
need to be informed and on top of all activity in a single cohesive interface.

Radar Security & Audit has a lot of depth, with features and reports for various scenarios or security contexts, but
here are some of the key use cases beyond traditional security measures.

Compliance and Security Officers gain back crucial visibility and insight into actions, potential threats or
breaches of policy, allowing risk mitigation actions to be taken prior to an event.
It is possible to interpret vast amounts of data from an entire Office 365 estate, allowing planning and policy to
be implemented sooner as part of your overall information security & compliance strategy.
Use Radar Security & Audit to demonstrate commitment to best practice by easily showing accreditors, auditors
and your customers that you have gone above and beyond standard security benchmarks to champion best
practice within your industry.

GDPR compliance in Office 365

Your Office 365 environment is teeming with data, stored in the various services used across your organization.
Whether you’re in the EU, or you deal with personal data that resides there, it is critical that your tenant complies
with the GDPR requirements to protect personal data, assess the protections in place, and control access to personal
data. Radar Security & Audit helps your organization achieve GDPR compliance in the following ways:

Data retention : The application stores audit data for a year minimum, with the option to extend this. If you
need to investigate a historic incident, you know that you have a full forensic trail to follow, that can be supplied
to any regulatory bodies, should the situation arise.
Data breach notification : One of the most worrisome aspects of GDPR is the strict deadline on breach
notification. Radar for Security & Audit’s advanced filtering, alerting and timeline features help you quickly
identify who had access to which files and what they did with them, greatly easing the process of identifying the
scope and impact of data breaches.

Find out more about Radar Security & Audit here, or see it in action using our full-feature demo . If you would like to
see what’s happening in your environment, why not try Radar Security & Audit free for 14 days.

The future of Quadrotech

Combining the talent and technologies behind Quadrotech and Cogmotive was the first step in our long-term vision.
Our new and existing products will continue to focus on the Microsoft cloud platform, in five main areas as depicted
on Figure 25-6 : Migration , Reporting , Security , Management and Self-Service . The integration between those
products will become tighter and more intuitive in time, allowing for more actions, better workflows and automation,
valuable insights into data, and more.
Figure 25-6: Our vision

We will continue developing our products with a focus on massively-scalable architecture, able to handle even the
biggest volumes of data, by utilizing the SaaS framework and provisioning engines of AWS and Azure. Unlike our
competitors, we use our own proprietary technology, which allows us to react quickly to market changes, and respond
fast to feedback and feature requests from customers, partners, and end users alike. Our in-house support
organization closes the loop and is able to provide best-in-class support experience by working directly with the
product owners and engineering teams.

Based on all these factors, we strongly believe that we will be able to:

• Significantly reduce costs for Office 365 management by enabling

transparent IT operations

• Bring business process closer to the End User

• Automate day-to-day business needs of our customers by

capturing Business processes and goals as Workflows and Actions

• Provide insights into data with automated actions via machine

learning

Our purpose makes the cloud a better, smarter and more intelligent place by offering Security, Management,
Reporting and Migration for Office 365 in one place, from one provider.

With all this in mind, Quadrotech is proud to announce our two newest products, Cloud Commander and Autopilot.

Cloud Commander: The Need for

Speed
Tenant to tenant migrations can be quite a challenge. Although we see some options popping up inside the set of
native Microsoft tools, these only cover some very basic scenarios are not applicable to large or complex migrations.
Third party migration tools, like Quadrotech Cloud Commander, fill this gap enabling organizations to move more data
in a faster and more controlled manner. Cloud Commander enables collaboration between Office 365 tenants and
delivers a systematic, structured approach for migrating data between them. Cloud Commander focuses on two Office
365 workloads: Exchange Online and OneDrive for Business. Support for SharePoint Online is coming soon.

Exchange migrations are a core part of Quadrotech’s technology and the company history. With our proprietary AIP
protocol, detailed in the Archive Shuttle section above, we can move data much quicker than any other solution on the
market. The Outlook profile update agent that can be installed on user workstations to reconfigure profiles
automatically, removing the need of manual steps and reducing service desk tickets.

Cloud Commander is a Software-as-a-service offering, deployed inside your own Azure environment. Your data doesn’t
leave the confines of your environment, ensuring privacy and control. It also allows for the update of both source and
target on-premises Active Directories objects, which facilitates the migration and coexistence for those in a hybrid
deployment. The same feature set is available for those who have a cloud-only deployment.

Achieving the best migration speed

To customers or competitors, speed is always important, no matter the migration type. Different companies have been
claiming for years they offer the fastest products, migration speed wise. In many cases however, when migrating from
on-premises systems, these advantages were minimal and oftentimes only noticeable in certain optimal scenarios.

On-premises systems do not typically allow for large data limits. For years, IT departments had to limit resource
usage in order to provide the organization with a stable service, at a reasonable operational cost. A common strategy
was to purchase an archive system and other tiering options to try to get users the capacity they needed in a cost-
effective manner. Thus, when trying to migrate into Office 365, or back to on-premises versions of Exchange, speed is
heavily influenced by the source system, regardless of the migration method being used. Factors that affect the speed
include slow drives, overwhelmed SQL server, unoptimized network, and so on.

For “traditional” migrations, it’s not uncommon to purchase expensive hardware for temporary projects. Things are
different with Office 365. Because Microsoft runs the service and has complete control over it, there is far better
standardization on the hardware layer, and thus performance is more predictable. This in turn allows Microsoft to
offer mailbox quotas that were unheard of in the on-premises days. In on-premises environments, many organizations
did, and still do, provide mailbox quotas of less than 2GB. In contrast, Exchange Online has SKUs ranging up to
“unlimited” mailbox storage.

Standardization also means that vendors offering migration between Office 365 tenants have to follow the same rules
and use the same configurations provided by Microsoft. Thus, the main factor affecting migration speed is the method
used, and not hardware. Instead of relying on physical or virtual machines, we chose to use Azure components that
are billed on consumption. This means that you or Quadrotech can host Cloud Commander inside an Azure
environment in a cost-effective manner.

What do we mean by Azure components? From the ground up, Cloud Commander was built with speed and efficiency
in mind, using the Azure Service Fabric, Web Jobs and Single Databases, all of which can be powered up and powered
down as needed for the migration. Using this technology, we have been able to drastically reduce the costs involved in
migrating user’s data. Cloud Commander gives you extreme speed, without an extreme price.

Speed specifics
So, when we talk speed, how fast are we talking? A project we recently completed saw sustained speeds of well over
50 GB an hour – with peak performance over 62 GB an hour. At 50 GB an hour we were moving just under 1.2 TB of
email data a day. According to Microsoft performance guidelines, typically, third-party providers of Exchange web
migration services can be expected to transfer 5-10 GB of data per hour . So, by industry standards, Cloud
Commander is incredibly fast.

Let’s explore an example. If you needed to move 3000 users with an average mailbox size of 5 GB, that would total
just under 15 TB of data. At 5 GB an hour, without overhead for startup and power down, you would have 3,000 hours.
With Cloud Commander it would take 300 hours in the same example. This reduces the duration for the migration part
of the project from 125 days to 12.5 days! One important note to call out here is that these speeds are for migrations
in the same Office 365 datacenter region. While the vast majority of projects we have seen are moving inside the same
region, generally speaking, and depending on location, the rate of transfer might be lower across regions.

Reducing the overall migration duration

Project Management
For experienced Migration Engineers and Project Managers migration speed is less crucial, while overall project
duration and efficiency is vital. Regardless of the migration method, the project planning aspect can be time
consuming and difficult. Oftentimes in a tenant to tenant migration the business driver is a merger, acquisition, or
divestiture activity. Such scenarios create unique challenges when coordinating the different aspects of the project.
The business is changing, the IT personnel are changing, IT policies are merging, and often IT staff are trying to cope
with a heavy workload. Data reports are often lowest on the list, and yet these are a critical function for planning a
migration.

Quadrotech’s Office 365 Radar Reporting , and its integration into Cloud Commander can help ease the burden of pre-
migration reporting. Once Radar Reporting is setup, you can use all the fields and usage information to drive your
schedule. This gives full insight into what Office 365 data exists, how it is being used, and how much there is. When
ready, you can group your users using this data and send this batch information to Cloud Commander. Not only does
this make the schedule easy, it allows you to plan effectively, using the actual data. This prevents missed users and
accounts, giving you an accurate plan and minimal cleanup once the project is complete.

Automation and workflows

Another key theme of migrations is that there is often far more to do than simply moving the data. For years,
Quadrotech has had a really robust custom script engine in our email migration solution Mailbox Shuttle. This engine
allowed us to build workflows and execute custom PowerShell scripts (set legal hold, apply a license, enable/disable
Skype, and so on). Mailbox Shuttle’s functionality has now been integrated with other products like Archive Shuttle
and PST Flight Deck, to give a smoother, connected experience. This capability has been drastically enhanced with
our integration into Microsoft Logic Apps. With tons of APIs and our own PowerShell engine, it greatly extends the
ability for us to build work flows beyond the migration. This allows organizations to focus on their project, without
needing to execute tons of scripts or deal with segmented systems while users migrated.

Compliance
Finally, in Cloud to Cloud projects, we want to ensure we get all the data across. Office 365 security and compliance
features are only effective when all the data that requires these protections is inside the platform. One major
challenge is getting all user’s data into their OneDrive. Most organizations have typically moved user’s home
directories to OneDrive, but there is no guarantee that all user’s data is moved there. From a migration stand point,
often the workstations are also replaced during the project.

Enter the OneDrive Collector feature in Cloud Commander, with release planned for July 2018. The collection client
will inventory all of the files on a workstation and using file extension rules it will allow an organization to move the
files into the Users OneDrive cache for synchronization. Our agent will then control OneDrive for Business settings to
upload the synchronization agent and other features (the limits are controlled by IP address range in the same
manner we control utilization in PST Flight Deck )

The file extension lists can be customized from you. We have three categories of extensions:

Ignore . This is designed for System files or files with extensions like .dll and .sys which cannot, and should not,
be moved from their current location.
Inventory . Some files you want to be aware of, but do not want to migrate them. Access Databases are a good
example of a file type that should not move.
Migrate . These files will be reported and marked for migration when enabled. All traditional files like Office
Documents, PDFs, and more are good examples for this category.

File extensions that are not known, for example for home grown applications, or applications with small install bases,
will be flagged for your organization to classify into one of the three categories. When done correctly, this remediation
tool can be used to bring compliance to your environment or prepare for a proper OneDrive move without having to
collect files from an end user workstation!

Learn more about Cloud Commander by heading to the product page or request a demo here:
https://www.quadrotech-it.com/request-a-demo/

Putting Office 365 Administration on

Autopilot
Autopilot is a management tool, developed by Quadrotech, that allows companies and service providers to set up
granular delegated administration of both local and cloud-based Microsoft solutions. Autopilot provides a SaaS
platform hosted in Azure and built on Service Fabric that will proxy administrative tasks via Graph API or PowerShell.
Delegated administrators are granted permissions solely in Autopilot and have no native rights in the target solution.

The Problem with the Native Approach

As organizations adopt the Microsoft cloud and start moving users and their data into the solution, they often find that
there are more complexities than initially expected. It’s not a wholesale “lift and shift” operation. Individual workloads
may move, stay on-premises, or transition into some sort of hybrid. Depending on the workload, this can create
various challenges.

For example, an organization might have a local Active Directory environment running alongside an Azure Active
Directory environment. To avoid duplication of management tasks, the two environments are usually linked by the so-
called directory synchronization. In such case, most of the admin tasks need to be performed in the local Active
Directory for every synced user. However, the organization may also have cloud-only accounts that need to be
managed in the Office 365 admin console. Autopilot helps organizations streamline administration. IT Admins no
longer need to decide where to update an object – it all happens via Autopilot.

A different class of problems exists for Service Providers, who are often required to manage multiple tenants
simultaneously as part of their offering. This brings a unique set of challenges. In order to perform basic
administration tasks across tenants, these providers need numerous credentials spread between browsers or
PowerShell sessions. Autopilot brings multi-tenant management into a single console, enabling efficient management,
with more flexibility and less risk.

Virtual OUs
One of the challenges of managing Office 365 users is the inability to granularly compartmentalize users and groups
for administrative purposes. Many organizations have managed their users in Active Directory for nearly two decades.
The transition from a familiar, hierarchical structure (Active Directory) into something that resembles more of a flat
list (Azure AD) can pose a challenge. With this challenge in mind, Autopilot brings the concept of the Virtual
Organizational Units (OUs) to Office 365. Within the software, service providers and companies alike can now
logically group Office 365 users and/or groups. If they still have on-premises Active Directory, Autopilot’s on-premises
agent can replicate the current OU structure into Autopilot. This provides familiarity when it comes to administrative
delegation, while maintaining the knowledge of where the object exists, and where it needs to be updated.

Authorization Policies
Every user with Admin privileges in Office 365 increases your overall risk. It’s good practice to reduce the number of
users with such rights by removing anyone who does not need them. While this may sound straightforward, many
organizations assign these rights to help split Office 365 administration responsibilities across different teams, or
even vendors. For example, in a growing Engineering department that adds new employees every week or so, IT can
decide to assign Admins rights to the Development Manager. This means users and licenses are managed from
directly within the team, and team members can get up and running as soon as possible. It’s a logical move, but as
administrative roles in Office 365 are not limited in scope, if such decision is applied for a number of departments, the
overall risk level for the tenant increases.

With Autopilot, you can retain this quick, agile administration, but through secure means. The IT team can create an
Authorization Policy to grant the delegated admin (Development Manager) the exact rights they need and remove the
need for native Admin privileges in Office 365. Once virtual OUs are defined, Autopilot uses these for Role Based
Access Control across both on-premises and cloud workloads. Like the Exchange Role assignment policy, an
Authorization Policy has four major components, as shown on Figure 25-7 below:

Figure 25-7: Authorization policies in Autopilot

In turn, policy creation consists of the following steps:

1. Decide to which OU the policy will apply. After naming the Authorization policy, we need to define its scope,
namely to which objects it will apply. This is done by selecting the virtual OU (Figure 25-8 ).

Figure 25-8: Selecting a virtual organization unit

2. Decide whom are we delegating rights to. Next, we designate the role assignees, the users who will be
granted permissions to perform actions on all the objects in the OUs we selected in the previous step. You can
select a single user or multiple users, as well as groups (Figure 25-9 ).
Figure 25-9: Selecting users to process

3. Decide what actions do we want them to perform. The next step determines the permissions that will be
granted on the assignees. First, we select the Actions (Figure 25-10 ).

Figure 25-10: Assigning actions

4. Based on the selected actions, what specific rights should they have? Lastly, we can drill down into
granular properties for each action. For example, should the assignee have permission to

“read” or “edit” the various properties for each user? The relevant controls are shown in Figure 25-11 .
Figure 25-11: Assigning permissions

Configuration Policies
Configuration (or Baseline) Polices create standardization within a Virtual OU. For example, you can create a policy
that directs everyone in Sales to have automatic access to the specific set of tools they use for their roles, as well as a
specific directory of information. So, when a user is added to the Sales Virtual OU, the following actions happen.

The user…

receives access to the “Sales” shared mailbox.

is added to the Sales Team with Member rights.
is granted a Microsoft Dynamics license.
is given rights to the Sales SharePoint document library.
has their user object added to the Sales distribution group.
has their department attribute set to “Sales.”

Use Autopilot to create a single policy that will apply to not only current members of the virtual Sales OU, but any new
users that get added to the container. This can greatly increase efficiency for new hires, or when someone changes
roles.

How Does Autopilot Work?

Autopilot relies on embedded “Actions” that define what can be done. These actions are either Graph API based or
PowerShell based. For example, if a delegated administrator chooses to update Exchange Online “Out of Office”
configuration for a group of users, the agent controller sends that request to the Graph API agent, which makes the
change and reports the status of that change to the core application. In instances where we cannot use Graph, we rely
on PowerShell. This means that Autopilot has the ability to perform any administrative action as long as there is a
corresponding Graph API endpoint or a PowerShell cmdlet… which is a long list.

What Workloads are Currently Supported?

Currently, Autopilot supports management of on-premises Active Directory, Azure Active Directory, Exchange Online,
Office 365 Licensing, OneDrive for Business, and Microsoft Teams. As stated above, we are able to add “Actions” for
any workload, either on-premises or in the cloud, as long as it is PowerShell based or can be managed via Graph API.
In the short term, we are developing Advanced License Management, on-premises Exchange, and third-party
integration. But, as the product grows and matures, additional workloads and Actions will be added. Using continuous
integration, we are able to add features based on feedback and requests from our customers and partners.
Learn more about Autopilot by heading to the product page or request a demo here .

About Quadrotech
Quadrotech enables organizations to make their Microsoft Cloud smarter, secure, and efficient, ensuring businesses
get the most out of their IT investment. By providing a suite of migration products, we help enterprises move data into
and out of Office 365, or from cloud to cloud. Once the migration is complete, our advanced Office 365 reporting and
security applications show how employees are using and configuring the service, and IT admins, or security and
compliance teams can protect and audit their environment using detailed, filterable activity logs. Finally, we also offer
a sophisticated management solution to streamline Office 365 administration, and make management simple, the tool
can be used to delegate control of Office 365 to other users in the company with enhanced security and flexibility.
Achieve smarter migration, reporting, security, and management with Quadrotech.

Table of Contents
Foreword

Introduction

Organization of this book

Companion Book

Products

Highlighting the important stuff

Book Updates

The Author Team

Tony Redmond

Juan Carlos González

Ståle Hansen

Paul Robichaux

Jussi Roine

Gustavo Vélez

Vasil Michev (Technical Editor)

Previous Contributors

Microsoft MVP Program

Thanks

Sponsor content

Comments and feedback

Chapter 1: An Overview of Office 365

Office 365 Infrastructure

Datacenters and Regions

Sovereign Clouds

Multi-Geo Office 365

Tenant Domains

Directories and Identities

Licenses

Office 365 Trial Tenants

Automation

Networks
 Monitoring

A Constant State of Change

Command and Control

Office 365 Plans

Service Families

The Functionality of Office 365 Enterprise Plans

Adding Cost to Office 365

Plan Add-ons

Enterprise Mobility & Security and Azure Active Directory

Premium

Microsoft 365

Buying Office 365 Through Partners

The Commercial Success of Office 365

Monthly Active Users

Competition for Office 365

Leveraging the Breadth of Office 365

Will we move?

Chapter 2: Making the Decision to Embrace the Cloud

Removing Cloud FUD

Dependability

Maturity of Cloud Platforms

Security of Cloud Platforms

Industry Standards

Quality of Cloud Services

Office 365 Support

The Question of Losing Control

Understanding Service Level Agreements for Cloud Services

Office 365 Performance Against SLA

Making the Decision to Embrace the Cloud

The Easy Decision

The Harder Decision

The Economic Imperative

Figuring Out Cloud Costs

Creating a Network to Support Cloud Services

Executive Buy-in and Communications

Dipping Your Feet into The Water

Office 365 Partners

The Value of Hybrid Connectivity

The Need for Plan B

Cancelling a Tenant Subscription

Identities
Chapter 3: Identities and Authentication

Azure Active Directory

Azure AD SKUs and Office 365

Workload-Specific Directories

Identity Models for Office 365

Standalone identities

Synchronized identities

Federated identities

Authentication

Basic Authentication

Modern Authentication

Multi-Factor Authentication

Pass-Through Authentication (PTA)

Seamless SSO (Single-Sign On)

Azure Conditional Access and Office 365

Restricting Access to a Single Tenant

Synchronizing with Azure Active Directory

Microsoft Directory Synchronization Tools

Directory Synchronization Technical Concepts

Supported synchronization topologies

Object Matching for Existing Objects

Preparing for Directory Synchronization

Installing and managing Azure AD Connect

Staging Mode

Synchronization Rules

Object filtering

Password hash synchronization

Password writeback and self-service password reset

Optional AAD Connect features

Writeback permissions

Multi-forest directory synchronization

Managing and monitoring directory synchronization

Active Directory Federation Services and Office 365

Configuring AD FS

The AD FS configuration database

AD FS proxy servers

Changing the token lifetime

Restricting access to Office 365 through AD FS

AD FS and modern authentication

AD FS auditing
 Troubleshooting AD FS authentication

Password hash synchronization as a backup

AD FS and password expiry notifications in Office 365

Enabling password updates through AD FS

Enabling "Persistent SSO"

Connecting to LinkedIn

Managing a Tenant

Chapter 4: Managing Office 365

Cloud versus on-premises management

Office 365 Administrative Interfaces

Managing Skype for Business Online and Microsoft Teams

Managing the Security & Compliance Center

Managing Other Office 365 Services

Managing Other Microsoft Cloud Services

Using PowerShell to Manage Azure Active Directory

PowerShell Gallery

Connecting to Office 365 Endpoints

Figuring Out What PowerShell Cmdlets to Use

Office 365 administration apps

Administrative roles

Office 365 administrative roles

Exchange Online administrative roles

Security & Compliance Center administrative roles

Teams administrative roles

Privileged accounts

Office 365 administrative units

License management

Azure Active Directory Group-Based License Management

Using PowerShell to manage licenses

Assigning and removing licenses with PowerShell

Tracking down unlicensed accounts

Managing Guest User Accounts

Managing Feature Releases

Backup for Office 365

Backup for Exchange Online

Backup for SharePoint Online

Backup for Office 365 apps

Backup Technology Changes

Monitoring

Service Health Dashboard

 Message Center

Monitoring Systems

Service requests

Adding a new service request

Gathering tenant information

Customer lockbox

Custom Help

Custom Tiles

Reporting Portal

Microsoft Secure Score

Managing Exchange

Chapter 5: Exchange Online

Overview of Exchange Online

Consumer Email

The Future for On-Premises is in The Cloud

The Role for Exchange Within Office 365

Exchange 2019

Cloud Mailbox Storage Quirks

Exchange Administration Center

Managing Exchange Online with PowerShell

Connecting to Exchange Online

Mixing Online and On-Premises Cmdlets

Multi-Factor Authentication for Exchange Online

PowerShell

Differences with On-Premises Exchange PowerShell

Limited Sessions

Blocking Basic Authentication

Creating a Protocol Authentication Policy

Changing Protocol Settings

Assigning Policies to Users

Checking Policies Are Applied to Accounts

Defining a Default Protocol Authentication Policy

Moving to Mailboxes

Chapter 6: Managing mailboxes

User Mailboxes

Creating a New Mailbox

Editing a Cloud Mailbox

User Role Assignment Policies

Creating a Mailbox with PowerShell

Creating a Mailbox-Enabled Office 365 Account

The Link Between EXODS and Azure Active Directory

 Mailbox Plans

Determining Mailbox Location

Administrator Mailboxes

Sending Messages with PowerShell

Personal Mailbox Settings

Granting Delegate Access to a Calendar

Regional Settings

Auto Replies and Out of Office

Calendar Configuration

General Mailbox Configuration

Controlling Email Sent by Delegates

Junk Mail Settings

Maximum Message Size

Enabling Third Party Cloud Attachments

Autodiscover

MAPI over HTTP and RPC over HTTP

Connecting to Your Mailbox

Inside Mailboxes

Quotas

Offline Storage

Mailbox Cleanup

Folder Associated Information

Hidden Mailbox Items

System Folders

The Files System Folder

The Big Funnel Mailbox Index

Figuring Out the Last Login for a Mailbox

Mailbox Backups

Automatic Mailbox Maintenance

Retaining Mailbox Items

Archive Mailboxes

How Information Moves into an Archive Mailbox

Effective Use of Archive Mailboxes

Auto-Expanding Archives

The Archive Folder

Inbox Rules

Using PowerShell to Manage Rules

Checking for Rules that Forward Email

Calendar Sharing

Allowing Cross-Tenant Free/Busy Sharing

 The Search-Mailbox Cmdlet

Running Search-Mailbox

Output from Search-Mailbox

Removing Mailbox Content

Audit Records for Search-Mailbox

Recovering Deleted Mailboxes

Inactive Mailboxes

Finding Inactive Mailboxes

Making Mailboxes Inactive with Holds

Checking Holds

Making a Mailbox Inactive Immediately

Restore or Recover Inactive Mailboxes

Preserving a Tenant

Securing the Data of Ex-employees

Blocking an Office 365 User Account

Putting the Mailbox on Litigation Hold

Long-term Mailbox Preservation

Dealing with Other Office 365 Information

Handling Inbound Email for Ex-Employees

When Someone Dies

Compromised Accounts

Even More Email

Chapter 7: Managing Shared Mailboxes and DLs

Shared Mailboxes

Mailbox Delegates and Permissions

Creating and Managing Shared Mailboxes with PowerShell

Who Sent That Mail?

Accessing Shared Mailboxes

Mailbox Automapping

Converting a Shared Mailbox to a Regular Mailbox

Redirecting Email

Redirecting Mail Sent to a Shared Mailbox

Personal Forwarding

Using Mobile Devices with Shared Mailboxes

Recipient Moderation

Mail Contacts and Mail Users

Distribution Lists

Creating a New Distribution List

Updating Group Membership with PowerShell

Dynamic Distribution Lists

 Room Lists

Reporting Distribution List Membership

Finding Inactive Distribution Lists

Controlling User-Created Distribution Lists

Distribution List Naming Policy

Migrating On-Premises Distribution Lists to Exchange

Online

On to SharePoint

Chapter 8: SharePoint and OneDrive For Business

SharePoint Online

Introduction to SharePoint Online

Managing SharePoint Online

SharePoint Settings

Using PowerShell to manage SharePoint Online

Sharing with Office 365 Groups or Teams

SharePoint Online Extensibility Options

Fast Wins with SharePoint Online

OneDrive For Business

Introduction to OneDrive For Business

OneDrive Client and Synchronization

Managing OneDrive For Business

Migrating to SharePoint Online and OneDrive For Business

Preparing to Move from On-Premises SharePoint

Moving Windows File Servers to SharePoint

SharePoint Migration Tools

Deprecated SharePoint Online Features

Other services that use SPO and ODFB

Moving Forward

Chapter 9: Delve and the Graph

Mastering Information

Office Graph

The Graph Framework

Delve and Office Graph

Enabling Office Graph

Delve Browser App

Delve Search

Delve User Profiles

Content cards

Delve Boards

Making Delve your Start Page

Privacy and Security

 Disabling Delve’s Ability to Show Documents Authored by a
User

Delve and Exchange Online

Hybrid Delve

Delve Apps

Automating Office 365

The Microsoft Graph

The Microsoft Graph Explorer

Clients

Chapter 10: Managing Office 365 Clients

An Array of Clients

Browsers

Protocols

Outlook

Features and Subscriptions

Office Clients

Office Update Channels

The Office Insider program

Installing Office 365 ProPlus

Installing Office 365 ProPlus from a network share

Managing updates to Office 365 ProPlus Click-to-Run

Managing Updates to MSI Versions of Office

Office 365 Support for Office 2016 MSI

Troubleshooting Office Clients with SaRA

Outlook Clients

A new version of Outlook makes its debut

Using Outlook in a virtualized desktop environment

Managing mailbox client protocols

Outlook for iOS and Android

Outlook Web App

Web browser compatibility

Outlook Web App mailbox policies

Customizing the OWA URL

Exchange Web Services

The Future of EWS

Using the EWS allow or block list

Blocking/allowing Mac clients

Blocking/allowing Outlook EWS access

Other uses for EWS

POP and IMAP

Client settings for POP and IMAP

 POP and IMAP protocol defaults

Client Access Rules

OneDrive for Business

Deploying the OneDrive Sync Client

Managing OneDrive sync client updates

Restricting OneDrive sync to domain-joined computers

Skype for Business

Teams

Proxies and Network Devices

Mobile Devices and Applications

On to Groups

Chapter 11: Office 365 Groups

A Group Identity and Membership Service

Applications that use Office 365 Groups

Explaining the Member Limit

Core Principles

The Evolution of Office 365 Groups

Group Members

Public or Private Groups

Conversations

Document Libraries

Use Cases

Implementing Office 365 Groups

Creating New Office 365 Groups

Updating Group Membership

Welcome Notifications

Creating Office 365 Groups from the Azure Portal

Updating Group Properties

Email Address Policies and Groups

Assigning Send As and Send On Behalf Of Permissions to

Groups

Using Office 365 Groups with OWA

Discovering Groups

Inviting Others to Join a Group

Searching Group Content

Creating and Managing Office 365 Groups Through OWA

Editing Group Properties

Controlling Group Email Traffic

Using Office 365 Groups with Outlook Desktop

Accessing Groups with Outlook

Groups Toolbar
 Creating a New Outlook Group with Outlook

Updating Group Properties with Outlook

Offline Access for Office 365 Groups

Outlook for Mac

Groups and Transport

Mobile Groups

Removing and Recovering Office 365 Groups

Recovering a Deleted Group with PowerShell

Recovering Individual Documents and Items

Yammer and Office 365 Groups

The Evolution of Yammer Groups

Requirements for New-Style Yammer Groups

Yammer Discussions

Yammer Files

Evaluating Yammer, Groups, and Teams

Power BI and Groups

Stream and Groups

More Groups

Chapter 12: Managing Office 365 Groups

Groups Policy

Licensing Groups Functionality

Creating and Updating the Groups Policy

Classifications Policy Settings

Controlling Group Creation

Policy-Based Group Creation

OWA Mailbox Policy for Group Creation in OWA and

Outlook

Group Naming Policy

Usage Guidelines Policy Settings

Group Policy Settings in Exchange Organization

Configuration

Guest Access for Office 365 Groups

Azure B2B Collaboration

What Guests Can Access in Office 365 Groups

Restricting Guest Access

Adding Guests to an Office 365 Group

How Guests Join an Office 365 Group

Guest Access to Group Resources

Guests and Group Conversations

Controlling Guest Access to Protected Information

Updating Guest Accounts

 Resetting Membership for Guests

Mail Contacts and Guest Accounts

Controlling Guest Access to Groups

Using the Groups policy to Control Guest Access

Blocking Guest Access for Selected Groups

Azure B2B Collaboration Policy

Adding Guests Programmatically

Cleaning up Guest Accounts

Office 365 Groups and Compliance

Dynamic Office 365 Groups

Checking Dynamic Membership

User Access to Dynamic Groups

The Cost of Dynamic Office 365 Groups

Group Expiration Policy

Expiration Policy Timeline

Expiry Notifications

Restoring Expired Groups

Case Study – Microsoft IT

Migration from Email Distribution Groups

Migration from the Exchange Administration Center

User-Controlled Conversion

Microsoft’s PowerShell Scripts

The DIY Approach to Converting Distribution Groups

Migration from Site Mailboxes

Comparing Office 365 Groups, Distribution Groups, and Shared Mailboxes

APIs for Office 365 Groups

On to Teams

Chapter 13: Teams

Chat-Based Collaboration for Office 365

The Competition for Teams

Teams as the Modern Outlook

Versions

Teams Architecture

Location of Teams Data

Use of Exchange Online

Microservices

Dependencies on Other Office 365 Components

The Transition from Skype for Business

The Structure of Teams

Owners and Members

 Channels

Tabs

The Teams Wiki and OneNote

Pinned Apps

Teams Clients

Desktop Client Updates

Teams Phone Application

User Settings for Clients

Keyboard Accelerators

Debugging Teams Clients

Teams Management

Teams and Skype for Business Admin Center

Licensing Teams

Release Notes

Access for On-Premises Users

Creating Teams

Using an Existing Team as a Template

Adding Team Members

Creating an Org-Wide Team

Joining a Team

Leaving a Team

Maintaining Team Membership

Teams Messaging Policy

Using Teams as Distribution Lists

Enabling Existing Office 365 Groups for Teams

Dynamic Teams

Hiding Teams from Exchange Clients

Managing Settings for a Team

List All My Teams

Members and Manage Team

Deleting (and Restoring) Channels and Teams

Conversations, Meetings, and Files

Keeping Conversations Focused

Making Your Point

Bookmarks

Translations

The Activity Feed

@Mentions

Processing the Activity Feed

Searching Teams Content

 Removing Messages

When Members Leave

Personal Chats

Meetings

Teams and the Phone System

Viewing Organizational Information

Files: The Link Between Teams and SharePoint Online

SharePoint Tabs

Linking to Files

OneNote

Sharing Files

SharePoint News Connector

Cloud Storage

Guest Access for Teams

Enabling Guest Access for Teams

Adding Guests to Teams

Multi-Factor Authentication for Guests

Blocking Guests from Specific Domains

Switching Between Tenants

Removing Guests

Emailing Teams

Channel Email Addresses

Message Hygiene for Inbound Email to Teams

Audit Records for Email Addressees

Capturing Email Sent to Teams

Teams and Compliance

Capture of Teams Compliance Records

Storage of Teams Compliance Records

Searching Teams Compliance Records

Teams and Office 365 Retention Policies

Team Expiry

Archiving Teams

Auditing Teams

Teams and Planner

Extending Teams

Teams Store

Teams and Connectors

Teams and Bots

Can Teams Replace Email?

Migration to Teams
 PowerShell

Chapter 14: Managing Office 365 Groups and Teams with PowerShell

The Power of the Shell

Choosing Cmdlets

Performance

Speed Group Retrieval with Get-Recipient

Basic Management of Office 365 Groups

The New-UnifiedGroup Cmdlet

Creating Office 365 Groups with Azure Active Directory

Cmdlets

Creating a Dynamic Office 365 Group

Outputting a List of Groups

Setting Group Properties

Controlling Who Can Send Email to a Group

Managing Who Can Edit Group Calendar Items

Using Exchange Online Cmdlets with Office 365 Groups

Managing Group Membership

Creating an All Users Group

Checking Group Membership

Checking if Someone is in a Group

Removing Members from Groups

Checking Groups with Guests

Removing Unwanted Guests

Synchronizing Office 365 Groups with Security Groups

Flagging Unowned Groups

Checking for Unused Groups

Blocking Guest Access to Classified Groups

Archiving Inactive Office 365 Groups

Group Reactivation

Tracking Archived Groups

Working with the Group Expiration Policy

Managing Teams with PowerShell

Permissions

Listing Teams

Creating New Teams

Viewing Team Settings

Updating Team Settings

Viewing Team Membership

Checking for Active Teams

Using Cmdlets from Other Modules with Teams

Office 365 Connectors

 Actionable Messages

Adding a New Connector for an Office 365 Group

Using the Incoming Webhook Connector

Disabling Connectors

Plans

Chapter 15: Planner

Lightweight Planning

To Do Lists

Planner and Project

Planner, Office 365 Groups, and Applications

Planner Services

Planner Clients and Integrations

Planning Basics

Creating New Plans

Tasks and Buckets

Changing Plan Settings

Private or Public

Closing a Plan

Charts

Schedule View

Task Filters

Creating and Managing Tasks

Task Assignment

Building Out Tasks

Task Status and Tabs

Comments and Conversations

Printing and Exporting Planner Data

Synchronizing Tasks to Outlook Calendars

Planner Guests

Planner PowerShell

Phones and Meetings

Chapter 16: Teams Meetings and Cloud Voice

Skype vs Teams

Moving from Skype to Teams

Skype and Teams interoperability

Licensing Requirements for Skype in Teams

Client Calling and Meeting Capabilities

Certified for Teams

Meetings in Teams

Audio Conferencing for Teams Meetings

 Cloud Recording

Teams Live Events

Teams Meetings and Cloud Video Interoperability (CVI)

Calling in Teams

Communications Credits

Phone System Capabilities

Phone System with Calling Plans

Phone System with Direct Routing

Succeeding with Call Quality

Understand Signaling and Media

Codecs and their impact on the network

Network factors that affect voice quality

Planning for Optimal Media Path

Network Planner

Network Assessment Tool

Call Analytics and role-based access control

Call Quality Dashboard

Mail Flow

Chapter 17: Mail Flow and Exchange Online Protection

Exchange Online Protection

How Exchange Online Protection Processes Email

Configuring Mail Flow

Setting up Basic Mail Flow

Managing Connectors

No Open Relays

Advanced Mail Flow Scenarios

Mail Flow Insights

Transport Limits

Mail Flow Rules

Mail Flow Rule Conditions

Mail Flow Rule Exceptions

Mail Flow Rule Actions

Creating a New Mail Flow Rule

Testing New Mail Flow Rules

Common Use Cases for Mail Flow Rules

Exporting and Importing Mail Flow Rules

Message Hygiene

Anti-Spam and Anti-Malware Protection

Anti-Spam Message Headers

The Decision to Use Multiple Filtering Solutions

 User-managed Safe and Blocked Sender Lists

Junk Mail and Phish Reporting

Message Forwarding and the Sender Rewrite Scheme

(SRS)

Sending Messages to the Junk Email Folder in an On-

premises Environment

Message Quarantine

NDR Backscatter Filtering

Directory-based Edge Blocking (DBEB)

Bulk Email Filtering

Advanced Filtering Options

Domain Keys Identified Mail (DKIM)

Domain-based Message Authentication, Reporting and

Conformance (DMARC)

Anti-spoofing Protection and Reporting

Safety Tips

Reporting Phishing messages to Microsoft

Zero-hour Auto Purge (ZAP)

Invalid and Non-Registered Domain Emails

Testing Anti-Spam

Delisting Your IP Address from the Office 365 Block List

Unblocking Blocked Users

Anti-Malware Protection

Office 365 Advanced Threat Protection

S/MIME

Message Tracing

Tracing Messages

Tracing Messages with PowerShell

Mobile Devices Need Management Too

Chapter 18: Mobile Devices, EMS, and Intune

Comparing the three solutions

Exchange ActiveSync

Configuring mobile devices and applications for Exchange

ActiveSync

Understanding device access state for ActiveSync clients

Mobile device authentication

Mobile device mailbox policies

Allow/block exemptions

Device access rules

Default access level

Dealing with existing devices when changing default access

level

Managing Mobile Device Associations

 Reporting Remote Devices

Remote Wipe

Establishing an ActiveSync policy for your organization

Office 365 MDM

Device and Application Support for Office 365 Mobile

Device Management

Activation and Initial Configuration

Managing Device Policies in Office 365 Mobile Device

Management

Enrolling Devices for Office 365 Mobile Device

Management

Removing or Wiping Mobile Devices

Renewing the APNs Certificate for Office 365 Mobile

Device Management

Microsoft Intune

Preparing Intune for an organization

Managing Intune users and groups

Managing Intune administrative permissions

Device Enrollment

Device Compliance Policies

Device Configuration Policies

Deploying Applications

Mobile Application Management

Implementing Intune Solutions

Moving on from mobility

Chapter 19: Office 365 Data Governance

Data Governance in Office 365

Keeping Content in Place

Principles of Data Governance

The Influence of GDPR

Retention Policies and Label Policies

Rules of Retention

Office 365 Retention Policies

Broad and Narrow Retention

Retention Policy Scope

Automatic Policies

The Effects of Retention Policies

Permissions

Planning an Office 365 Retention Policy

The Effect of Retention

Office 365 Retention Policies and Inactive Mailboxes

Advanced Retention Settings

 Behind the Scenes with Advanced Retention Policies

Knowing That a Retention Policy Works

Applying Retention Policies to Office 365 Groups

Removing Retention Policies

Preservation Locks

Office 365 Labels

Retention Label Concepts

Planning Retention Labels

Creating New Retention Labels

Naming a Retention Label

Retention Actions and Periods

Comparing Retention Labels and Retention Policies

Publishing Retention Labels Through a Label Policy

Applying Retention Labels Through Auto-Label Policies

Using Retention Labels

Using Retention Labels with Exchange

Integration with Exchange Retention Policies

Using Retention Labels with Office 365 Groups

Using Retention Labels with SharePoint and OneDrive for

Business

Audit Records Generated When Retention Labels are

Applied

Label Activity Explorer

Default Assignment of Retention Labels

Record Labels

Processing Manual Dispositions

Deciding Whether to Delete

Event-based Retention

Removing Retention Labels

Finding Items Marked with Labels in Content Searches

Using the Compliance PowerShell cmdlets

Working with Retention Labels

Working with Retention Policies

Reporting Retention Policies Applied to SharePoint

Tracking Retention Holds for Mailboxes

Exchange Retention Holds

Migration

Exchange Mailbox Lifecycle

Cleaning Mailboxes

Recoverable Items Quota

Exchange Mailbox Retention Policies

 Managing Hybrid Mailbox Retention Policies

Benefits of a Mailbox Retention Policy

The Default Retention Policy

Retention Actions and Retention Periods

The Recoverable Items Folder Tag

Why Items Stay in The Deleted Items Folder

Creating a Mailbox Retention Policy

How Clients Use Mailbox Retention Policies

How MFA Processes Retention Policies

Stopping MFA Processing a Mailbox

Moving from Exchange Retention Policies

On to eDiscovery

Chapter 20: eDiscovery and Content Searches

Content Searches

Content Search Scalability

Creating a Content Search

Previewing Search Results

Exporting Search Results

Downloading Exported Results

Export Search Report

Reopening a Search

Compliance Boundaries

Office 365 eDiscovery

eDiscovery Case Components

Creating a New eDiscovery Case

eDiscovery Case Holds

eDiscovery Case Searches

eDiscovery Case Exports

Bulk Actions

Closing eDiscovery Cases

GDPR Data Subject Requests

Protected Documents and Searches

Auditing of Search Activities

Advanced eDiscovery

Using PowerShell with Content Searches

Creating and Running a New Content Search

Searching SharePoint and OneDrive for Business

Content Search Actions

Using a Content Search to Find and Remove Mailbox Items

Exporting Search Data

 Using PowerShell to Manage eDiscovery Cases

GDPR DSRs

Adding eDiscovery Managers

Reporting Holds for eDiscovery Cases

In-place Holds and Litigation Holds

Ingesting Items for Review from Non-Office 365 Sources

Reporting what happens

Chapter 21: Auditing and Reporting Office 365

Office 365 Auditing

Enabling the Office 365 Audit Log in a Tenant

ISV Access to Audit Data

Searching Office 365 Audit Events

User and System Events

Examining an Audit Record

Viewing User Activity

Searching Office 365 Audit Data with PowerShell

Practical Examples of Using Audit Information

Activity Alerts and Alert Policies

Activity Alerts

Alert Policies

Handling Alerts

Activity Alerts and Policies and PowerShell

Office 365 Cloud App Security

Alerts

Resolving Alerts

Filters

Policies

Scoping Policies to Groups

Third-Party Audit Alternatives

Exchange Online Administrative Auditing

Accessing Administrative Audit Log Entries

Exchange Online Administrative Audit Configuration

Exchange Online Mailbox Auditing

Enabling Auditing for a Mailbox

Making Sure Auditing is Enabled for All Mailboxes

Mailbox Auditing Enabled by Default

Supervision Policies

Components of a Supervision Policy

Creating a New Supervision Policy

Updating a Supervision Policy

 Processing Messages for Review

Reviewing Selected Content

Removing a Supervision Policy

PowerShell and Supervision Policies

Reporting Supervision Policies

Reporting Office 365

Office 365 Reporting Web Service and DataMart

Using PowerShell to Create Custom Office 365 Reports

Office 365 Admin Center Reports

Security and Compliance Reports

Anonymized Report Data

Microsoft 365 Usage Analytics Power BI Content Pack

Independent Reporting Products

Service Accounts

Extracting More Detail About Tenant Activity

Protection is Important

Chapter 22: Data Loss Prevention

Stopping Leaks

Office 365 Data Loss Prevention

Sensitive Data Types

Custom Classifications

Office 365 DLP Policies

Checking Documents for Sensitive Data

Default DLP Policy

Creating Office 365 DLP Policies

Default Rule Settings

Advanced Rule Settings

Protecting Sensitive Email with Encryption

GDPR Support

Sample Test Data

PowerShell support for Office 365 DLP policies

Exchange DLP policies

Designing an Exchange DLP policy

Blocking Actions

Policy Modes

Starting with a DLP Template

Creating a New DLP policy for Exchange

Outlook and DLP

OWA and DLP

Customizing Standard Policy Tips

 DLP Incident Reports

Building out the DLP policy

Document Fingerprinting

Hybrid Exchange DLP

Flow

Chapter 23: Automating Office 365 with Flow

The Need for Automation

Microsoft Flow

What is Microsoft Flow?

Older Office 365 Workflow Solutions

Enabling Flow

Licensing

Enabling Flow for Users

Using Flow Admin Center

PowerShell for Flow

Building Automated Solutions with Flow

Building Your First Flow

Sharing Flows

Building and running Flows in Excel

Using Data Loss Prevention Policies

Accessing On-premises Data with the Data Gateway

Building Advanced Solutions with Flow

Flow is Here to Stay

Protection

Chapter 24: Protecting Office 365 Content

The Need to Protect Data

Office 365 and Rights Management

The Flow of Protection

Protection Templates

External Users

Contacting the Azure Information Protection Team

Enabling Rights Management

PowerShell for Rights Management

Configuring Rights Management for Exchange Online

Bring Your Own Key (BYOK)

Protecting Email

Enlightened and Unenlightened Clients

How Rights Management Templates Protect Email

Protecting Email with Outlook

Refreshing Templates for Outlook

 Viewing Rights

Protecting Email with OWA

Inbuilt Message Encryption

Revoking Encrypted Messages

Encrypt or Protect

Protecting Email Sent to Shared Mailboxes

Office 365 Sensitivity Labels

Sensitivity Labels

Creating a New Sensitivity Label

Publishing Sensitivity Labels

No Auto-Label Policies

Removing Sensitivity Labels

Using Sensitivity Labels with Auto-Signature Products

Applying Protection with Transport Rules

Configuring a Transport Rule for Protection

Applying Sensitivity Labels with Transport Rules

Customizing the OME Configuration

Moving from Office 365 Message Encryption V1

Protecting SharePoint Online and OneDrive for Business

Enabling Rights Management for SharePoint Online and

OneDrive for Business

Protecting Document Libraries and Lists

Synchronizing Protected Libraries with OneDrive

OneDrive for Business Permissions

Protecting Individual Documents

Functionality Lost for Protected Documents

Indexing Labels with SharePoint

Hybrid Protection

Super-Users

Protecting Individual Files

Protected PDFs

Using PowerShell to Protect Information

Processing Protected Documents Found in Content

Searches

Sensitivity Labels and PowerShell

Logging Protected Documents

The End or The Beginning

Chapter 25: Office 365 Migration for Enterprises

A Short History of Quadrotech

Archive Shuttle

Mailbox Shuttle
Public Folder Shuttle

PST Flight Deck: eradicating the PST pests

Other PST problems:

Compliance

User experience

Automation

Mastering Cloud Migration

Beyond Migration: Analysis, Reporting, and Insights

Radar Reporting

Radar Security & Audit

GDPR compliance in Office 365

The future of Quadrotech

Cloud Commander: The Need for Speed

Achieving the best migration speed

Reducing the overall migration duration

Putting Office 365 Administration on Autopilot

The Problem with the Native Approach

Virtual OUs

Authorization Policies

Configuration Policies

How Does Autopilot Work?

What Workloads are Currently Supported?

About Quadrotech