You are on page 1of 281

Microsoft Solutions for Security and Compliance Server and Domain Isolation Using IPsec and Group Policy

© 2006 Microsoft Corporation. This work is licensed under the Creative Commons Attribution-NonCommercial License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.

Table of Contents

iii

Table of Contents
Chapter 1: Introduction to Server and Domain Isolation.....................1 Executive Summary............................................................................1 Who Should Read This Guide...............................................................2 The Business Challenge.......................................................................3 Creating a Project Team......................................................................6 Guide Overview..................................................................................7 Scenario Outline.................................................................................9 Summary..........................................................................................14 Chapter 2: Understanding Server and Domain Isolation....................15 Chapter Prerequisites.......................................................................15 Who Should Read This Chapter.........................................................16 Identifying Trusted Computers.........................................................20 How Does Server and Domain Isolation Fit into My Overall Network Security Strategy?............................................................................24 Terminology Refresher......................................................................27 How Can Server and Domain Isolation Be Achieved?.........................30 What Does Server and Domain Isolation Protect Us From?................39 How Can We Deploy Server and Domain Isolation?...........................40 Summary..........................................................................................41 Chapter 3: Determining the Current State of Your IT Infrastructure. .43 Chapter Prerequisites.......................................................................43 Who Should Read This Chapter.........................................................44 Identifying Current State..................................................................45 Capacity Considerations ...................................................................56 Predeployment Concerns..................................................................57 Trust Determination..........................................................................59 Capturing Upgrade Costs for Current Hosts.......................................64 Summary..........................................................................................66 Chapter 4: Designing and Planning Isolation Groups.........................69 Chapter Prerequisites.......................................................................69 Creating the Server and Domain Isolation Design..............................70 Group Implementation Methods........................................................86 Group Implementation for Woodgrove Bank.....................................89 Summary..........................................................................................90 Chapter 5: Creating IPsec Policies for Isolation Groups.....................93 Chapter Prerequisites.......................................................................93

iv

Server and Domain Isolation Using IPsec and Group Policy

Summary........................................................................................131 Chapter 6: Managing a Server and Domain Isolation Environment...133 Chapter Prerequisites.....................................................................133 Change Management.......................................................................134 Backup/Restore Considerations......................................................148 Mitigation of Network-Based Infections..........................................149 Summary........................................................................................152 Chapter 7: Troubleshooting IPsec...................................................153 Support Tiers and Escalation...........................................................153 Tier 1 Troubleshooting....................................................................154 Tier 2 Troubleshooting Preparation.................................................162 The IPsec Troubleshooting Process.................................................171 Tier 3 Troubleshooting....................................................................209 Summary........................................................................................210 Appendix A: Overview of IPsec Policy Concepts...............................213 Introduction...................................................................................213 IPsec Policy Filters..........................................................................214 IKE Negotiation Process..................................................................217 Security Methods............................................................................219 IPsec Encapsulation Modes and Protocol Wire Formats...................219 IKE Authentication..........................................................................220 IKE Authentication Methods and Security Method Preference Order 223 Security Negotiation Options...........................................................224 Appendix B: IPsec Policy Summary.................................................227 General Policy Configuration...........................................................227 Isolation Domain Policy..................................................................231 No Fallback Isolation Group Policy..................................................231 Boundary Isolation Group Policy.....................................................232 Encryption Isolation Group Policy...................................................233 Appendix C: Lab Build Guide............................................................235 Prerequisites..................................................................................235 Deployment of the Baseline Policy...................................................237 Implementing the IPsec Policies.....................................................237 Using the Policy Build-up Method to Enable the Baseline IPsec Policy 245 Test Tools and Scripts for the Functionality Tests............................248 Enabling Organization Secure Subnets Filter List on Remaining Policies 250 Enabling Network Access Group Configuration................................251

Table of Contents

v

Enabling the Isolation Domain........................................................256 Enabling the No Fallback Isolation Group........................................258 Enabling the Encryption Isolation Group.........................................260 Enabling the Boundary Isolation Group...........................................262 Configuring the Isolation Domain as the Default Isolation Group. . . .264 Final Functional Tests—All Isolation Groups Enabled.......................265 Summary........................................................................................270 Appendix D: IT Threat Categories....................................................271 Threats Identified by STRIDE..........................................................271 Other Threats..................................................................................274 Summary........................................................................................275 Links...............................................................................................277 Acknowledgments...........................................................................283

Feedback The Microsoft Solutions for Security and Compliance team would appreciate your thoughts about this and other security solutions. . We look forward to hearing from you.com. Please direct questions and comments about this guide to secwish@microsoft.

Chapter 1: Introduction to Ser ver and Domain Isolation The practice of physically isolating computers and networks to protect data or communications from being compromised has been used for many years. Although personal or host-based firewalls help secure clients when connected to the Internet. server and domain isolation allows IT administrators to restrict TCP/IP communications of domain members that are trusted computers. Most importantly. controlling physical access to the network can become impossible. The advent of wireless networks and wireless connection technologies such as General Packet Radio Service (GPRS) and Bluetooth has made network access easier than ever. employee misuse of technology assets. Server and domain isolation can be an important strategy in the defense against virus propagation. and information theft. If an attacker manages to gain physical access to a company's internal network and attempts to access a server that contains valued data assets. even if the attacker used a valid user account and password. either clients or servers. As organizations grow and business relationships change. At its core. vendors. Executive Summary Microsoft recognizes that large organizations face increasing challenges in securing the perimeter of their networks. Server and domain isolation provide a number of business benefits. In addition. Server and domain isolation make it possible to create a layer of security to achieve logical isolation of the network traffic that moves between computers or networks. it provides a layer of network security that can significantly reduce the threat of untrusted hosts accessing trusted domain members on an organization's internal network. The logical isolation approach using server and domain isolation techniques enables the development of a flexible. These trusted . scalable. server and domain isolation can block access simply because the computer that the attacker is using is not a trusted company device. and manageable isolation solution that provides the security of isolation without the cost or inflexibility of physical boundaries. they are not yet well-suited for protecting internal servers and clients. Every day. customers. internal hackers. This increased connectivity means that domain members on the internal network are increasingly exposed to significant risks from other computers on the internal network. The prevalence of mobile clients and the nature of distributed network environments make such physical limitations too inflexible to implement and operate. in addition to breaches in perimeter security. Server and domain isolation can also be used either as a primary or an additional strategy for meeting data privacy or other protection requirements for data in network traffic. it can be used to require domain membership of all clients that seek access to trusted resources. The problem with physical isolation is that the information technology (IT) infrastructures of many enterprise organizations cannot easily be protected behind hard physical boundaries. and consultants connect mobile devices to your network for valid business reasons. without modifying existing Microsoft® Windows® applications or deploying virtual private networking (VPN) tunneling hardware on the network. so that they can be better managed by professional IT staff.

Communication with non-Windows platforms or untrusted systems is controlled by the IPsec configuration having a list of exemptions and/or being allowed to fall back to clear text (non-IPsec) communication. the various chapters that comprise this guide have been written to meet the needs of a variety of readers. Furthermore. starting at the initial evaluation and approval phase and continuing through to deployment. or a specific group of trusted computers. and server isolation to ensure that a server accepts network connections only from trusted domain members or a specific group of domain members. Microsoft supports the long-term direction they take toward a highly secure and manageable infrastructure for trustworthy computing. and management of the completed implementation. Who Should Read This Guide This guide is designed to support a server and domain isolation solution through all stages of the IT lifecycle. The concept of logical isolation presented in this guide embodies two solutions—domain isolation to isolate domain members from untrusted connections. The tested solution provided in this guide requires the following minimum platform configuration: • • • Windows 2000 with Service Pack 4 or later Microsoft Windows Server™ 2003 Windows XP with Service Pack 2 or later These configuration requirements ensure that the IPsec components are at the required revision level. Because both server and domain isolation scenarios exist within Microsoft's global internal network. Nearly all TCP/IP network connections are able to be secured without application changes.computers can be configured to allow only incoming connections from other trusted computers. The access controls are centrally managed by using Active Directory® Group Policy to control network logon rights. This guidance uses the Woodgrove National Bank scenario to demonstrate the implementation of both server and domain isolation in a representative lab environment. This chapter is designed primarily for the business decision maker who is trying to determine whether his organization will benefit from a server and domain isolation project. It also presents the experience that Microsoft gained in implementing both of these solutions internally as well as in customer environments. and both Microsoft professional IT staff and a number of customers through a Beta program have reviewed it. in a variety of customizable scenarios. in recommending these solutions to its customers. 3. because Internet Protocol security (IPsec) works at the network layer below the application layer to provide authentication and per-packet. state-of-the-art security endto-end between computers. A Microsoft team of subject matter experts developed this guidance. These solutions can be used separately or together as part of an overall logical isolation solution. Microsoft's business depends on the security of these solutions. Network address translators are no longer a barrier to using IPsec on the local area network (LAN) with the new network address translation (NAT) traversal capabilities. The planning chapters of this guide (Chapters 2. Understanding the contents of this chapter requires no specific technical knowledge beyond the comprehension of the organization's business and security needs. and 4) are intended to be most helpful to the technical architects and IT professionals who will be responsible for designing a . Network traffic can be authenticated. The Group Policy and IPsec configurations are centrally managed in the Active Directory. For this reason. or authenticated and encrypted. testing.

Although logical isolation cannot be considered an antivirus defense on its own. This approach makes it possible to reduce the potential risk to important business data from unidentified and unmanaged computers that connect to your network. A logical isolation defense layer provides additional security for all managed computers on the network. The question is how to limit such access to the organization's resources to only those computers that need it. Chapter 7 provides information about troubleshooting an IPsec deployment. vendor. in addition to remote computers in small branch offices or in employees' homes. because it provides an additional layer of security that can reduce the likelihood of an attack and minimize its scope should one occur. The Business Challenge The nature of today's highly-connected organizations and mobile network devices can introduce many risks into your organization's IT infrastructure. The authentication occurs before the user logs on and is effective while the computer is connected. A good level of technical understanding of both the technologies involved and the organization's current infrastructure is required to get the most benefit from these chapters. The answer is server and domain isolation. In many cases. The Business Benefits The benefits of introducing a logical isolation defense layer include the following: • • Additional security. Chapter 5 and the appendices are designed for the support staff that is responsible for creating the deployment plans for the organization's solution. however. computers will not automatically gain access to all network resources simply by connecting to the network. Note For more information about malware and specific ways that your organization can defend itself. the customer has bypassed any perimeter defenses that are in place to protect against network-based attacks. including mobile employee. and customer computers. a technique that identifies and authenticates the computer itself to determine which resources it is allowed to access. Tighter control of who can access specific information. Chapter 6 of the guide is intended as a reference for the staff that is responsible for the day-to-day operations of the solution after it is implemented and fully operational. it can be part of a broader antivirus solution. these risks come from malicious software (also known as malware). By connecting directly to your physical network. By using this solution. A number of operating processes and procedures highlighted in this chapter should be built into the organization's operations framework. a visiting customer who connects a mobile computer to your network to provide you with a spreadsheet introduces a risk to your organization's IT infrastructure. . troubleshooting information and techniques can significantly help organizations that choose to implement IPsec. such as viruses and worms. see The Antivirus Defense-in-Depth Guide. For example. Included in this guidance are a number of recommendations about the process of completing a successful solution deployment as well as practical implementation steps to create the test lab environment.Chapter 1: Introduction to Server and Domain Isolation 3 customized solution for an organization. These risks can come from a variety of sources. this risk would be mitigated. If the internal network to which the customer connects did not allow direct access to your organization's servers. Because IPsec fundamentally affects network communications. that has been inadvertently downloaded or installed onto an innocent person's computer.

If an organization's information is only available to managed computers. Logical isolation makes it possible to require encryption of all network traffic between selected computers. This solution is typically far less expensive to implement than a physical isolation solution. such as those in a lobby. Rapid emergency isolation.4 Server and Domain Isolation Using IPsec and Group Policy • • Lower cost. Improved auditing. It might help to see exactly where the solution fits within a typical IT infrastructure and how it is designed to complement existing network defenses. The isolation solution will significantly restrict the ability of an untrusted computer to access trusted resources. . This solution provides a way to log and audit network access by managed resources. Protect publicly available network connections. all devices will have to become managed systems to provide access to their users. For this reason. Certain network connection points. will not provide direct access to all resources on your network. Improved levels of protection against malware attacks . a malware attack from an untrusted computer will fail because the connection will not be allowed. Increase in the number of managed computers. even if the attacker obtains a valid user name and password. A mechanism to encrypt network data. This solution provides a mechanism to quickly and efficiently isolate specific resources inside your network in the event of an attack. • • • • • The Technology Challenge Defending a modern IT infrastructure from attackers while simultaneously allowing employees to work in the most agile and productive manner is not an easy task. Simply understanding the wide range of technologies that can help secure an environment is difficult enough for many people.

which means they are exposed to higher risk of attack. which shows a typical network infrastructure consisting of a number of network defense layers. this network represents the grouping of the organization's networks that are physically located on the sites owned and managed as part of the IT infrastructure.1 is designed to provide a simple illustration of the various technologies that you can use to provide a defense-in-depth security design for a typical network infrastructure. illustrates where logical isolation fits within a typical environment: Figure 1.1 Infrastructure areas and network defense layers Figure 1. Typically. These devices often reside in a perimeter network that provides a higher level of protection from the external threats that Internet connectivity poses. This network is specifically set aside for servers and devices that require direct access to or from the Internet. • • • . These remote entities typically use VPNs to connect to the organization's internal network and access the organization's IT infrastructure. The organization's internal network is often connected to the Internet through one or more perimeter firewall devices. Internet.Chapter 1: Introduction to Server and Domain Isolation 5 The following figure. Internal network. Perimeter network. Such an infrastructure is typically made up of the following elements: • Remote workers and networks.

documented. these various network defenses are typically installed and managed as separate components of a defense-in-depth network design. It is essential that you complete appropriate planning and testing both before and during the solution deployment. The real technology challenge with logical isolation is implementing it in a manner that is both manageable and scalable for your organization. you should not view the internal network as simply one network where all physically connected computers are trusted to have full network access to one another. the computer and user will be authorized for full connectivity to the internal network. After logical isolation has been achieved. it is vital that all the needs and expectations of the organization are communicated. see the Network Access Protection page. a highly controlled level of access is usually granted to allow specific business applications or processes to occur through a VPN tunnel or perimeter router that provides extranet communications. Microsoft provides new capabilities for quarantining remote access computers using Network Access Protection (NAP).6 Server and Domain Isolation Using IPsec and Group Policy • Quarantine network. After successfully passing the necessary tests. Network edge firewalls protect communications between the Internet and the internal networks. This network is a relatively new component that provides limited connectivity to computers that fail to meet the minimum required security standards that the organization prescribes. This guide makes it possible to design a solution that is both scalable and manageable and that can also be deployed in a manner that is controlled and allows for testing at various points in the deployment phase. Producing a design that is so complex and restrictive that it impairs users' abilities to perform necessary business tasks could be worse than having no isolation solution at all. For more information. the additional layer of security will help reduce the risk to the various information assets on the network without restricting the functionality of authorized clients.1 illustrates that logical isolation is focused directly on the communications of internal network hosts. The goal of logical isolation is to allow the internal network to be segmented and isolated to support a higher level of security without requiring hard physical boundaries. For this reason. • Figure 1. which will in turn affect all departments and users that rely on these communications. Because these networks are not owned or managed by the organization. the quarantine network will provide them with enough connectivity to download and install the required elements that will allow them to pass the tests. Large internal networks sometimes support many organizations. and considered at all stages of this project. over the next few years these components will likely converge into a common network defense solution that can be implemented and managed as a single end-to-end solution. However. Creating a Project Team This solution will potentially affect all areas of the organization's internal network communications. understood. If they fail to pass the tests. Consequently. RAS in combination with NAP provides the functionality to manage remote worker connections using a quarantine network. and in some cases multiple IT departments must manage the computers and physical access points. . VPNs are managed by Remote Access Services (RAS) to allow traffic from remote workers or networks to connect securely from remote locations. Currently. Partner networks. What is missing in many current network designs is the ability to protect computers on the internal network from one another.

It explores the relationship between the concept of server and domain isolation. such as NAT. and Group Policy. and shows how they fit in a typical IT infrastructure. along with the steps required to obtain the information. This chapter provides guidance on the information that is required for planning.Chapter 1: Introduction to Server and Domain Isolation 7 Because it is not realistic to expect a single person to be able to perform all the tasks required in a project of this scope in a typical organization. design. internal firewalls and proxies. Guide Overview This section provides a brief synopsis of the contents of each chapter in the Server and Domain Isolation Using IPsec and Group Policy guide. server and workstation configurations. This chapter also provides information about the Woodgrove National Bank scenario. a project team is recommended. and testing purposes. and domain trusts. This project team should consist of representatives from all departments within the organization in addition to the key technical areas of the current IT infrastructure. IPsec-based remote access VPN clients. Chapter 2: Understanding Server and Domain Isolation Chapter 2 defines the concept of trusted hosts and discusses how trust can be used to create domain or server isolation solutions. it is assumed that a suitable project team will be in place during the lifetime of this project and that the requirements and goals of the solution will be adequately communicated to the project stakeholders and solution users at various stages of the project. The technical content of this guide provides a detailed technical explanation of what can be expected from the solution in terms of both the security threats it will help defend against and the technical issues that could be faced when using IPsec to create a domain or server isolation solution. Chapter 1: Introduction to Server and Domain Isolation The first chapter (this chapter) provides an executive summary and a brief introduction to the content of each chapter in the guide. It also covers the potential impact of other networking technologies. This information includes the current state of all network devices. see the Microsoft Solutions Framework (MSF) web site. For more information about how to organize a project such as this. . Because it is beyond the scope of this guide to explain how a project team should work within your organization. IPsec. which was adopted for proof-of-concept. it is imperative that the designers of the solution have upto-date and accurate information about the current IT infrastructure. It introduces the concept of logical isolation and the server and domain isolation approaches for an organization. discusses business justification. and internal port-based filtering. Chapter 3: Determining the Current State of Your IT Infrastructure Before a project is undertaken.

This chapter provides the information required to understand how to create these IPsec policies and how to deploy them to the recipient computers. and tips that you can use to help identify whether IPsec is the cause of such problems and. testing and deployment of the test lab environment at Microsoft during the development of this guide. All of the steps and processes in this chapter are illustrated with examples from the Woodgrove Bank scenario. job aids.8 Server and Domain Isolation Using IPsec and Group Policy Chapter 4: Designing and Planning Isolation Groups This chapter provides guidance on how to link the business requirements of the organization to the server and domain isolation design that will help achieve the requirements. This chapter presents a model for supportability and various management processes and procedures that you should use as part of a broader operations framework such as the Microsoft Operations Framework (MOF). Chapter 7: Troubleshooting IPsec As the solution is deployed and used. This chapter details a number of IPsec troubleshooting procedures. it is almost inevitable that problems will arise. there are a number of processes that you should understand and document to help ensure that the solution is managed correctly and supported on a day-to-day basis. tasks. . if it is. how to troubleshoot the problem. These rules are assigned and delivered to members of the trusted domain using Group Policy objects. The materials provided here are designed to be of assistance in all phases of the project. For more information about MOF. and scripts that were used in the planning. this guide includes a number of reference materials. Chapter 5: Creating IPsec Policies for Isolation Groups IPsec policy is the mechanism that you use to enforce the rules by which each computer communicates with its peers. tools. Chapter 6: Managing a Server and Domain Isolation Environment After the solution is in place and operational. You can use the information in these appendices to help in the implementation of your own server and domain isolation solutions. This chapter also describes different deployment approaches that you can use to help minimize the impact on the organization during deployment and to help maximize the chances of a successful implementation. from the initial envisioning right through to the day-to-day operations of a fully-deployed solution. A step-by-step approach is provided to help you create an isolation group design to achieve your organization's security requirements with regard to isolation. see the Microsoft Operations Framework web site. Appendices In addition to the main chapters.

corporate. there are two other primary corporate locations. WG owns five companies: Woodgrove National Bank. secondary sites. Sydney and Johannesburg. and Humongous Insurance. England. What Is Woodgrove Bank? Woodgrove National Bank is a fictitious proof-of-concept organization that Microsoft uses to provide a tangible customer example for demonstrating public deployment guidance. The autonomous branch locations have 2-MB connectivity. and brokerage services for financial institutions. and Tokyo (500 employees). As a bank. were used to develop this solution. Woodgrove Bank is a fully owned subsidiary of WG Holding Company. The details of this connectivity are illustrated in Figure 3. (For example. Connectivity Between Sites Tokyo and London are connected to enterprise headquarters in New York through private Internet connections.000 people. and individual clients in its role as a financial intermediary. The requirements of Woodgrove Bank are derived from the many experiences that Microsoft has had with enterprise customers. Specific legislative and regulatory requirements are not discussed here to avoid making the solution specific to one country or region.000 people in more than 60 offices worldwide. The micro branch offices typically have 1-MB wide area network (WAN) connectivity. each employing more than 5. financial advisory services.) In addition to the hub locations. . Its business includes securities underwriting. in addition to those of Microsoft." in this guide. Contoso. Each hub location supports a number of small. Ltd. However. and application servers. All regional hub locations are connected to enterprise headquarters with 2-megabyte (MB) to 10-MB connectivity. the guide makes special note where Woodgrove Bank requirements and staffing may differ from Microsoft's unique considerations. a representative physical lab implementation was tested based on the Woodgrove Bank customer scenario to provide a concrete and public model for this solution.Chapter 1: Introduction to Server and Domain Isolation 9 Scenario Outline Both the server and domain isolation solutions have been deployed internally within Microsoft's internal network.1 of Chapter 3. each with its own dedicated file.. venture capital. All of the companies owned by WG are large organizations. which is a leading global financial services company headquartered in London. This guidance incorporates many of the support and management techniques that Microsoft internal IT administrators use routinely. Woodgrove Bank is a leading global investment bank that serves institutional. the Woodgrove organization is highly reliant on security to ensure the safety of both their monetary assets and their customers' private data. print. Geographical Profile Woodgrove Bank employs more than 15. respectively.200 employees). Northwind Trading. "Determining the Current State of Your IT Infrastructure. The business and technical requirements of this fictitious organization. sales and trading. with committed bandwidth of 6 megabits per second (Mbps) and 10 Mbps connectivity. Litware Financials.000 employees). Woodgrove Bank also must comply with a number of regulatory requirements from government and industry groups. investment research. However. London (5. government. They have corporate headquarters (hub locations) with large numbers of employees in New York (5. New York supports sites in Boston and Atlanta.

improve manageability and reduce the cost of management. reduce the number of servers in the environment. and customers operate in the most secure environment Deliver the ability to have service level agreements (internally and externally) Increase business agility by providing meaningful real-time access to information from everywhere IT Organization Profile Although Woodgrove Bank has a mixed server environment that uses Windows and UNIX. They have a total of 1. Woodgrove Bank has a number of additional corporate initiatives that also affect IT. partners. most of which are running Windows 2000 or later: • • • • • • • • File and print servers Web servers Infrastructure servers Microsoft Exchange servers Development servers Monitoring servers Other (Lotus Notes.10 Server and Domain Isolation Using IPsec and Group Policy Enterprise IT Challenges Woodgrove Bank faces the same challenges that most enterprises do.712 Windows-based servers. there are additional specific challenges that IT decision makers face. they want to increase revenue and reduce costs while decreasing the cost of fixed assets. their infrastructure runs on a Windows Server backbone. including the following: • • • • • Diversify into new markets through mergers and acquisitions Exceed customer satisfaction Improve employee productivity Improve processes and operations Provide a secure environment Not only do these initiatives affect IT. including the following: • Reduce the overall cost of IT • Reduce operational costs. These challenges have an ongoing affect on IT. Oracle) 785 123 476 98 73 33 51 Microsoft SQL Server™ servers 73 . and consolidate heterogeneous applications and services onto single servers Capitalize on existing IT investments Create an agile IT infrastructure Increase utilization Improve availability and reliability Exploit new hardware platforms • • • • • • • • • Increase the return on investment Ensure that employees.

In total. Approximately 85 percent of those PCs are desktop computers. and 15 percent are mobile computers.2 The Woodgrove Bank centralized IT management model . Woodgrove Bank implements a centralized management model to manage servers and desktops centrally from New York City. System and Management Architecture Overview The Woodgrove Bank network consists of several IT zones: one corporate data center. Most employees have desktop PCs. As illustrated in the following diagram. two satellite offices. and sales representatives have mobile computers.Chapter 1: Introduction to Server and Domain Isolation 11 The majority of servers are located in the three corporate headquarter locations (New York. Woodgrove Bank has more than 17. and Tokyo). and a perimeter network to support remote users. two hub locations. PC Environment Most employees at Woodgrove Bank have at least one personal computer system.000 end-user PCs. Figure 1. More than 95 percent of the end user PCs are Intel-based PCs running some version of Windows. There are some Mac workstations and a few UNIX workstations being used by specific departments for Line of Business (LOB) applications. London.

. which is sufficient for management of perimeter servers.12 Server and Domain Isolation Using IPsec and Group Policy Directory Service Woodgrove Bank elected to adopt a service provider forest model for their Active Directory design. Replication requirements are minimal in the perimeter. Europe. so there is no need to provide replication boundaries or use the multiple regional domain model to segment the forest.3 The Woodgrove Bank directory service design The perimeter forest uses the single forest domain model. and Tokyo are the central hubs to the entire topology. and a subset of the OU structure is created in the forest root. This model provides a means of managing the replication topology as well as delegating administration of domain level autonomy to each region. Figure 3. This capability meets the isolation requirements of the servers in the perimeter network. In addition. a different domain design would have been required. and Asia (Asia Pacific). The OU structure is replicated in its entirety in each of the regional domains. It is crucial to note that Woodgrove chose this design for management of the perimeter servers only. If user accounts were placed in the perimeter and the perimeter was in multiple locations.2 in Chapter 3 of this guide provides a detailed diagram of the OU structure that Woodgrove Bank uses. Europe. The following figure illustrates the Active Directory logical structure that Woodgrove Bank uses: Figure 1. London. and Asia. Woodgrove did not choose the single forest model because it does not allow the perimeter servers to be isolated from critical corporate data. New York. Woodgrove created three regional domains: Americas. Woodgrove implemented a dedicated forest root to manage the forest level functionality. The designers at Woodgrove Bank chose to go with an organizational unit (OU) design that is primarily object-based. Site topology for Woodgrove Bank is divided into three regional areas: Americas. The reason was that this model provides the flexibility of having one forest for the perimeter and a separate shared forest for internal resources. The internal forest is based on a multiple regional domain model.

The goal of the proof-of-concept project was to provide enough diversity in the lab design to ensure that the solution functioned as expected. .Chapter 1: Introduction to Server and Domain Isolation 13 Woodgrove Strategy for Server and Domain Isolation Rollout To help the organization understand the design that best meets their requirements. Woodgrove Bank created a proof-of-concept lab project. This project used a small lab implementation in which to test Woodgrove's proposed design. Figure 1. while avoiding impact to production servers and the company users. which is shown in the following figure.4 Woodgrove Bank pilot design This diagram shows the subset of computers that were used in the Woodgrove Bank scenario to test the scenarios that this guide presents.

In addition. From this information it is possible to understand what this solution will provide to your organization. and troubleshooting. and what skills will be required to make the solution a success. Next. The IT administrators and support staff were trained in troubleshooting techniques. After completing these pilots. where it will fit within a typical IT infrastructure. These servers are ones that have low business impact if connectivity is affected. These servers were carefully monitored for performance and support call impact. The impact of this domain-isolation project was minimized by using IPsec only for the network subnets where most of the domain members were located. operational procedures. and support methods were tested. Summary This chapter provided an introduction to logical isolation and explained how server and domain isolation can be used to create an enterprise level solution using IPsec and Group Policy. Microsoft strongly recommends testing each isolation scenario in a lab environment first to avoid impact to production environments. one of Woodgrove's smaller domains was chosen to pilot domain isolation. processes. Note Because isolation changes many aspects of computer networking. . the project team had all the information it needed to create and implement a complete design for the entire organization. After a successful lab implementation project.4. a group of servers were identified to implement a basic server isolation scenario. Impact was further reduced by allowing non-IPsec communication when these domain members communicated with other computers not involved in the pilot. IT administrators should review chapters 6 and 7 for supportability issues. this chapter provided an executive overview and a brief description of each chapter within this guide.14 Server and Domain Isolation Using IPsec and Group Policy The examples provided throughout this guide are based on the findings of the pilot design infrastructure illustrated in Figure 1. Organizational management roles.

and use of Group Policy). standard name resolution methods and concepts such as Domain Name System (DNS) and Windows Internet Naming Service (WINS). The purpose of this chapter is to organize the layer of security that IPsec provides with respect to other layers and explain how it is used with Group Policy in a solution that will achieve isolation in a manageable and scalable manner in an enterprise-class environment. and access control lists (ACL). and many more. information technology (IT) professionals have struggled to provide resilient. a successful implementation will be far more likely if you meet all these prerequisites. mutual authentication concepts. network switches. you should be completely familiar with the following concepts and technologies. or isolate one or more hosts or networks from other hosts or networks. Many different technologies have been introduced to work with TCP/IP to address the issue of implementing security at the network and transport layers. These technologies include IPv6. 802. The unintentional result of the introduction of these technologies is a multi-layered approach to network security. security concepts such as users. Although you might benefit from this guidance without meeting these prerequisites. groups. and using Group Policy or command-line tools to apply security templates. the use of security templates. groups. auditing. Microsoft Windows® system security. Chapter Prerequisites Before using the information provided within this chapter. Authentication concepts including use of the Kerberos version 5 protocol and public key infrastructure (PKI). manipulating users. Internet Protocol security (IPsec). virtual LAN (VLAN) segmentation.1X.Chapter 2: Understanding Ser ver and Domain Isolation In the time since local area networks (LANs) became prevalent. These layers can be used to separate. Knowledge Prerequisites Familiarity with Microsoft® Windows Server™ 2003 is required in the following areas: • Active Directory® directory service concepts (including Active Directory structure and tools. segment. • • . standard Windows diagnosis tools and troubleshooting concepts. and other Active Directory objects. highly available services while maintaining adequate security.

Who Should Read This Chapter This chapter is designed for technical decision makers and technical architects who will be responsible for designing a customized server and domain isolation solution for an organization. and other tradeoffs. and operations personnel DNS." of the Windows Server 2003 Deployment Kit discusses certain scenarios for IPsec transport mode that were not recommended at the time. including subnet layout. impact. or fewer people may span several roles. However. Address Resolution Protocol (ARP). You should consult with other people in your organization who may need to be involved in the isolation planning. such as with the support staff or the users who will be affected during the deployment. Also. Two leading causes of failure in complex projects are poor planning and poor communications. protocols. Web server.• Knowledge of TCP/IP concepts. Organizational Prerequisites Planning the security for an organization is unlikely to be the responsibility of a single individual. user impact. means that the recommendation can now be changed. and the overall project process. the work that Microsoft has done for its own internal deployment of IPsec. including those people who perform the following roles: • • • • • • Business sponsors User group representatives Security and audit personnel Risk management group Active Directory engineering. knowledge of low-level functionality. network masking. The scope of a server and domain isolation project requires a comprehensive team to understand the business requirements. The information that is necessary to determine the exact requirements for an organization will often come from a number of sources within the organization. Each customer must evaluate the benefits of deploying IPsec in domain or server isolation scenarios against the costs. and Maximum Transmission Unit (MTU). all types of unicast IP traffic should be able to be secured with IPsec. Knowledge of security risk management principles. and network engineering. along with the availability of additional guidance. . and routing. It is often beneficial to have a high-profile individual who can act as the primary point of contact for this project when wider input is required. these roles may be filled by several different people. The project team must understand these potential risks and ensure that steps are taken to mitigate them. administration and operations personnel Note Depending on the structure of your IT organization. and terms such as Internet Control Message Protocol (ICMP). administration. technical issues. • Note Chapter 6. "Deploying IPsec. While multicast and broadcast traffic still cannot use IPsec. Technical understanding of both the technologies involved and the organization's current infrastructure is required to obtain the greatest benefit from this chapter. However. Microsoft now recommends and supports the wider use of IPsec on customer networks in accordance with this guidance.

Although HIPAA does not mandate or recommend specific technology. The additional security services being provided for network traffic may also require additional server memory or hardware acceleration network cards in some cases. by preventing alteration of ePHI during network transmission by providing integrity and authenticity for all network packets in application connections. other solutions may be available to accomplish the same or similar isolation goals. integrity.312(b) by auditing which computers communicate with one another. Audit controls 164.Chapter 2: Understanding Server and Domain Isolation 17 Business Requirements It is important to understand that the business requirements of your organization should drive the solution. Also.312(c)(1) by restricting inbound network access to computers that have ePHI to only a specific group of authorized and trusted computers and users. This solution will require time for a project team to plan and investigate feasibility and will also require training of IT support staff and the provision of. Also. Integrity 164.312(d) by requiring authentication and authorization of trusted computers for inbound network access to other trusted computers. Security restrictions will always have an impact on the day-to-day operations of employees within an organization. You should evaluate the use of domain or server isolation with IPsec protection as a technical safeguard to help address requirements of the following HIPAA sections: • Access control 164. Controlling access to customer and employee information is no longer just good business practice. and to use encryption to protect EPHI from disclosure in network traffic. at least. Isolation is defined as a logical or physical separation of one or more computers from network communication with other computers. For example. the focus on data privacy has also increased. Transmission security 164. a minimal employee awareness program. Therefore. Ensuring Regulatory Compliance As more personal information is stored on computers. it is important to assess the monetary value that the solution is intended to deliver to the business. an organization that fails to protect confidential information can be open to significant financial and legal liability. Person or entity authentication 164.312(a)(1) by protecting inbound network access to trusted computers using Group Policy authorizations. Depending on the local laws for the countries in which it operates. it does specify what capabilities are required for compliance and how to mitigate risks to ePHI. • • • • . The changes introduced as part of the solution will alter the way that computers in the domain communicate with one another and with untrusted computers.312(e)(1) by providing authenticity. organizations that operate in the United States may need to meet the requirements of one or more of the following regulations: • • • • Federal Information Security Management Act (FISMA) Sarbanes-Oxley Public Company Accounting Reform and Investor Protection Act Gramm-Leach-Bliley Financial Services Modernization Act (GLBA) The Health Insurance Portability and Accountability Act (HIPAA) HIPAA has a security rule that specifies strict guidelines about how healthcare organizations must handle electronic personal healthcare information (ePHI). and encryption.

2003. Compliance of This Solution with U. Office of Management and Budget (OMB) released a memorandum on the subject of "E-Authentication Guidance for Federal Agencies. access controls." identifies the technical requirements of authentication levels 1-4.NET technology with SSL/TLS to help meet HIPAA security regulations. NIST Special Publication 800-63. related regulation is emerging all over the world. server and domain isolation solutions add an initial layer of trusted computer authentication. application communications must properly integrate SSL/TLS usage and algorithm controls. and related IT environments. both of which impose strict guidelines regarding identity management and data privacy. government.3 Systematic Flaw Remediation. For more information. On the Windows platform. strong levels (3 and 4) of user authentication require applications to be rewritten or replaced. Windows 2000. you can meet these requirements by using Secure Sockets Layer (SSL) and Transport Layer Security (TLS). In many cases. "Electronic Authentication Guideline: Recommendations of the National Institute of Standards and Technology. The main advantages of an IPsec isolation solution are that it protects all applications as well as the host computer operating system and can provide network traffic security for existing applications without changing them. the U. For more details. Thus. and Windows Server 2003 IPsec cryptographic components have been certified to meet FIPS 140-1 cryptographic requirements. as demonstrated by statutes such as the European Union Data Protection Directive of 1998 and Canada's Personal Information Protection and Electronic Documents Act (PIPEDA).S Government Regulations On December 16. see the following links: • • • National Information Assurance Acquisition Partnership Overview: Windows 2000 Common Criteria Certification FIPS 140 Evaluation The information that this section provides is specific to organizations operating in the United States. Windows 2000. . For example. and encryption prior to user authentication at the application layer. This memorandum specifies that the level of risk of an authentication compromise corresponds to the level at which electronic authentication (eauthentication) is required. Thus. However." which is available as a PDF file format. network traffic authentication. Windows XP. applications can use Microsoft . See the white paper "Healthcare Without Boundaries: Integration Technology for the New Healthcare Economy". To enable compliance with government regulations for information assurance products. you can use a less costly level of user authentication for access to highly sensitive information. using a server and domain isolation solution might reduce or delay the requirement for application changes and help comply with risk management mandates. server and domain isolation solutions can be used in military. If the overall security risks can be reduced. and Windows Server 2003 with SP1 have been certified to meet the Common Criteria for IT Security Evaluation (ISO Standard 15408) evaluation assurance level 4 (EAL4) augmented with ALC_FLR. Microsoft is committed to several certification processes.S. This certification applies to both the operating system and sensitive data protection categories.18 Server and Domain Isolation Using IPsec and Group Policy Frequently. Also. However. see the "Comparison of SSL/TLS and IPsec" section later in this chapter. Windows XP with SP2.

add together the individual cost of each of the following: • • • • • • Loss of intellectual property required to develop information Loss of future revenue across all products due to customer mistrust. Refer to your local government agencies or legal counsel for specific legislative issues that pertain to your organization. add together the individual costs of each of the following: • Cost of legal action if the attacker can be identified but your company loses the court decision . To determine the total cost incurred by the requirement for subsequent legal action. add together the individual cost of each of the following: • • Internal effort required to respond to the attack and replace the server Internal mitigation of attacks on other computers that were made possible by the compromise of administrative credentials on the server • Cost incurred by subsequent legal or regulatory action . The cost estimates are extremely important for evaluating different technical solutions for each problem. employees. the business impact and costs of a successful hacker attack against their high-value data are of more concern.Chapter 2: Understanding Server and Domain Isolation 19 Business Risk Assessments of IT Infrastructure Business risk assessments should identify how the business is dependent on the IT infrastructure. if the theft is publicized Internal response time required by marketing and development Loss of revenue opportunity due to internal response effort Time required to mitigate the malicious use of information against business. The security risk assessment should provide clear justification for why these risks must be addressed and should estimate the costs associated with not addressing the risks. Consider the following categories as a guide for estimating the total cost of a security incident: • Cost incurred by loss of service. An IT security risk assessment should identify and prioritize risks to the integrity of information and stability of services. compare each solution against the others and their associated costs. Because no single solution will address 100 percent of the risk. To determine the total cost incurred by the theft of information from an internal network server. laws require that all security breaches be reported to customers that may be affected. add together the individual cost of each of the following: • • • • Incident response time required by support personnel Lost revenue due to application service interruption Lost internal productivity Cost incurred by theft of information. Decision makers may want to evaluate the cost of an isolation solution in terms of how it will reduce the risk of degraded or lost service due to network propagation of virus and worm infections. To determine the total cost incurred by the loss of service on a network server. To determine the total cost incurred by the compromise of administrative credentials on an internal network server. Note In some countries and states. For some organizations. if the theft is publicized Loss in market value due to investor mistrust. or customers by outsiders • Cost incurred by compromise of administrative credentials on the server .

isolation is the ability for any particular trusted host to decide who can have network-level access to it. How frequently these updates are applied will depend on the length of time your organization needs to test and deploy each update. wireless. For more information about virtual private networking (VPN) and quarantine access controls.20 Server and Domain Isolation Using IPsec and Group Policy • • Cost of legal action if the attacker can be identified and your company wins the court decision. In general. At its core. In future releases of Windows.1x. or misused user credentials. . VPN. the organization has significant defenses against infection and other attacks from untrusted computers and from compromised user credentials. or any combination. business-related. 802. restrictions. Microsoft plans to deliver more manageable and comprehensive network access protection. security-focused. you should apply updates as soon as is possible for your environment. For more information. Untrusted computers are computers that cannot be assured of meeting these security requirements. the remote computer must use IPsec to negotiate trust and to secure TCP/IP traffic end-to-end with the destination computer.11 WEP). Ideally. LAN. In the context of this solution. For more information about the NAP initiative. and other good faith efforts to reestablish current business environment Invest in Long-Term Directions for Information Security The Microsoft Network Access Protection (NAP) initiative establishes the long term direction to firmly control policy compliance of devices connecting to the network and to one another. By combining both perimeter and internal isolation capabilities. see the "Introduction to Network Access Protection" white paper. you should manage and enforce these updates by using a patch management system such as the Windows Update Service or Microsoft Systems Management Server (SMS). audits. Microsoft recommends that the specification for trusted computers includes a regularly updated list of security updates and service packs that are required. for optimal security. These requirements also dictate the state that a computer should be in before establishing communications with other computers. Internet). compromised. These requirements can be technical in nature. This end-to-end security model provides a level of protection of network communications that other link-based network access control and security technologies cannot (for example. Thus. Identifying Trusted Computers A discussion of trust and how it relates to computers is an important part of the topic of server and domain isolation. There is significant value in trusting the remote computer first to protect against stolen. see the Virtual Private Networks Web site. However. trust is the ability for an organization to be reasonably assured that a particular computer is in a known state and that it meets the minimum security requirements that the organization has agreed on. regardless of how the remote computers are connected at the remote end of the connection (for example. Remote access quarantine and server and domain isolation are two parts of this direction that can be implemented now with current Windows 2000 and later platforms. see the Network Access Protection Web site. 802. a computer is considered untrusted if it is either unmanaged or unsecured. but the defendant cannot pay the court-awarded damages Cost of regulatory fines.

control access at the network layer by using permit or deny permissions and ACLs for specific users and computers within the trusted environment. the business can reduce the overall risk to its data assets. "Determining the Current State of Your IT Infrastructure. and processes that will safeguard the organization's assets. Additional business benefits may include: • • • A high level of understanding of data flow across specific areas of the network. Generally. which include all computers running Windows 2000 or later that use Group Policy to provide security settings. the network security features of packet encryption and strong authentication and authorization. a computer that does not provide the required security management capabilities is also considered unmanaged. You should allow trusted resources. in the Woodgrove Bank scenario. The solution ensures that: • • Only those computers that are considered trusted (those that meet specific security requirements) can access trusted resources. For example. support for central management of security settings. In addition. These features include access controls (for example. Such operating systems include Windows 9x. or Windows Server 2003 or later in any of the domains that Woodgrove owns and manages. . the features required for the security infrastructure are found in more modern operating systems. In addition.Chapter 2: Understanding Server and Domain Isolation 21 The purpose of the server and domain isolation solution is to mitigate the risk posed to trusted resources by implementing tools. Unsecured computers may fall into one of the following four groups: • Low operating system security rating. Unmanaged Computers An unmanaged computer is one whose security settings the IT department does not centrally control. Microsoft Windows NT®. For more information about which computers are considered trusted within the context of the solution. and support for other security technologies (such as the Kerberos authentication protocol and certificate services). support for ensuring confidentiality and integrity of data. technologies. trusted computers include all computers running Windows 2000 Service Pack (SP) 4." in this guide. IT also examines trusted assets to ensure that the installation and configuration of specialized security software (such as antivirus software) is centrally controlled according to Woodgrove's own security requirements. see the "Trust Determination" section of Chapter 3. Improved adoption of security programs that are used to obtain "trusted" status. Windows XP SP2 or higher. by default. file permissions). and Windows CE. Computers that are untrusted are denied access to trusted resources unless a specific business reason is identified to justify the risk. Unmanaged computers are considered untrusted because the organization cannot be assured that they will meet the security requirements of the trusted computers they seek to access. network-level access only from other trusted resources. trusted assets. Unsecured Computers Untrusted computers also include those computers that use an operating system that has not or cannot be configured to the required level of security. such as Windows XP and Windows Server 2003. Creation of an up-to-date host and network device inventory. In addition. are periodically examined by IT staff to ensure that they continue to meet the minimum requirements. This grouping applies to computers running an operating system that lacks the required security infrastructure rating. By creating this trusted environment and restricting the permitted communications inside and outside of this environment. differing levels of privilege (user and administrative).

It is possible that a trusted computer could be compromised. With current platforms. during. you cannot trust that computer. Goals Directly Achievable Using Server and Domain Isolation Overall. User authentication is involved only after successful computer authentication has been achieved and IPsec security associations protect all upper layer protocol and application connections between the two computers. integrity. you can achieve the following goals by using server and domain isolation: • • • • Isolate trusted domain member computers from untrusted devices at the network level. An organization might identify a minimum level of updates that a host must have to be considered trusted. Trusted computers that might have been compromised.22 Server and Domain Isolation Using IPsec and Group Policy • Incorrectly configured. Lastly. and to weaknesses in user credential protection. most software vendors will issue software updates to ensure that the latest vulnerabilities are properly addressed. Even the most secure operating systems can be configured in a manner that will leave them open to an attack. it is no longer considered trusted until some remediation effort has been performed on the computer to return it to a trusted state. Lacking the required level of updates. Focus network attack risks on a smaller number of hosts. Ensure that inbound network access to a trusted domain member on the internal network requires the use of another trusted domain member. the goal of server and domain isolation is to mitigate the threat posed by unauthorized access to a trusted computer by an untrusted computer. In such a case. antireplay and encryption. Allow trusted domain members to restrict inbound network access to a specific group of domain member computers. Such computers must be considered to be unsecured devices. monitoring. usually by a trusted attacker. the ability to isolate a remote computer by restricting inbound network access is based on the ability to successfully authenticate as a domain member computer using the IPsec Internet Key Exchange (IKE) security negotiation protocol. Improve security by adding strong per-packet mutual authentication. and after an attack. Because IT security is a constantly evolving area. without the need to change applications and upper layer protocols (such as server message block [SMB] or NetBT). It is important to understand that if you cannot trust the user of a computer. Thus. where maximum risk mitigation strategies (such as logging. Server and domain isolation makes hosts less vulnerable to weaknesses and mistakes in the other types of network-based security. server and domain isolation solutions . those computers that lack the updates would be considered unsecured devices. • • Those devices that fall into one of these four groups are categorized as untrusted because the organization cannot be certain that they have not been compromised in some manner. When a trusted computer has been compromised. Focus and accelerate remediation and recovery efforts before. and intrusion detection) can be applied more effectively. • • • Server and domain isolation seeks to protect all network services on the trusted host from untrusted network access and attacks. which provides a boundary to the trusted domain. Therefore. Focus and prioritize proactive monitoring and compliance efforts prior to an attack. they represent a significant risk to the trusted computers that they attempt to access.

Examples of serious security risks that will not be directly mitigated by this solution include: • The risk of trusted users stealing or disclosing sensitive data . Failure to address these security risks with additional security processes and technology could ultimately negate the security benefits of the isolation solution. An attacker may also attack trusted hosts by using compromised credentials from hosts that are exempted from using IPsec with trusted hosts (for example. For example. However. domain administrators. The risk of untrusted computers attacking certain trusted computers . For example. These trusted. The risk of compromise of trusted user credentials . Although the isolation solution can control where computers communicate within the internal network. • • • • . Also. It is not possible for this solution to eliminate the risk of trusted users inappropriately copying or disseminating sensitive data. as a practical deployment matter. Physical access to a trusted host computer can enable an attacker to gain unauthorized and administrative access to it. but non-IPsec enabled. it is vital to limit the default scope of access and the number of administrators (including enterprise administrators. This solution cannot mitigate the risk of untrusted computers being used by an attacker to attack other untrusted computers. Legitimate users who abuse their access also fall into this category. Although an administrator can choose to encrypt most traffic with IPsec to protect network logon information. or hosts that accept inbound connections from untrusted computers. only support the user identity. administrative users can subvert these controls. An attacker who gains control of an exempted or boundary host can then attack all other trusted hosts inside the isolation domain. domain controllers and DNS servers). domain controllers). Risks Addressed Using Server and Domain Isolation The number one risk that server and domain isolation addresses is the risk posed by unauthorized access to a trusted computer by an untrusted computer. thus ensuring end-to-end protection for that authorized communication. a server and domain isolation solution can authorize inbound access for specific trusted host computers based on host and user domain identity. Most network-based security technologies. IPsec protection of user logon traffic to domain controllers is not supported. and local administrators on workstations or member servers). Rogue users. Although an administrator can control whether trusted hosts communicate outbound to untrusted hosts. this solution cannot mitigate the risk of a disgruntled employee deciding to steal information using trusted hosts to which they have access because of their job role. Because administrators can disable server and domain isolation protection. Certain security risks cannot be mitigated fully using the solution alone. however. computers are members of an exemption list (for example. this solution cannot mitigate the risk of trusted users losing their credentials to an attacker who tricks them to reveal their passwords. this solution identifies certain trusted hosts to be accessed by untrusted computers to provide boundary services for the isolation domain. The risk of untrusted computers accessing other untrusted computers .Chapter 2: Understanding Server and Domain Isolation 23 address similar network threats but provide different levels of network access control and traffic protection than other types of network-based security technology. Server and domain isolation solutions are designed to protect trusted hosts. Server and domain isolation can force an attacker to use a trusted host to attack other trusted hosts. this solution identifies trusted domain members that for various reasons do not use IPsec to negotiate trusted access to other trusted hosts.

and assets that demand protection. it is necessary to first understand the infection sources and adversaries who stand ready to attack an organization. An organization that follows the industry best practice of protect-detectreact realizes that attacks will occur and understands that detecting them quickly and minimizing service interruptions or loss of data is paramount. For example. confidentiality. as well as to have some idea what their targets might be. This solution depends only upon successful IPsec IKE domain-based (Kerberos) authentication to establish trust and thus IPsec-protected connectivity. it is helpful to review the concept of defense-in-depth and the high-level ideas behind it. Isolation involves all three of these areas: it is achieved through a thorough understanding of the risks. the recommended security hardening configurations and templates were applied to all systems in the Woodgrove lab environment. To successfully layer defenses. if perimeter security failed due to a misconfiguration or a malicious employee. For example. office? Defense in Depth Defense in depth is best described as a layered approach to protecting a computer instead of reliance on a single mechanism for that protection. the . Generally speaking.S. processes. isolation requires knowledge of the current state of the network and its devices. it is more cost-effective to defend against an attack than it is to clean up after one has occurred. All information assurance mechanisms focus on people. requirements. Practicing protect-detectreact makes it possible to recognize that because the probability of an attack is high. authorization. Over time trusted hosts may for various reasons not meet the full criteria of being a trusted host yet still be able to authenticate successfully as a domain member. an understanding that encompasses the people and process elements. and non-repudiation. How Does Server and Domain Isolation Fit into My Overall Network Security Strategy? Server and domain isolation is employed in addition to other proactive and reactive mechanisms to defend the network and its connected devices. A comprehensive network security strategy applies the appropriate technologies to mitigate the highest priority risks without significant dependence on single points of failure. It is the responsibility of the organization's IT management systems and processes to ensure that domain members comply with the definition of trusted hosts. more effort should be spent on protecting data and assets rather than on preventing an attack from occurring. what other layer of defense would stop network attacks and infections on internal trusted hosts? What stops attacks on all trusted hosts in Europe or Asia when the attacker connects to the Ethernet port in a conference room in any U.24 Server and Domain Isolation Using IPsec and Group Policy • Assuring security compliance of trusted hosts. and technologies. These methods include authentication. In addition. This solution suggests how trusted hosts might be defined and in particular requires that they be members of a Windows 2000 or Windows Server 2003 domain. For more information about Windows platform security technologies and management procedures. After gaining an understanding of the attackers and their possible targets. it is then necessary to apply incident response procedures for computers that could be compromised. see the TechNet Security Center. an adversary could be a competitor who hires a corporate espionage firm to steal information about a new product or service that is under development. including computers. To address these issues. Because security is a multi-faceted problem that requires multiple layers.

IPsec can secure all packets between the two computers. a "logical isolation" solution such as server and domain isolation: • • Does not secure network devices. For information and practical design examples for this process. and the security requirements that may limit those requirements to achieve the appropriate balance between security and communication. such as specifying which computer is allowed to establish a remote access VPN connection. The following figure illustrates how a logical isolation solution would fit into the defensein-depth approach used in the Windows Server System Reference Architecture: Figure 2. As defined within the context of this solution. or provide protections supplied by network-based firewalls. instead of the host firewall providing permit and block services for ports. A more detailed discussion of this subject can be found in the U. However. such as 802. such as routers. After access is granted. see the Enterprise Design chapter of the Security Architecture Blueprint within the Windows Server System Reference Architecture.11 WEP encryption for wireless links. Does not provide physical network access control. National Security Agency's “Defense in Depth” white paper in PDF format.S. However.Chapter 2: Understanding Server and Domain Isolation 25 communication requirements that define how computers should interact with one another. • .1 Defense in depth with logical isolation One important point to understand from this figure is that the logical isolation layer of security is aimed directly at securing the host computer through controlling network communications.1x for access control and 802. This role is most similar to that of a host-based firewall. IPsec does provide protection end-to-end across all network links in the path between source and destination Internet Protocol (IP) address. Does not secure network links. IPsec provides permit and block services and negotiates trusted network access services.

Does not secure application level paths. Because IKE is not integrated with applications. which Microsoft Knowledge Base article 245030.NET messaging flows. Windows XP. and 1024-bit ephemeral Diffie-Hellman). and Windows Server 2003 provide registry key controls for SSL/TLS algorithms.26 Server and Domain Isolation Using IPsec and Group Policy • • Does not provide security for all hosts on the network—only those participating in the isolation solution. • • IPsec secures traffic between source and destination IP addresses. Help prevent user credentials from being entered into untrusted computers—because users are not prompted to log on to an internal SSL/TLS Web site until IPsec establishes mutual trust between client and server. such as the end-to-end path through which e-mail and . Establish a defense-in-depth approach against potential improper or non compliant use of SSL/TLS (for example. Using IPsec in environments in which applications use SSL/TLS can provide the following benefits: • • • Help protect all applications and the operating system against network attacks by untrusted computers and other devices. The solution presented in this guide uses Kerberos protocol signatures for IKE authentication between domain members instead of certificate-based signatures." describes. IPsec with certificate authentication provides protection that is similar to SSL/TLS. Provide security when you cannot use Windows registry settings to select regulatory compliant SSL/TLS algorithms. but there are some differences. The National Institute for Standards and Technology (NIST) is developing guidance on using TLS. Any users—even an administrator—will be denied access to the server. such as SSL/TLS. SHA-1. "Guidelines on the Selection and Use of Transport Layer Security. These guides are available from the NSA Security Recommendation Guides download site. it does not matter which user logs on at that computer. However. Special Publication 800-52. Similar guidelines do not exist for the use of IPsec. For more information about this guidance. Windows IPsec supports a small subset of the cryptographic algorithms supported by TLS and recommended by NIST 800-52 (for example. SSL/TLS can secure traffic through the entire application path (for example. One of the advantages of using IPsec is that it can provide network traffic security for existing applications without having to change them. and Hypertext Transmission Protocol (HTTP) requests that can be proxied several times between the client and the Web server destination.S. Organizations that need to meet NSA guidelines should evaluate the design of this solution in addition to the NSA guides. it cannot verify that the destination computer name is the one to which the application is expected to connect. "How to Restrict the Use of Certain Cryptographic Algorithms and Protocols in Schannel. For example. National Security Agency has released guides for using Windows 2000 IPsec. You should consider security at each layer as part of IT security risk mitigation analysis. Comparison of SSL/TLS and IPsec IPsec is not intended as a replacement for application level security. Windows IPsec IKE negotiation establishes mutual trust between computers using the Kerberos protocol and certificate-based authentication. see the NIST Computer Security Division Web site. Windows 2000. 3DES. . Provide security where certificates are not available. from a Web browser through a Web proxy to a Web server)." is a guideline for implementing TLS in the federal government. the U.dll. if a certain computer is not allowed access to a server at the logical isolation layer. if all eHPI data is not encrypted and authenticated).

a group of Exchange servers require a client or server computer to be a trusted domain member in order to make an inbound TCP/IP connection. by using Group Policy security settings for network logon rights. Isolation group. Isolation Terms The following terms are unique to the concept of logical isolation. If you are already familiar with these terms.Chapter 2: Understanding Server and Domain Isolation 27 thereby allowing a sophisticated man-in-the-middle attack from another trusted host. and Network Access Protection (NAP) to isolate computers at the network layer. you may find that some of the explanations in this guide will be confusing. A computer need only present valid credentials and successfully authenticate with IKE to be isolated. If the domain has bidirectional trusted domains. it would be a good idea to review a number of the terms that are often used in the context of this solution. essentially the same inbound and outbound network traffic requirements. Server isolation. the destination name is not only authenticated (trusted). which is not allowed to make outbound TCP/IP connections to non-domain members (with certain exceptions) would be called an Exchange Server isolation group. Terminology Refresher Before continuing with this chapter. These are specifically created to enforce the inbound access requirements for isolation groups. Isolation domain. there may be an Allow network access group (ANAG) and a Deny network access group (DNAG). A logical separation of one or more computers from other computers. As an isolation group. However. network services. members of those domains would be part of the isolation domain. A server that is in a server isolation group might have clients that are part of the isolated domain. But because the application integrates with SSL/TLS. if you do not understand these terms adequately. This term defines the type of isolation that separates trusted computers from untrusted computers. The domain is any domain in the trust path that is accessible via a two-way trust between the two hosts that are attempting to secure their communications. This term refers to the Windows domain security group that is used to control network access to a computer. This group. and protocols should have a specified configuration to meet traffic requirements at their layer. For example. you can skip this section. or with IPsec negotiating security in combination with Group Policy network logon rights (and potentially with other network configuration or connection settings). Logical isolation. This term defines how servers can restrict inbound access using IPsec and the "Access This Computer From the Network" right to a specific group of trusted computers. The broader term for isolation technologies that can include Domain Isolation. Network access group. the inbound and outbound requirements are simple: Inbound connections can only be from other trusted host domain members. Ensure that you understand these terms fully before continuing with this chapter: • • Isolation. Individual applications. You can implement an isolation group's inbound and outbound access controls by IPsec policy alone using permit/block actions. • • • • • . An isolation group where the membership in the group is the same as the membership of the Windows domain. Server Isolation. For each isolation group. Domain isolation. the name can also be verified against the name that was expected. A logical grouping of trusted host computers that share the same communications security policy.

a secret configured administratively on two devices (preshared key). There is little or no assurance that the host (or the user of the host) will not be the source of an infection or malicious act if it is connected to the network. Authentication requires that the person. or device requesting access. A host computer that is not a trusted host. process. Authorization is dependent on the identity of the person. services. computer process. or functionality. Trust also means that a user or computer is considered to be an acceptable and presumably low risk with which to communicate. Trusted hosts are typically defined to meet certain additional management and security requirements. Domain trust implies that all members of the domain trust the domain controller to establish identity and properly provide group membership information for that identity. The latter may or may not appear in the exemption list. "Determining the Current State of Your IT Infrastructure. because they represent a higher risk to the rest of the trusted hosts. This computer has an unknown or unmanaged configuration. computer. Trust is necessary to communicate with a remote computer by using IPsec. The process of granting a person. The configuration of the host is controlled such that the host's security risks are considered low and managed. Common forms of credentials are private keys for digital certificates. And there are computers that are exempt from using IPsec policy to negotiate secured connections. Exemption. Trustworthy host. a secret password for user or computer domain logon. This term is used to define the fact that a computer is willing to accept the identity validated through the authentication process. This term refers to a platform computer running Windows 2000. • . • • • • • Security Terms Ensure that you thoroughly understand the following security-related terms: • Authorization. Knowing which computers are trustworthy is important when planning the membership of any trusted isolation group. A computer. Boundary host. process. A trusted host that is exposed to network traffic from both trusted and untrusted hosts and therefore must have closer monitoring and stronger defenses against attacks than other trusted hosts. or device. or a biological object such as a person's fingerprint or retina scan. There are computers using a static IP address whose addresses are included in the IPsec policy "exemption list" so that trusted hosts do not use IPsec with these computers. or device making the request provide a representation of credentials that proves it is what or who it says it is." of this guide for detailed discussion of this topic. either domain joined or not. Authentication. Trusted host. which is verified through authentication. Trusted hosts are less likely to be the source of an infection or malicious act. but does not use IPsec protected communication with other trusted hosts in the isolation domain. computer. There should be as few boundary hosts as possible. or Windows Server 2003 that is at least a member of a Windows 2000 security domain and is capable of enforcing an IPsec policy. An exemption may meet the requirements of a trusted host. Windows XP. See Chapter 3. This term is used to identify a computer that is capable of being configured to meet the organization's minimum security requirements but may or may not currently be a trusted host. or device access to certain information. which does not use IPsec. The process of validating the credentials of a person. There are two types of exemptions.28 Server and Domain Isolation Using IPsec and Group Policy • Trust. Untrusted host. process.

A security rule contains packet filters that are associated with a permit. including authentication and digital signing. Use the Group Policy Object Editor to create an individual GPO and the Group Policy Management Console to manage GPOs across an enterprise. spoofing is taken to mean the act of impersonating a legitimate IP address by an attacker in an attempt to disrupt communication or intercept data. Plaintext (sometimes called cleartext). By associating a GPO with selected Active Directory system containers—sites. A computer that is starting a network communication with another computer. A computer that replies to a request to communicate over the network. or negotiate action.Chapter 2: Understanding Server and Domain Isolation 29 • Spoofing. If there is a change in the input data. Data that has been encrypted. • • Basic IPsec Terms Ensure that you thoroughly understand the following IPsec terms: • IPsec policy. Hash. The connection path that is established for network traffic that passes between an initiator and a responder. Nonrepudiation provides sufficiently undeniable proof that a user or device took a specific action such as transferring money. The hash can be used in many operations. Communications and data in their unencrypted form. or sending a message. When negotiation is required. block. A group of security rules for processing network traffic at the IP layer. and organizational units (OUs)—you can apply the GPO's policy settings to the users and computers in those Active Directory containers. authorizing a purchase. Domain policy. • • • • Network Terms The following terms refer to the network elements of this solution: • • • Initiator. Communication path. Ciphertext. Responder. Local policy. A fixed-size result that is obtained by applying a one-way mathematical function (sometimes called a hash algorithm) to an arbitrary amount of input data. The Group Policy settings that you create are contained in a Group Policy object (GPO). . domains. A technique used to ensure that someone performing an action on a computer cannot falsely deny that they performed that action. Hash functions are chosen so that there is an extremely low likelihood of two inputs producing the same output hash value. the hash changes. Ciphertext is the output of the encryption process and can be transformed back into a readable form plaintext by using the appropriate decryption key. Group Policy Terms The following terms relate to Windows Group Policy: • GPO. Nonrepudiation. In this guide. Also called a message digest. A policy that is stored on an individual computer. A policy that is stored centrally in Active Directory. the IPsec policy contains authentication and security methods to negotiate with the peer computer.

• • • Fall back to clear. Quick mode SA. VLANs. Persistent policy is applied first on IPsec service startup. Technologies and procedures such as network segmentation. Appendix A. The agreements that two hosts make about how to communicate using IPsec and the various parameters that define this negotiation. network quarantine. the responder will not successfully communicate with a non-IPsec-aware initiator. and physically segmenting network traffic all provide a level of isolation. so it overrides settings in local or domain IPsec policy. Server and domain isolation complement all of these existing techniques and provides a new protection level for managed Windows domain members. . and network-based intrusion detection are achieved by implementing or changing the configuration of network devices. This is the Accept unsecured communication. The normal upper layer protocol response will be an outbound packet that then triggers an IKE initiation back to the remote computer. perimeter network access controls. "Overview of IPsec Policy Concepts. This is the process that occurs at the start of a network connection to determine whether a computer using IPsec will allow the connection to occur. This option allows an IKE initiator to allow normal TCP/IP traffic (not IKE or IPsec) if there is no IKE response from the responder. How Can Server and Domain Isolation Be Achieved? The concept of isolating computers from risk is not new. Security associations (SA). These SAs are negotiated after the main mode SA is established for each session of communication between the hosts. Additional terms are important to understand that refer to elements of IPsec specifically. A type of IPsec policy introduced in Windows XP and Windows Server 2003 that allows IPsec policy settings to be applied in a persistent manner. but not Fall back to clear on a responder.30 Server and Domain Isolation Using IPsec and Group Policy • Persistent IPsec policy. • Note If you have Inbound passthrough enabled. and with an existing Windows 2000 or Windows Server 2003 domain infrastructure. The solution presented in this guide is designed to work with the existing devices and techniques in your network infrastructure. • • Main mode SA. This is the Allow unsecured communication with non-IPSec-aware computer option on the filter action properties page of the IPsec Policy Management tool. These SAs are the first to be established during the IKE negotiation between the initiator and responder computers. with little or no change to applications. applying access control and filtering to routers. IKE negotiation. The key point is that isolation is implemented by making changes to existing host software in Windows 2000 and later platforms. This option allows an IPsec-enabled computer to accept a normal TCP/IP (not IKE or IPsec) inbound packet from a remote computer. Windows IT administrators can implement server and domain isolation with little or no change in the existing network paths and connection methods. but always respond using IPSec option on the filter action properties page of the IPsec Policy Management tool. Techniques such as using firewalls to provide segmentation. Inbound passthrough." of this guide provides an overview of these IPsec terms and explains the overall IPsec process that the computers in the isolation groups created in this solution will use.

you can achieve this trusted status only if the computer is running a secure and managed operating system. the next component of the solution is to confirm the computer's status through authentication. such as a credential from the Kerberos ticket.1X has not been developed to provide encryption for wired connections. but not for hard-wired Ethernet ports. .1X Protocol 802.11 wireless link or an 802. For more information about determining status. or perhaps a preshared key. This protocol requires one or more dedicated servers (for example. perhaps secured by various application-layer mechanisms.1X is designed to provide controls over network access and cryptographic keys for 802. Trusted Hosts Trusted hosts are computers that the IT organization can manage to meet minimum security requirements.1X could enforce that only trusted computers be granted access through wireless and some wired links. After the computer is determined to be trusted.Chapter 2: Understanding Server and Domain Isolation 31 Server and Domain Isolation Components The server and domain isolation solution is made up of a number of important components that collectively enable the solution. control of authorization. Woodgrove decided to use 802. Host Authentication The host authentication mechanism determines whether the computer attempting to initiate a session has a valid authentication credential. the Remote Authentication Dial-In User Service [RADIUS]) and a network infrastructure that supports the protocol. 802. it is forwarded in normal plaintext TCP/IP form to the end destination. This allows for control of the user experience (for purposes such as billing). After data is decrypted at the wireless access point. see Chapter 3.1X for wireless security.11 Wired Equivalent Privacy (WEP) encryption of wireless traffic between the client and wireless access point. The following subsections describe these components. Even when wireless connections are encrypted. a certificate.1X authentication. Thus.1X is a standards-based protocol for authenticating users and devices so that they can be authorized to gain connectivity through a link-layer network port. and current application and operating system updates. "Determining the Current State of Your IT Infrastructure. the data is not protected after packets are forwarded onto the internal LAN after decryption by the wireless access point. 802. and additional features.1X is not able to meet the end-to-end encryption requirement. antivirus software. A significant hardware purchase and installation cost would be required to upgrade every physical LAN access wall port and all buildings worldwide. 802. The following sections explain these two technologies. Monitoring technology will be required to determine the computer's baseline configuration and report any changes from that configuration. Currently there are two technologies that can provide this type of authentication mechanism on Windows-based computers. Woodgrove security requires protection of all trusted hosts from untrusted computers on the internal network." Note IPsec cannot make any determination of a computer's compliance to particular host criteria by itself. Consequently. only wireless. the switches used for most internal 802. Very often. The 802. such as an 802. After the device is granted network access.3 Ethernet ports is not capable of 802. Although 802. it typically has open connectivity to the rest of the internal network.3 Ethernet port. Another Woodgrove security requirement was to encrypt traffic end-to-end between trusted clients and the encryption servers.

1X to secure wireless networks and to provide access control for wired networks where possible.32 Server and Domain Isolation Using IPsec and Group Policy Although 802. It is useful in normal operations to allow trusted hosts to talk to untrusted computers and devices only when the trusted host initiates the connection request. IPsec uses the Internet Key Exchange (IKE) negotiation protocol to negotiate options between two hosts for how to communicate securely by using IPsec. IPsec does not support stateful filtering of outbound traffic either for traffic within an AH or ESP security association or for plaintext traffic involved with a soft SA. IPsec provides that ability to establish secure communications between hosts. The IKE negotiation establishes a main mode SA (also called the ISAKMP SA) and a pair of quick mode SAs (called IPsec SAs—one for inbound traffic.509 digital certificate with corresponding public and private Rivest. To avoid the need for a PKI. The agreements that two hosts make about how to communicate using IPsec and the various parameters that define this negotiation are called security associations or SAs. For more information about the use of 802.1X does not meet all of Woodgrove's security requirements. the packets no longer appear as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) . Microsoft recommends 802. Policy can be managed centrally in Active Directory using Group Policy objects for policy assignment to domain members. Stateful filtering for outbound connections can be added in addition to IPsec protection by using a host-based firewall product. when you use the IPsec policy designs presented here for server and domain isolation. Thus. it is still used for wireless security. the other for outbound traffic). But it is also a design that supports certain protocols that negotiate open ports to receive inbound connections. not exactly a password) The most common reason for not deploying IPsec is the misperception that it requires public key infrastructure (PKI) certificates. IKE requires mutual authentication to establish the main mode SA. IPsec uses the term soft security association to describe a communication that IPsec cannot protect with either Authentication Header (AH) or Encapsulating Security Payload (ESP) formats. policy-based IP layer security mechanism that is ideal for providing host-by-host authentication. When traffic flow ceases after an idle time (5 minutes by default). a new attempt to negotiate security is required when new outbound connections are made. This is a potential window of vulnerability to attack. the trusted host is able to receive inbound connections from the untrusted computer to any open port through the soft SA. This capability is referred to as Fall back to clear and is a practical necessity during rollout of IPsec. see Wireless Networking. It provides a general. Shamir. Windows IKE can use one of the following three methods: • • • The Kerberos version 5 authentication protocol X. An IPsec policy is defined to have security rules and settings that control the flow of IP traffic inbound and outbound on a host. Windows IKE negotiation can be configured to allow communication with a computer that does not respond to the IKE negotiation request. Network traffic that is protected by IPsec AH or ESP without encryption is not considered to be in plaintext form because it is does have authentication and protection against spoofing and modification. such as Windows Firewall. Microsoft integrated Windows 2000 domain authentication (Kerberos) in the IKE negotiation protocol. IPsec logs this communication with a Security Log success audit containing the destination IP address and monitors the traffic flow activity. which are often difficult to deploy. IPsec IPsec is the IETF standard security protocol for the Internet Protocol. & Adleman (RSA) key pair A preshared key (a passphrase. Because IPsec encapsulates normal IP packets in a secure format.1X.

and quality of service classification. The increasing use of Web services will likely accelerate traffic identification issues. It is possible. . This solution does not attempt to use IPsec only for specific applications or protocols because of the risk of network attacks on other network services. port 25 for e-mail traffic and port 80 for Web traffic). and it is becoming rapidly obsolete. Because the network management applications could still analyze the addresses of the packets.Chapter 2: Understanding Server and Domain Isolation 33 packets when traversing the network. but not certain types. Windows IPsec supports the ability to permit all multicast and broadcast traffic. some network management tool vendors were willing to modify their tools to inspect inside nonencrypted forms of IPsec packets. The attempt to deploy IPsec inside Woodgrove Bank identified that most network management tools assumed that applications can be readily identified by their TCP or UDP port numbers (for example. because RPC services pick random ports in the ephemeral range (above 1024) for various services. to run a Web server at an arbitrary port number and to document the port number in the URL of the site. even in the absence of IPsec. Woodgrove decided that the loss of some network management functionality was not significant enough to affect the plans for the project. the assumption about traffic visibility is not entirely accurate. security filtering. because the traffic for these Web services will run on a single port or a few ports instead of each application or service running on its own discrete port number. investigate with the vendor to determine whether or when a fix or upgrade will be available to address this issue. It is also possible to bypass the e-mail filtering efforts of some Internet service providers (ISPs) by running a mail service on an alternate port number. and the routers and network intrusion detection system cannot distinguish which application is being used or inspect the data in the packet. If applications do not work properly with IPsec. the traffic becomes opaque. Most host-based applications do not need to be modified to work properly with IPsec securing all traffic between IP addresses. defeating the purpose of isolation. It is not recommended to permit applications based on a well-known port because doing so opens a static inbound hole for attackers to make inbound connections to any open port. If the application cannot be upgraded or replaced with a compatible one. Applications that use dynamically allocated ports cannot be permitted outside of IPsec protection. Unfortunately. because it reduces the value of the tools currently used for traffic monitoring. All applications that move to a Web service architecture will appear as a single undifferentiated data stream to the routers. that is. Additionally. This factor creates a network management issue. and had visibility for all other traffic that was not IPsec-protected. Many peer-to-peer (P2P) applications are port agile. using either port 80 or port 443. computers that must use this application might not be able to participate in the isolation domain or group. the administrator may be able to choose to permit this traffic outside of IPsec protection based on the IP addresses used for servers. Applications that use multicast and broadcast IP addressing may have problems working with the IPsec design for server and domain isolation. they use port numbers picked at random in an attempt to avoid detection. Applications based on remote procedure call (RPC) services are also port agile. The use of SSL and TLS for HTTPS connections and RPC encryption decreases the value of network-based inspection as a defense against attacks. If there are application compatibility problems. When you use IPsec. Clients and servers then use the HTTP URL to identify the traffic. for example. because the traffic for these services runs on top of HTTP or HTTPS. The relationship between port numbers and applications is becoming increasingly weak. weighted fair queuing.

The IPsec filters specify source and destination IP address (or subnets). The original IP header is preserved. For these reasons. Issues with using IPsec transport mode include: • An initial delay. Predefined priority order for filters. The IPsec filters control what traffic is blocked. Windows Server 2003 Network Monitor is capable of parsing ESP that is not encrypted to expose the normal TCP/IP upper layer protocols. ESP/null allows for data authenticity. Using AH prevents replay attacks from occurring and provides strong integrity by confirming that each received packet was not modified from the time it was sent until it was successfully received. IPsec transport mode is the recommended way to secure traffic between end-to-end hosts. • . • Note ESP with encryption or using null encryption provides strong IPsec peering relationships with authentication. The IPsec driver simply inserts an IPsec header after the original IP header. Windows IPsec supports RFC 2410 that defines the use of ESP with null encryption (ESP/null). IPsec policy filters can overlap and so have a predefined priority order. or they can apply for all possible destination addresses and protocols. While continuous communication is happening. ESP is the logical choice because ESP does not force packets to be encrypted. This requires both sides in a communication to have a compatible set of IPsec transport mode filters for IKE negotiation. this solution uses a more general filter for "all traffic" that negotiates IPsec security. Thus. and the rest of the packet is secured in by either AH or ESP cryptographic processing.34 Server and Domain Isolation Using IPsec and Group Policy In addition to its authentication capabilities. It also provides replay protection and will properly traverse a NAT device. filters can apply very specifically to one computer. AH will not function correctly when passing through a device that performs network address translation (NAT) because NAT replaces the source address in the IP header. If internal networks are segmented with devices that are using NAT. Transport mode was designed to accommodate dynamic IP addresses by automatically updating filters configured with "My IP Address. There is a 1-2 second initial delay required for IKE to start and complete the full successful negotiation. IKE attempts to refresh cryptographic keys that protect traffic automatically. integrity. or encapsulated by IPsec. and source and destination port. therefore violating the security that IPsec AH provides. IPsec can use AHs to provide data and address integrity for each packet." It has less overhead and is overall much easier to use than IPsec tunnel mode. For example. If ESP encryption is used. permitted. monitoring of packets is possible only when the computer uses an IPsec hardware acceleration network card that decrypts the incoming packets first. There are two modes in which IPsec can operate—tunnel mode or transport mode: • IPsec transport mode. IPsec ensures data integrity plus confidentiality through encryption by using the Encapsulating Security Payload (ESP) option for the IP protocol. IPsec can provide two other useful services for host communication: ensuring address integrity and encrypting network traffic: • Ensuring address integrity. and anti-replay without requiring encryption. protocol (such as ICMP or TCP). it can be made to successfully traverse a NAT device using UDP encapsulation. most specific first. just like AH. Windows AH uses a keyed hashing mechanism in the Windows IPsec driver. IKE negotiating transport mode is an effective way to authorize inbound IPsec-protected connections. Although ESP does not provide address integrity (unless it is used with AH). Woodgrove decided not to implement AH and instead uses only ESP/null and ESP with encryption on its corporate network. Encrypting network traffic. in combination with a more specific filter to permit only ICMP traffic instead of securing that traffic with IPsec. Thus.

tunnel mode creates a new IP header with an IPsec header. To help differentiate the groups that were specifically created to achieve the host access permission from those of the standard share access permissions. This method creates a new layer of authorization for both the computer accounts and the user accounts at the network and application level by using permissions on the user rights assignments of the local policy on the host being accessed. Using both the "Access this computer from the Network" (ALLOW) and the "Deny access to this computer from the network" (DENY) user right assignments. Woodgrove did not have a scenario in which tunnel mode was required. because the fact that a device is capable of authenticating is no guarantee that it is also allowed to access a given host. The original packet with original IP header is encapsulated entirely to form a tunnel packet. This step is important. it is possible to restrict the ability of a computer and a user to access a resource even though they share common IPsec policy parameters and the logged-on user has the right to access the resource. and Windows Server 2003 have interfaces for network cards to be able to accelerate IPsec cryptographic operations in hardware. IPsec tunnel mode is typically used for gateway-to-gateway VPN tunnels between static IP addresses of VPN gateways. Thus. the host needs to determine whether to allow access to the source computer and user. . tunnel mode could be used to secure traffic from a static IP server to an IPsec-capable router. For more information about the technical details of both transport mode and tunnel mode in IPsec. This extra level of control is fundamental to the isolation approach that this solution describes. The group capabilities of Active Directory organize the computers and users in such a way as to allow the required authorization levels to be assigned in a manageable and scalable manner. Host Authorization After a host determines that the communication it is receiving is from a verifiable source.Chapter 2: Understanding Server and Domain Isolation 35 • Computational expense. For server and domain isolation scenarios. this guide uses the term network access groups. CPU utilization may peak at 80-100 percent during encrypted file copies. IPsec ESP transport mode encryption can be computationally expensive. Windows XP. • IPsec tunnel mode. The approach that Microsoft recommends is to use standard Windows groups to limit the users' and computers' abilities to access the resources on other computers in the design. This might be necessary if the destination host does not support IPsec. see Determining Your IPSec Needs. Windows 2000.

After the IKE main mode SA negotiation has completed. The outbound TCP connection request triggers an IKE negotiation to the server. During the authentication process. If the client computer has the required user right assignment.2 shows a five-step process. typically). IKE checks that the client computer has the required host access rights as assigned in the ALLOW or DENY users rights in the Group Policy. After the server receives the initial IKE communication request from the client computer. 3. The client IKE obtains a Kerberos ticket to authenticate to the server. IKE main mode negotiation.2 User and host authorization process Figure 2. . The client has IPsec policy assigned as part of the solution. 2.36 Server and Domain Isolation Using IPsec and Group Policy The following figure illustrates the main steps in the overall host and user authorization process of the solution. that is followed for all network communications after the isolation solution is in place. IPsec security method negotiation. User attempts to access a share on a departmental server. the server authenticates the Kerberos ticket. Figure 2. 1. as detailed in the following list. and an IPsec main mode SA will be established. the security methods of the IPsec policy are checked to negotiate a connection by using security methods for IPsec SAs that are acceptable to both hosts. A user that is logged on to the client computer attempts to access a share on a trusted host within the logical isolation solution. This action causes the client computer to attempt to connect to the trusted host using the file sharing protocol (Server Message Block protocol using TCP destination port 445. the IKE negotiation will complete.

3 Computer host access permissions-checking process .Chapter 2: Understanding Server and Domain Isolation 37 The following flowchart illustrates the complete process from steps 2 and 3: Figure 2.

The following scenario provides a practical example of how the steps in the logical isolation solution work: During a meeting. The . the files can contain malware that could infect internal computers. On the server. Finally. Because the contractor's computer is not a known or trusted host. Share and file access permissions checked. the computers fail at step 2 in the process. The following flowchart illustrates this process: Figure 2.4 User host access permissions checking process If the user account has the required user right assignment. So. The use of network access groups makes it possible to achieve an extremely high level of control in the solution. the user account is checked to see if it has the required host access permissions as assigned in the ALLOW and DENY user rights in the Group Policy for the trusted host. provides the contractor with the path for the share on the HR server. a contractor plugs her mobile computer into a network connection point in a conference room to copy some data to a share on the HR server of an employee. and the level of security precautions in place on the mobile computer is unknown. Donna. 5. the IT department does not manage the computer. User host access permissions checked for user. After IPsec-protected communication is established. the logical isolation solution has finished conducting its security checks. the standard Windows share and file access permissions are checked by the server to ensure that the user is a member of a group that has the required permissions to access the data that the user requested. When the contractor's computer attempts to connect to the HR server.38 Server and Domain Isolation Using IPsec and Group Policy 4. a member of the HR department. After this process is complete. potentially. and the user logon token is created. the process completes. the SMB protocol authenticates using the client user account.

see Chapter 4. Interestingly. The IPsec policy requirements of the isolation group that the HR server is a member of do not allow the server to communicate with a host that does not use IPsec. categorize. see the "Trust Determination" section of Chapter 3. it is not part of a trusted domain." Modeling and Categorizing Threats In recent years. and technical threats to an . However. and enumerate threats to an organization have been documented over the years. The following section describes some typical threats and briefly discusses threat modeling. and without a thorough understanding of the required communication paths." What Does Server and Domain Isolation Protect Us From? Server and domain isolation are about placing limits on how computers communicate with each other and with devices that attempt to initiate communications. both the types and styles of attacks and the basic methods used to counter them have remained relatively static. Placing boundaries such as authentication. it does provide an additional layer of defense in an overall security strategy. and network location around a host by using IPsec policy is an effective way to mitigate many threats. However. some features and methods of implementing these defenses are different. "Designing and Planning Isolation Groups. it is easier to understand the relationships between the groupings and simpler to define the communication paths (after the requirements have been clearly documented). the logical isolation solution helps protect the IT infrastructure from the threat of untrusted and unmanaged computers even if those computers have physical access to the internal network. environmental. In addition. In summary. Before you can begin to understand the planning that a server and domain isolation solution requires. This documentation often presents a methodology that can be used for modeling common business. your organization should conduct a detailed risk analysis of the threats that are present and how you can counter them by using various technologies and processes. Although IPsec is not a complete security strategy. Another important requirement of the solution is to provide isolation at reduced administrative costs. so all communication attempts are blocked from this untrusted computer. authorization. For more information about the design and framework for isolation groups. When these host access permissions are coupled with IPsec and the hosts are organized into isolation groups. this approach will be difficult to manage and does not scale well without using the centralization capability of Group Policy and Active Directory. "Determining the Current State of Your IT Infrastructure. For more information about trusted devices within the Woodgrove Bank scenario and the design process that Woodgrove used to identify and classify computers as trustworthy. This example explains how isolation can be achieved on a host-by-host basis. this approach alone does not provide strong authentication or the ability to encrypt data. the frequency and complexity of attacks on network-based applications and servers have significantly increased. It is possible to group computers in the same manner as you do users and then assign those groups the ALLOW or DENY user rights assignment in each computer's local policy.Chapter 2: Understanding Server and Domain Isolation 39 computers cannot negotiate an IKE main mode SA because the mobile computer is unable to provide the required Kerberos ticket to allow the computer's credentials to be checked. These limits or boundaries are used to restrict communications by depicting the level at which a device is trusted. Processes that can identify.

40 Server and Domain Isolation Using IPsec and Group Policy organization." of this guide. STRIDE stands for categories of threats to guard against. ." provides a detailed explanation and rationale for obtaining information about the current state of your network and the devices on it. This section describes how this deployment process can take place. it will enable the organization to closely analyze and scrutinize the required communication paths for its information systems and to make informed choices about the compromises that may exist among risk. For detailed discussions about threat models. is to ensure that you have an up-to-date and accurate picture of the current state of your organization's network that includes workstation and server configurations as well as communication paths. Chapter 3. "Determining the Current State of Your IT Infrastructure. Most importantly. see Appendix D. Microsoft uses the STRIDE method for threat modeling. Information Gathering The very first step. For an explanation of particular threats and attacks that server and domain isolation can help mitigate. even before beginning the design process. Defining the Security Landscape. This process will provide all of the requisite information for the subsequent chapters of this solution in addition to providing value to other projects undertaken in the organization. communication requirements. How Can We Deploy Server and Domain Isolation? After you gain an understanding of the threats to your organization and realize how the solution mitigates these threats. It is not possible to develop an effective logical isolation solution without knowing exactly what the solution is expected to protect. the next thing to do is to examine the manner in which such a solution can be deployed. and business requirements. These categories are: S T R I D E spoofing tampering repudiation information disclosure denial-of-service elevation-of-privilege It is essential that adequate time and resources be devoted to defining as detailed a threat model as possible to ensure protection for all assets that need to be protected. see Securing Windows 2000 Server: Chapter 2. "IT Threat Categories.

Although IT professionals have well understood the benefits of IPsec for many years. When the team is confident that the design will work as expected in the lab environment. The information presented in the remaining chapters focuses on the stages that are required to plan and implement this solution. Chapter 6. You should test the proposed IPsec policies in an isolated. With IPsec implementation. . This is not a trivial process! You should take a great deal of care in the planning of this step. Pilot the tested and approved design. "Managing a Server and domain Isolation Environment. and a reliable test methodology. You should test and organize your deployment plan to enable the changes that the solution introduces to be implemented in a way that will allow for a rapid return to a known good state in the event that a configuration or design error somehow remains undetected during the testing phase. However. Chapter 7. the complex nature of the technology has led many people to avoid implementations. "Designing and Planning Isolation Groups.Chapter 2: Understanding Server and Domain Isolation 41 IPsec Deployment Process Overview After you have decided upon and created a design. 2." provides procedures that you can implement for the day-to-day running of an operational environment that is using server and domain isolation. "Designing and Planning Isolation Groups. The final step in the process is to have a plan that can be used to deploy the design to the rest of the organization. and Active Directory to provide a manageable and scalable enterprise solution that can minimize the risk to which data assets are exposed. the next step in the process is to identify a limited number of computers to include in a pilot deployment of the solution into a production environment." provides detailed information on the isolation domain design process and presents options for a phased roll out approach to the solution. Chapter 4. Summary This chapter discussed the goals and processes behind the solution presented in this guide. Test the design and IPsec polices in a proof-of-concept lab. IPsec. 3. "Troubleshooting IPsec. potential exists for serious consequences if the solution does not have in place a solid design. The guidance in this chapter should convey that logical isolation is an additional layer of security that uses server and domain isolation techniques with the capabilities of the Windows platform. Implement a phased roll out of the solution." discusses in detail a number of different approaches that you can use to achieve this. the basic process can be summarized as follows: 1. a well-planned deployment. The identified computers and users should be given proactive support to ensure that any issues that emerge during testing have minimal effect on users' abilities to perform their job roles." provides supportability and troubleshooting information. Group Policy. the next priority is establishing a process for implementing the design into the organization in a manner that is both manageable and of minimal impact to users. non-production environment to ensure that the design works as expected and to test any issues that might arise in the policy settings or deployment mechanisms. It is possible to engineer a design that can (with a single setting change in an IPsec policy) disable many computers' abilities to access network resources. Chapter 4.

groups. and use of Group Policy Windows system security. After reading this chapter. manipulating users. The guidance that this chapter provides is specifically tailored to meet the needs of an organization that is closely aligned with these prerequisites. and access control lists (ACL) Your organization's system naming conventions Your organization's physical and logical locations Risk management practices used in the organization Core networking concepts. including port filtering using firewalls • • • • • Before proceeding with this chapter. the Kerberos version 5 authentication protocol) Active Directory® directory service concepts. including security concepts such as users. Chapter Prerequisites Before using the information in this chapter. Knowledge Prerequisites You should be familiar with concepts of IPsec. auditing. you will have a good understanding of the information that is required for you to complete the design of a server and domain isolation solution. but you will maximize the value of this guidance if you meet them. Failure to meet all the prerequisites will not necessarily have a negative impact on your project. The final part of this chapter discusses the process of understanding and documenting which identified computers are likely to be able to work as "trusted" computers within the solution. and also have a solid understanding of the following: • • Microsoft® Windows® authentication (specifically. The design team will require this information when they create the plans for the solution. and other Active Directory objects.Chapter 3: Determining the Current State of Your IT Infrastructure This chapter provides information about how to obtain the necessary information to plan for and deploy a server and domain isolation solution. ensure that you and your organization meet the following prerequisites. including Active Directory structure and tools. groups. . you should also read the previous chapters in this guide and have a thorough understanding of the architectural and design concepts of the solution. Successfully deployment requires an accurate and up-to-date picture of the information technology (IT) infrastructure.

this solution does not use it. It is also possible that software and hardware will need to be updated. existing asset management systems can be used to collect this information. Note Windows Server 2003 introduced a number of improvements that affect IPsec policies. Web server. For more information. If the organization is not running in mixed or native mode. administration. these roles may be filled by a number of people. • A tool is in place that can capture network configuration data about hosts on the network. analyze monitoring data. There is nothing specific to this solution that would keep it from working with Windows 2000. Finally. and make capacity planning decisions based on that data. A . These IT professionals will usually be members of the organization's support or operations staff. additional network tools may be required to help support and troubleshoot the IPsec environment. it is important to understand that current operational practices will need to be updated with the introduction of IPsec into the environment. For example. the solution was only tested using Windows Server 2003 Active Directory. so care should be taken to ensure that the correct operational procedures are followed when using these tools. You should consult with those members of your organization who may need to be involved in the planning of these groups. The information that is needed to determine the exact requirements will often come from a number of sources throughout the organization. This solution uses universal groups for Group Policy object (GPO) application. including the following people: • • • • • • Executive sponsors Security and audit personnel Active Directory engineering. IT Infrastructure Prerequisites This chapter also assumes that the following IT infrastructure exists: • A Microsoft Windows Server™ 2003 Active Directory domain running in mixed or native mode. and operations personnel DNS (Domain Name System). or there may be fewer people spanning several roles. see the "Host Discovery" section in this chapter. Note In many organizations. However. use of network monitoring tools is restricted. see New features for IPSec. it is still possible to apply the GPO through the use of standard global and local group configurations. However. such as BIOS and firmware updates for network devices. and application owners Network administration and operations personnel Internal education and help desk personnel Note Depending on the structure of your IT organization. Who Should Read This Chapter This chapter is designed for IT professionals who are responsible for creating the IT infrastructure inventory that solution architects and designers will use. • The capability and expertise to perform network monitoring. In addition to requiring information from the listed teams.For more information about the enhancements made to IPsec in Windows Server 2003.Organizational Prerequisites Planning the isolation groups for an organization is a task that usually involves many different people. because this option is more complex to manage.

Chapter 3: Determining the Current State of Your IT Infrastructure 45 good level of technical understanding of both the tools and technologies involved in the information-gathering process and the organization's current infrastructure is required to get the most benefit from this chapter. Do not use a complex and poorly documented network as a starting point for the design. . or other network optimization methods and techniques. the network. and traffic patterns are part of this effort. The obtained information will be used to formulate the implementation and operational components of the server and domain isolation solution. A successful project will depend on the information obtained from such a process. because it can leave too many unidentified areas that are likely to cause problems during implementation. and the directory services that are already deployed in the organization. Network Discovery Perhaps the most important aspect of planning for server and domain isolation is the network architecture. It is better to start with a network architecture that is very well thought out. There may be specific business requirements or circumstances (such as mergers and acquisitions) that do not allow for a "streamlined" network. but the following two components are vital to completing the planning phase of this project: • • Documentation of network segmentation Documentation of the current network traffic model The goal is to have enough information to identify an asset by its location on the network in addition to its physical location. and network devices is a classic IT challenge. because IPsec is layered on the Internet Protocol itself. If the gathered information is not accurate. Understanding subnet layout. This information will allow you to create a design that accounts for all possible elements of the existing infrastructure. Identifying Current State The process of obtaining and maintaining a reliable record of an organization's computers. IP addressing schemes. but if the current state is documented and understood. The following sections outline the required information in each area and provide examples of the information that was gathered for the Woodgrove Bank server and domain isolation solution. and has as few overlaps or discontinuous ranges as possible. software. virtual local area network (VLAN) segmentation. the task of identifying the network and its managed assets is simpler. has easily identifiable address ranges. An incomplete or inaccurate understanding of the network will prevent any isolation solution from being successful. such as TCP/IP addressing and variable length subnet masking. Before starting the planning process for a server and domain isolation project. problems can arise when devices and computers that were not considered during the planning phase are encountered during implementation. you need to collect and analyze up-todate information about the computers. This guidance helps obtain the most relevant information for planning server and domain isolation but does not attempt to address other issues.

try to avoid flooding them with too much information. Undertake a discovery project. When creating network diagrams. For example. . you have two basic options: • • Accept that the information can cause undue risk to the project. the team created the following diagram to illustrate the high-level view of its existing wide area network (WAN) and site environment: Figure 3. If necessary. either through manual processes or with a network analysis tool that can provide the information that is required to document the current network topology. a series of schematic diagrams is often the most effective method of illustrating and understanding the current network configuration. it proceeded to create more detailed diagrams for each site.1 Woodgrove Bank WAN and site network diagram After the team created this diagram. If the documented information is not current or has not been validated recently. and ultimately each subnet within each site. use multiple diagrams showing different layers of detail.46 Server and Domain Isolation Using IPsec and Group Policy Documentation of Network Segmentation If your organization does not have its current network architecture documented and available for reference. Although the required information can be presented in a number of different ways. such documentation should be obtained as soon as possible before proceeding with the solution. in the Woodgrove Bank scenario.

1. For this reason. This information can be useful when making determinations about capacity or about the impact of IPsec on the device.10 prioritize" would be unaffected. However. and firewalls) will communicate using IPsec after the solution is implemented. In relation to IPsec. Note Microsoft Network Monitor added an ESP parser for version 2. switches.0. Amount of RAM. there is no workaround for encrypted packets. any ACLs that specify port or protocol rules will not be processed on the ESP packets. the host itself must be capable of any traffic management or prioritization functions. Unfortunately. a rule that says "From anyone to anyone using port 80 prioritize" would not function. Next. Current examples of such incompatibility include the following: • • Cisco NetFlow on routers cannot analyze packets between IPsec members based on protocol or port. Router-based Quality of Service (QoS) cannot use ports or protocols to prioritize traffic. For more information. Weighted Fair Queuing and other flow-based router traffic priority methods may fail. If traffic management or prioritization must be based on ports or protocol. You can use this information to determine the features that the device supports. you should check BIOS or software running on the device to ensure that IPsec is supported. Traffic analysis. • • . whereas a rule that says "From anyone to 10. Network Infrastructure Devices The devices that make up the network infrastructure (routers.Chapter 3: Determining the Current State of Your IT Infrastructure 47 While documenting your network you may find some network applications and services that will not be compatible with IPsec. If problems arise after IPsec is implemented. using specific IP addresses to prioritize traffic will not be affected. ACLs that specify port or protocol rules will not be processed on the ESP packets. the information helps provide a snapshot of the device and how it is used over a period of time.1 to help troubleshoot unencrypted IPsec packets. load balancers. In addition. it is important to record exactly what information is needed for the various devices in the network. IPsec breaks network-based prioritization and port/protocol-based traffic management when ports and protocols are used. If the device has an ESP parser and uses encryption. the information can help determine whether the root cause is related to high utilization of the device. For example. see Microsoft Systems Management Server. This information (peak utilization in addition to daily/weekly trends that show daily use of the device) is always helpful to have. Devices that do not support or permit IP protocol 50. it is important to examine the following characteristics of these devices on the network to ensure that they will be able to handle the technical and physical requirements of the design: • Make/model. • • • • • If the device cannot parse ESP. the port used by Encapsulating Security Payload (ESP). Network monitors may be unable to parse ESP packets that are not encrypted (ESPNull). Network Monitor is included with Microsoft Systems Management Server (SMS).

The links to which the device is connected. you can determine whether the device is capable of using IPsec in its current state or if it requires a memory or firmware upgrade to support the expected load. or to the Internet. In this case. PMTU discovery must be enabled to allow IPsec to function properly. and set the do not fragment bit (DF bit) to 1." provides NAT-T functionality as a downloadable fix for Windows XP Service Pack (SP) 1 and Windows 2000 SP4. The MTU defines the largest datagram that can be transmitted on a particular interface without being divided into smaller pieces for transmission (a process also known as "fragmentation"). The solution planner should know where this process is taking place on the network.48 Server and Domain Isolation Using IPsec and Group Policy • ACLs that affect IPsec directly. Determining how VLANs are currently implemented on the network will help you understand the traffic patterns or security requirements that currently exist. For example. but protocol) 50 or 51 will prevent IPsec from working. However. Information about other configuration parameters or features that might be peculiar to that make and model may also be helpful. It also helps identify the logical network and connections to various locations on a particular interface. Some sources. The number of ACLs . Networks/subnets connected to device interface(s). Defining the boundary of the network based on an address range is very straightforward and helps identify whether other addresses are either unmanaged or foreign to the internal network (such as Internet addresses). because it helps you to understand the current configuration and "health" of the network infrastructure before configuring those devices to use IPsec (or placing additional load on them if they are already configured to pass IPsec traffic). as well as to determine how IPsec will augment or potentially interfere with these requirements. Note Microsoft Knowledge Base article 818043. you do not have to gather the MTU size on device interfaces. The information identified in this section is critical for the success of the project. However. IPsec will configure the MTU size on the session to the minimum-discovered MTU along the communication path being used. In IPsec communications. Note If Path MTU (PMTU) discovery is enabled and functioning properly. recommend disabling PMTU discovery. Devices must also be configured to pass Internet Key Exchange (IKE) traffic (UDP 500 and 4500) if using network address translation traversal (NAT-T). • • Also note that IPsec can affect an intrusion detection system (IDS). because a specific parser will be required to interpret data inside of packets. If the IDS does not have a parser. Network address translation is commonly used to present an entire address range as a single IP to a connected network. Whether the device is performing network address translation (NAT) . deciding to use IPsec indicates that your organization needs IPsec more than it needs the IDS. The maximum transmission unit (MTU) size on device interface(s) . By studying peak use information. The make and model information will prove useful when obtaining hardware to upgrade the devices (if necessary). • • • VLAN segmentation. Packet fragmentation must be tracked for the Internet Security Association and Key Management Protocol (ISAKMP) by the router. This information is necessary for IPsec policy decisions that are made in subsequent chapters of this guide. an IPsec responder will not function properly unless you have installed Windows XP SP2 or Windows Server 2003 SP1. such as the Windows Server 2003 Hardening Guide. This information will help you to depict the connections that the device maintains between which networks. "L2TP/IPSec NAT-T update for Windows XP and Windows 2000. not allowing the Kerberos protocol (User Datagram Protocol [UDP] and TCP port 88) or IP protocol (not port. This information helps provide the best possible analysis of what the internal network looks like. it cannot examine data in those packets to determine whether a particular session is a potential threat. the MTU is needed to determine areas where fragmentation will occur. ACLs directly affect the ability of certain protocols to function.

UNIX. If such an attack were a threat. it is common to have ports 137. After you obtain this information. Ask yourself such questions as. "Understanding Server and Domain Isolation. Do certain communications occur at the port/protocol level. change the ACLs. and your network has a large number of routers. When you use the threat modeling process performed in Chapter 2. using Windows XP SP2 and Windows Firewall to "lock down" certain ports such as UDP 500 would cause the IKE negotiation to fail. Some of the more common applications and protocols are: • NetBIOS over TCP/IP (NetBT) and server message block (SMB) . and Mac in addition to versions of Windows that are earlier than Windows 2000 SP4. including non-Windows based devices such as Linux. A highly secured building that is connected by copper cable to another building that is not protected can be compromised by an eavesdropping or information replay attack. For example. 138. IPsec could assist by providing strong mutual authentication and traffic encryption for trusted hosts. they also provide for the creation of what are known as null sessions. the solution cannot account for the fact that a lack of physical security on trusted hosts will remain a threat. Analysis of Current Network Traffic Model After you gather the addressing and network infrastructure information. you can easily find those protocols and applications that generate traffic that should be secured from untrusted or unmanaged devices. the next logical step is to carefully examine the communications flow. you can quickly determine whether it is necessary to upgrade the devices to support the requirements of the project. look closely at how all managed and unmanaged devices interact. A null session is a session that is established on a host that does not use the security context of a known user or entity. if a department such as Human Resources (HR) spans several buildings. device configuration could be a significant effort. you need to know how those buildings are connected to determine the level of "trust" to place in the connection. . Unfortunately. These ports provide NetBIOS name resolution services in addition to other features. and 139 enabled for NetBT and port 445 enabled for SMB.Chapter 3: Determining the Current State of Your IT Infrastructure 49 configured on devices will help you estimate the effort that will be required to configure the devices to support IPsec. When you examine traffic flow. On a LAN." to examine the different types of network traffic. Frequently. and you want to use IPsec to encrypt information in that department. or are there many sessions between the same hosts across a wide array of protocols? How do servers and clients communicate with each other? Are there any current security devices or projects that are implemented or planned that could impact the isolation project? For example. However. or take other measures to ensure that the devices will be able to handle the loads that will be expected of them. these sessions are anonymous. If no device on the network is configured to allow IPsec traffic.

The following list describes the necessary information for completing this portion of the discovery effort. In this solution. the method of authentication is the Kerberos protocol. this RPC communication presents a configuration challenge because it means opening the RPC listener port and all ports above 1024. For example. For this solution. or Windows Server 2003). Site architecture is usually aligned with the network topology. the advantage of RPC is that it presents an abstraction of the functionality in layers 1-4 of the Open Systems Interconnection (OSI) model for the applications that use it so that developers do not need to provide low level calls to the network for their distributed applications. The important thing to do is to identify what needs to be protected. see Chapter 5. "Creating Isolation Policies for Isolation Groups. Mac workstations and UNIX workstations. Because the forest is the security boundary in an Active Directory implementation. including domain layout. IPsec can help secure transmissions between hosts by providing authentication of the packets in addition to encrypting the data that they contain. Typically. and site topology. Other traffic or traffic types that need to be secured should be examined and modeled appropriately. IPsec authentication happens as a result of the IKE negotiation process. the IPsec policy will need to account for RPC. Understanding how sites are defined in Active Directory will help provide insight into replication and other details. a special-purpose database that only a few clients are allowed to access. Capturing the number and types of trusts is important because the trusts affect the logical boundaries of isolation and also define how IKE authentication can (or should) occur in the solution. Because many applications depend on RPC for basic functionality. Windows XP. Names and number of domains. RPC operates by presenting an application with a listening port (also known as the endpoint mapper.50 Server and Domain Isolation Using IPsec and Group Policy • Remote procedure call (RPC). This protocol assumes that computers are domain-joined and that they meet the operating system version prerequisites (Windows 2000. • • • . • Number of forests. In a network that is segmented by firewalls. or mainframe-to-terminal connections)? ● Do you need to separate different types of traffic or traffic between locations? Active Directory After the network. or a specialized application for the HR team that is only used by HR managers. site architecture provides a deeper understanding of the Active Directory implementation as it currently exists. organizational unit (OU) architecture. Opening so many ports increases the attack surface of an entire network and reduces the effectiveness of the firewalls. their configuration. and the impact of making changes to Active Directory as a result of implementing IPsec. and the threats that must be mitigated. However. Active Directory is the second most important item from which to gather information." Other traffic. it is necessary to understand the current Active Directory architecture to determine which hosts should be isolated and how to accomplish that isolation. the next step is to examine current traffic patterns and answer the following questions: ● Do you currently have subnets that are dedicated to specific types of traffic (for example. Number and types of trusts. Names and number of sites. You should understand the forest structure. TCP port 135) which then advises the "caller" (often another application) to begin communication on another port in the ephemeral range. For more information about creating IPsec policy. • After documenting the physical and logical network. This information makes it possible to know where computers are currently placed.

corp.woodgrovebank.corp.000 Domain controller type Forest root global catalog PDC Americas regional GC Europe regional DC Asia regional DC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Americas regional GC Chicago.com) that contains four domains spread across five sites. or global catalog (GC) server as shown in the following table: Table 3. MA Toronto. Each site may contain a domain controller (DC). GA Boston.450 480 250 50 50 50 75 36 25 . NY Domain corp.woodgrovebank.com americas.1 Woodgrove Bank Active Directory Information Physical site New York.corp.corp.woodgrovebank.com 750 1.com americas. primary domain controller (PDC). Chapters 4 and 5 of this solution discuss the OU architecture and provide details about how this architecture can be used to apply IPsec policy. IL Atlanta.woodgrovebank. be molded to fit many different requirements and goals.com americas.woodgrovebank. AZ Miami.corp. FL Washington.woodgrovebank.woodgrovebank. • The Woodgrove Bank scenario uses a single forest (corp. CA Pittsburgh.com americas. The OU structure is an ideal place to examine how Group Policy is currently used and how the OUs are laid out.com americas. therefore.corp.com americas.woodgrovebank.woodgrovebank.com americas.com Users 5. Because this project will culminate in the implementation of IPsec policy.com americas. Whether you are using simple IPsec filters (such as filters prescribed in a hardening guide) or implementing complete policies. Canada americas.corp.woodgrovebank.corp.Chapter 3: Determining the Current State of Your IT Infrastructure 51 • OU structure. DC Cambridge. OUs are logical constructs and can.corp.com americas.corp. PA Phoenix.woodgrovebank.woodgrovebank.com americas.corp. A significant amount of operational efficiency can be gained with a little planning when creating an OU structure. Existing use of IPsec policy.woodgrovebank. understanding your current usage and requirements will ensure smoother integration with this solution. MA San Francisco. it is very important to understand how your network currently uses IPsec (if at all).

woodgrovebank. Japan asia. Taiwan asia.com The Woodgrove Bank Active Directory design used the two-way trust relationships that are automatically created by the forest.52 Server and Domain Isolation Using IPsec and Group Policy Physical site London.corp.woodgrovebank.com asia.com corp.corp.woodgrovebank.woodgrovebank. Thailand Singapore Sydney. Ireland europe.woodgrovebank.woodgrovebank. India asia.woodgrovebank.corp.woodgrovebank.com asia.com europe.woodgrovebank.corp.woodgrovebank. .corp.com asia.woodgrovebank.corp.com 100 150 210 45 35 65 Taipei. Netherlands Munich.corp. Switzerland Amsterdam.woodgrovebank.corp.corp.com Hong Kong. europe.com Users 5.com Edinburgh.K.corp.com corp. No additional cross forest or external trusts were in place. China Bangkok. U.com asia.corp.woodgrovebank.com South Africa Tokyo.corp.woodgrovebank.com europe. Domain europe. Australia Bangalore. Italy europe.corp. Scotland europe.woodgrovebank.woodgrovebank.corp. Germany Rome.com Johannesburg.com 350 295 149 80 79 40 37 500 Dublin.200 Domain controller type Forest root GC PDC emulator Europe regional GC Americas regional DC Asia regional DC Europe regional GC Europe regional GC Europe regional GC Europe regional GC Europe regional GC Europe regional GC Europe regional GC Forest root GC PDC emulator Asia regional GC Europe regional DC Americas regional DC Asia regional GC Asia regional GC Asia regional GC Asia regional GC Asia regional GC Asia regional GC Geneva.corp.woodgrovebank.com europe.

• • . It can be used to help differentiate between different network interfaces on the same device. and Asia. The IP address is the address that is used with the subnet mask to identify a host on the network. Host Data Requirements This section describes the host information that is needed and discusses how to represent this information physically and logically. It should be noted that an IP address is not an effective way to identify an asset because it is often subject to change. This structure was used consistently across the three main regional domains of Americas. IP address(es) for each network interface card (NIC) ." decisions are made that require accurate information about the state of all hosts to ensure that they are able to participate in the design's IPsec communications. Because a computer can have more than one media access control (MAC) and/or IP address. Host Discovery The most valuable benefit of conducting an asset discovery project is the large amount of data that is obtained about hosts (workstations and servers) on the network. "Designing and Planning Isolation Groups.Chapter 3: Determining the Current State of Your IT Infrastructure 53 The following figure shows an example of the high-level OU structure that Woodgrove Bank used. The MAC address is a unique 48-bit address that is used to identify a network interface.2 Woodgrove Bank OU structure example Because the server and domain isolation project was Woodgrove Bank's first IPsec implementation. MAC address for each NIC. In Chapter 4. Europe. Figure 3. Computer names can be duplicated under some circumstances. This name is the computer's NetBIOS or DNS name that identifies the computer on the network. there were no existing active IPsec policies in place. • Computer name. so the uniqueness should not be considered absolute. the computer's name is one of the criteria that can be used to determine uniqueness on the network.

For example. It may not be possible to gather this information from an automated process. because these are often used to determine that the minimum security standards have been met. Some tools that perform host discovery can provide this information by querying the hardware information and running applications to determine its type. This information is used to determine whether a computer can obtain IPsec policy from Active Directory or whether it will need to use a local IPsec policy. One problem with automated systems. Even the most automated systems require an element of manual management to ensure that the hosts are accessible and accounted for correctly. service pack. ensuring that the scope of the effort is accurate (such as whether all networks were scanned and host information captured or just the client subnets). and obtaining the information in a timely manner so that it is useful to the project. perform regular discovery efforts at periodic intervals to keep the information current. You can use various methods to gather data from the hosts on the network. • • • After collecting all this information and consolidating it into a database. the use of automated methods to gather data is preferred over manual methods for reasons of speed and accuracy. regardless of which method is used. Note The hardware type information is derived by data interpolation or by a software product that performs queries to provide this information. Automated Discovery Using an automated auditing network management system such as SMS will provide much valuable information about the current state of the IT infrastructure. workstation. Generally.54 Server and Domain Isolation Using IPsec and Group Policy • Operating system. Domain membership. only the amount of time spent obtaining the information is different. and if you can query to the hardware level (or have an asset tag that identifies it as such). or tablet PC. and hotfix versions . A discussion of all elements of a complete IT system's audit is outside the scope of this project. Typical problems that manual processes encounter include having duplicate information. is that hosts that are offline. It is also important to track the current state of service packs and hotfixes that may be installed. These methods range from high-end. unplugged. and some use agents that are installed on the client computers. Hardware type/role. You could use this information to determine the eligibility of a particular computer to participate in isolation and in which isolation group to place the computer. It can be used to determine whether a device should participate in a particular isolation group based on its location or the location of the devices that it communicates with regularly. Some methods use a central scanning mechanism. or otherwise physically (or logically) unable to respond to queries for information will not show up in the final database. however. fully automated systems to completely manual data collection. the information that is required is the same. Physical location. The operating system version is a key factor in determining the ability of a host to communicate with IPsec. Many products and tools are available from a variety of vendors. It is outside of the scope of this guide to compare or recommend products for . such as server. it is important to understand that this audit information should be available to the organization for many more reasons beyond the needs of this solution. Asset tracking and security risk management are just two important examples of processes that require a current and accurate system inventory. However. it is well known that an HP Evo n800 is a mobile computer. This information is simply the location of the device in your organization. You will need the most complete and up-to-date picture of the managed hosts on their networks to create a design that will match your organization's requirements. However. you can more readily determine the appropriate IPsec policy to assign to the device.

0 Windows 2000 Windows XP Windows Server 2003 VBScript Install WSH 5. there is still an element of automation that you can obtain by using the standard remote management and scripting interfaces that are available on the Windows platform. You can use the Windows Scripting Host (WSH). . with the exception of role and physical location. Even if your organization does not possess an automated auditing tool. it is important that the obtained information be available in an electronic format such as a spreadsheet or database. and Windows Management Instrumentation (WMI) to create a script file that can collect the system configuration information. An example VBScript called Discovery. The sheer volume of data that can be generated can make the process of filtering and analyzing very difficult if you cannot quickly and accurately generate specific queries that return the required information. Manual Discovery The biggest difference between manual discovery methods and automated methods is time. You could modify the script to collect additional information or to place the collected information on a remote file share.5 Built-in Built-in Built-in Note You can download the Microsoft Windows Script 5.5 installation package from the Microsoft Download Center.5 Built-in Install WMI CORE 1.5 Install WMI CORE 1. This script uses WMI to retrieve all of the information listed in the "Host Data Requirements" section of this chapter.6 Update from the Microsoft Windows Script Downloads page. Table 3. The information is collected in the text file C:\Discovery\<systemname>_info. The following table shows the availability of both VBScript and WMI by platform. and possibly longer in a larger enterprise. In addition. Microsoft Visual Basic® Scripting Edition (VBScript).vbs is provided in the Tools and Templates folder of this solution. you can use local IT administrators to obtain the information or validate any information that was collected previously. You can download the Windows Management Instrumentation (WMI) CORE 1.6 Built-in Built-in Install WSH 5. If you plan to use a manual method to audit the current state of your infrastructure.txt on the local computer. but the solution requires that the discovery data highlighted in this chapter is present for the design considerations made in chapters 4 and 5. see the demonstration and the asset management datasheet on the SMS 2003 Asset Management page.Chapter 3: Determining the Current State of Your IT Infrastructure 55 purchase. The primary issue with using these tools is ensuring that clients are not missed simply because they are not compatible with the management tool or script. It could take a few dozen people days or weeks to manually gather the information required for this project.6 Built-in Built-in Built-in WMI Install WMI CORE 1. For more information about how SMS 2003 can help perform asset management (or can help gather the information that this project requires).2 VBScript and WMI Availability by Platform Platform Windows 95 Windows 98 Windows Millennium Microsoft Windows NT® version 4.

And because it does not use encryption. This section describes some of the essential aspects of properly planning for IPsec in your environment. Other factors include CPU utilization on network infrastructure servers. because they will contain more main mode SAs than clients in most cases). Whether you use an automatic. Another primary concern is networks that are connected with an NAT device between them. Note In Microsoft's own deployment of this solution it was found that it is normal to expect a one to three percent increase in utilization on your network as a direct result of using IPsec. the impact of IPsec (including some capacity planning) will be your next area of focus. one of the biggest issues that can cause problems to the design is capturing the changes between the original inventory scan and the point at which the implementation is ready to start. it can consume significant overhead on a computer.56 Server and Domain Isolation Using IPsec and Group Policy If WMI or VBScript is not available on the host computer. which provides the strongest IPsec peer-to-peer communication possible without breaking communications with NAT. you can collect some information by using a batch script and external tools. but does not require encryption. Each main mode SA occupies approximately 5 KB of RAM. After the first scan has been completed. it is necessary to query hosts and provide a place to store the results of those queries. The scenarios that require closer analysis will be those that require encryption. Capacity Considerations After information-gathering is complete. This factor is important from a capacity consideration perspective because encryption has overhead that must taken into account during the planning and design phases of the project. Impact of IPsec Because IPsec uses various cryptographic techniques. one might use the Triple Data Encryption Standard (3DES) and Secure Hash Algorithm (SHA-1) to check integrity in situations that require the strongest available encryption and key exchange protection. but as with many security considerations you should expect to have to make tradeoffs. increased overhead on servers and workstations running IPsec (especially servers. The difficulty is that the batch language is extremely limited in functionality. You could use a shorter lifetime for the main mode SA such as three hours. To perform a host discovery. including the header. ESP can also be implemented with null encryption. you should make support staff aware that all further changes need to be recorded and the updates noted in the inventory. and increased network latency as a result of IPsec negotiation. which subsequent chapters discuss. it would have a lower impact than ESP with encryption. or hybrid option to gather the information. . For example. manual. A situation in which a server is expected to broker tens of thousands of concurrent connections could lead to overutilization. ESP provides for the ability to encrypt data. and configuration information is not easily obtained from the command line in earlier versions of Windows. Another scenario involves security association (SA) negotiation. This inventory will be critical for planning and implementing IPsec policies. ESP would need to be selected instead of AH. because NAT violates the very principle that AH is designed to provide: the authentication of an unchanged packet. The problem is that NAT does not allow Authentication Header (AH) conversations between hosts. If NAT devices exist on the internal network.

administration. If such devices need to communicate with a managed device. certain aspects of IPsec are completely predictable.Chapter 3: Determining the Current State of Your IT Infrastructure 57 Impact of Policy IPsec policy and Group Policy will both have an impact on computers' startup times as well as the time that it takes for users to log on. and the hosts to help make an isolation effort succeed. Predeployment Concerns The preceding sections described information that you should gather from the network. Although the following items do not comprise a comprehensive list. Devices Configured Incorrectly Incorrectly configured devices can increase the possibility of failed communications and increased load on the device. Overused Devices You might need to upgrade or reconfigure switches or routers that currently exceed 75 percent utilization to allow for increased traffic on the device and still provide some extra utilization for bursts of traffic. proper planning and considerations for how network devices will be affected is imperative. Incompatible Devices Devices that are not compatible with IPsec cannot participate in IPsec communications. For example. Each packet using IPsec with ESP will be exactly 36 bytes larger than the same packet that does not use IPsec. careful analysis of the data will reveal most of the problems that you will face during pilot and deployment. You can use tools such as the Tivoli Switch Analyzer from IBM or other network analyzer solutions to assist in capacity planning for IPsec in your IT environment. you should place them in the IPsec exemption list. traffic patterns. Network Infrastructure Information gathering enables you to plan for IPsec isolation in a network. it is easier to account for how utilization might be affected on a particular device. and applications are exactly the same for any two customers. they are important to consider when beginning your analysis of gathered information prior to testing and deployment. Because there are six messages required to establish a single main-mode SA. and they can even lead to compromised devices. Following best practices for configuration. it is useful to note the computer startup times and logon times of users before implementing the solution. One alternative is to either upgrade device hardware or software to support IPsec. After gathering this information. Another alternative is to allow those unmanaged devices to communicate with boundary computers when they fall back to clear for their outbound communications. Active Directory. and security of the devices will help alleviate this concern. Because it is extremely unlikely that the scenario. While you are in the information gathering phase. Proper capacity planning for the implementation of IPsec is more about testing and anticipated traffic loads than exact calculations. user concentrations. Recording these times here will provide a baseline to which you can compare the test systems during the pilot to determine the impact on the overall time that it will take for a user to be logged on. This section lists areas of concern that you should examine closely before deploying IPsec. .

Adding IPsec to such a network will only complicate troubleshooting and cause people to suspect IPsec as the source of the problems. There are a number of reasons why these services may have to be included in the solution. You should evaluate these services individually to determine whether they are candidates for server and domain isolation. duplicate addresses. the route tables are nicely summarized. and Windows XP-based computers can communicate with IPsec. or other business requirements. perhaps not all of the clients that can participate should do so. and so on). such as a file server that provides Web services and File Transfer Protocol (FTP) and is also a mail server. Determining Services That Need to Be Isolated In the context of the server and domain isolation project. However. mergers and acquisitions. An important part of the design process is to determine from the information gathered in this chapter which computers and devices will be included in the solution and in which group they will reside. only Windows 2000-. Microsoft SQL Server™. there may be an organizational policy that dictates that all communications between mail servers must be secured by using IPsec. or Internet Information Services (IIS) that provides its services to clients on the network. but the communications between clients and servers do not need to be secured in this manner. Host Information The previous sections have helped you gather sufficient information about the hosts on your network. carefully consider those servers that operate multiple services. . When attempting to isolate services. and other factors can quickly lead to an outdated and inaccurate picture of IP addressing on a network. nothing should affect IPsec. For example. Windows Server 2003-. and other problems can complicate normal network operations. incorrect subnet masks. For example. Determining Client/Server Participation After gathering information about the hosts. such as Microsoft Exchange Server.58 Server and Domain Isolation Using IPsec and Group Policy IP Addressing Because IP addresses are the fundamental building block of an IPsec solution. and address aggregation is easily determined. Chapter 4. To eliminate the biggest issues with IPsec. it is important that the addressing be as "clean" as possible. it is relatively straightforward to determine which hosts would be eligible for integration into the isolation solution. government regulation." discusses the isolation process in greater detail. Active Directory Information If the current Active Directory implementation is functioning properly (that is. Organizational growth. name resolution. "Designing and Planning Isolation Groups. trust relationships. After you determine which clients and servers are able to use IPsec. a service is an application. such as organizational policy. application of Group Policy. You should scrutinize any changes made between the time Active Directory was examined and the time that the isolation project begins to ensure that you will not encounter any compatibility problems. Things such as overlapping address spaces. note how you need to isolate them and what applications you need to closely examine for the impact that the isolation solution may have on them. Dynamic Host Configuration Protocol [DHCP]. ensure that the network is divided into subnets that do not overlap.

Because these computers should still be managed. The following section addresses how to determine trust. In other cases. DNS. you should not attempt to use NLB in "no affinity" mode with IPsec. It is considered a best practice to keep the number of exceptions to the minimum number possible. The solution architect or architecture team will need to use this information in Chapter 4 "Designing and Planning Isolation Groups. An organization might decide to address this limitation by treating some platforms as trusted. Managing Exceptions Managing exceptions is an important part of IPsec planning. Failure to do so can lead to problems with the security of the trusted environment. Technical needs may dictate that some computers or services are exempted from IPsec. and hardware platforms when planning an IPsec deployment. You must account for diversity of elements such as operating systems. and WINS servers. the server and domain isolation solution will not protect them. Management and Monitoring Devices One aspect of IPsec that is often overlooked during initial planning is its impact on management and monitoring of traffic on the network. Determining where to use computers that will allow access from untrusted hosts and controlling access between managed and unmanaged computers is a crucial element of isolation. The term trusted can mean many different things to different people." Trust Determination After obtaining information about the host devices that are currently part of your IT infrastructure. some devices will no longer be able to monitor or manage IPsec enabled computers. Non-Windows-Based Devices Most enterprise networks are not homogenous. . you must make a fundamental determination that will directly affect the ability of a host to participate in the solution. it will be necessary to evaluate what changes you can make to an application or device to enable IPsec monitoring (such as an ESP parser that can look at ESP traffic). If it is not possible to upgrade monitoring or management devices to support IPsec. such as domain controllers and DHCP. In cases where encryption is required. IPsec is not compatible with NLB in "no affinity" mode because IPsec prevents different computers from using the same client connection. the potential risk is lower than allowing unmanaged computers to communicate with managed. If your organization has non-Windows-based computers. an organization is asserting that security is more important than the operational need to monitor the data as it transits the network. it is vital that you record this information. you should understand that the consumption of IPsec policy outside of the Windows realm is not currently possible. so it is important to communicate a firm definition for it to all stakeholders in the project. because the overall security cannot exceed the level of security set by the least secure client that is allowed to achieve trusted status. but because these platforms cannot consume IPsec policy. Because IPsec requires authentication and can allow for encryption.Chapter 3: Determining the Current State of Your IT Infrastructure 59 Network Load Balancing and Clustering Organizations that are using server and domain isolation may choose to exempt computers that use Network Load Balancing (NLB) and clustering. network infrastructure. This determination is to decide at what point a host can be considered trusted. trusted computers. Because of this incompatibility.

An environment that is designated as managed has computers that are running Windows 2000 SP4 or later and that are properly configured and patched. A trusted host that is poorly managed will likely become a point of weakness for the entire solution. lowest risk first): 1. For example. regardless of where they reside in the managed environment. The responsibility for this managed state falls to the IT administrators and users who are responsible for the configuration of the host. The required resources are secure and available. trusted hosts should expect that other trusted hosts will not execute a virus that attacks them. Known. Fundamentally. the security team defined the following five key goals that were used to plan what technologies would be required by a host to allow it to achieve trusted status: • • The computer's or user's identity has not been compromised. and free from unauthorized access. because all trusted hosts are required to use mechanisms (such as antivirus software) to mitigate the threat of viruses. not by using IPsec. virus-free. The device owner/operator understands and will comply with policies and procedures to ensure that the environment remains trustworthy. because a trusted host can assume that other trusted hosts have no malicious intent. When a host is considered trusted. Data and communications are private. . other trusted hosts should be able to reasonably assume that the host will not initiate a malicious act.60 Server and Domain Isolation Using IPsec and Group Policy To understand this concept. Trusted State Classifying a host as trusted does not imply that the host is perfectly secure or invulnerable. Trusted 2. • • • • • • A resource that is designated as secure is tamper-free. These states are (in order of risk. You should implement any restrictions that are needed to control host communications by port or protocol by using a host-based firewall on the computer itself. Trustworthy 3. Unknown. For example. The support team at Woodgrove Bank used these goals to define a set of technology requirements to determine whether a host could be considered trusted. There is a timely response to risks and threats. the blocking of IP-level packets to restrict access by other trusted hosts is not generally required. Untrusted 4. In the Woodgrove Bank scenario. Untrusted The remainder of this section defines these states and how to determine which hosts in your organization belong in which states. Communication between trusted hosts should not be obstructed by the network infrastructure. meaning that information is read and used only by intended recipients. consider the four basic states that are applicable to hosts in a typical IT infrastructure. trusted implies that the host's security risks are managed. A resource that is designated as available meets or exceeds promised levels of uptime and is free of security vulnerabilities.

• • • • It is important to understand that the trusted state is not constant. For this reason. you should make an accompanying configuration note that records what would be required to allow the host to achieve trusted status. Management client. A trusted host should run Windows XP SP2 or Windows Server 2003. if necessary. Spend some time defining the goals and technology requirements that your organization would consider suitable as the minimum configuration for a host to obtain trusted status. Additionally. it is important to identify which hosts can become trusted and which ones cannot (and therefore must be considered untrusted). these systems should be capable of issuing updates or configuration changes if required to help maintain the trusted status. For each computer that is assigned a trustworthy status. A host that continues to meet all these security requirements can be considered trusted. However it is very likely that most host computers that were identified in the discovery process discussed earlier in this chapter will not currently meet these requirements. Antivirus software.Chapter 3: Determining the Current State of Your IT Infrastructure 61 When defining the trusted status. it is imperative that the organization's management systems continually check the trusted hosts to ensure ongoing compliance. All trusted mobile computers will be configured with a BIOS-level password that is under the management of the IT support team. A trusted host will belong to a managed domain. the design team used the following list of technology requirements in the Woodgrove Bank scenario: • • • Operating system. you should take care to ensure that the requirements of a specific asset do not conflict with the requirements of the trusted status. For example. and software. . Domain membership. File system. Password requirements. configurations. This requirement will have no bearing on the ability of the servers to be trusted in the context of this solution. All trusted clients will run antivirus software that is configured to check for and automatically update the latest virus signature files on a daily basis. This information is especially important to both the project design team (to estimate the costs of adding the host to the solution) and the support staff (to enable them to apply the required configuration). BIOS settings. For example. you can define an intermediate trustworthy state. Trustworthy State It is useful to identify as soon as possible those hosts in your current infrastructure that will be able to achieve a trusted state. All trusted hosts will be configured with the NTFS file system. The remainder of this section discusses the different states and their implications. To help with this process. it is a transitive state that is subject to changing security standards and compliance with those standards. at a minimum. Trusted clients must use strong passwords. New threats and new defenses emerge constantly. your HR department may require an additional biometric logon mechanism for specific servers. or. Windows 2000 SP4. Therefore. which means that the IT department needs security management rights. A trustworthy state can be assigned to indicate that the current host is physically capable of achieving the trusted state with required software and configuration changes. ensure that asset owners are free to impose additional security requirements to meet the business needs of a specific asset. All trusted hosts must run a specific network management client to allow for centralized management and control of security policies. However.

Software required. a host with a FAT32-formatted hard disk would fail to meet this requirement. the host may be required to run an older operating system because a specific line of business application will only work on that operating system.62 Server and Domain Isolation Using IPsec and Group Policy Generally. If the host's current operating system cannot support the security needs of the organization. This group of computers will require system upgrades before it can be considered trusted. For example. It is highly recommended that you contact the business owner or independent software vendor (ISV) for the host to discuss the added value of server and domain isolation. The host may have to remain in an untrusted state as a result of a political or business situation that does not allow it to comply with the stated minimum security requirements of the organization. such as an antivirus scanner or a management client. A host that is missing a required security application. The following list provides some examples of the type of upgrade that computers in this group might require: • Operating system upgrade required. Hardware upgrade required. For example. In some cases. • . Computer replacement required. and software allow the host to achieve a trustworthy state. The host needs to run an insecure operating system or needs to operate in an insecure manner to perform its role. you will identify some hosts that cannot achieve trusted status for certain well-understood and well-defined reasons. if the organization requires a secure file system before a host can be considered trusted. These reasons may include the following types: • • Financial. For example. Untrusted State During the process of categorizing an organization's hosts. The current hardware. security software that requires additional storage space on the computer would prompt a requirement for more hard disk space. Upgrade required. additional configuration changes are required before the prerequisite security levels can be achieved. This category is reserved for hosts that are unable to support the security requirements of the solution because their hardware cannot support the minimum acceptable configuration. a host may require a particular hardware upgrade before it can achieve trusted status. Political. Known. a computer that was unable to run a secure operating system because it has an old processor (such as a 100-megahertz [MHz] x86-based computer). Functional. This type of host usually needs an operating system upgrade or additional software that forces the required hardware upgrade. cannot be considered trusted until these applications are installed and active. trustworthy hosts will fall into one of the following two groups: • Configuration required. operating system. an upgrade would be required before the host could achieve a trusted state. However. For example. The funding is not available to upgrade the hardware or software for this host. • • • • Use these groups to assign costs for implementing the solution on the computers that require upgrades.

Windows NT does not support all of the required security capabilities. Although the operating systems of computers that are members of this untrusted domain may fully support the minimum required basic security infrastructure. Additionally. A computer that is a member of a domain that is not trusted by an organization's IT department cannot be classified as trusted. security configurations (even those that can be centrally managed) can be easily overridden by the untrusted domain’s administrators. the required security management capabilities cannot be fully guaranteed when computers are not in a trusted domain. software. smart cards for strong authentication. • In addition. Finally. For example. Computers running any version of Windows that are configured as stand-alone computers or as members of a workgroup are usually unable to achieve a trustworthy state. Computers that run Windows NT. there is no available mechanism to ensure that local Administratorlevel access by a trusted user can be obtained if needed. • Stand-alone computers. these operating systems have only rudimentary central management capabilities for user-specific computer configurations (through System Policy and user logon scripts). Although their operating systems may fully support the minimum required basic security infrastructure. or centralized management of computer configurations (although limited central management of user configurations is supported). upgrading to the Windows Server 2003 platform presents the most secure and manageable solution. For the servers in this project. and standards cannot be assured. Computers in a Windows NT domain. its security configurations are not reapplied on a regular basis to ensure that they remain in effect. policies. Windows NT does not support “deny” ACLs on local resources. these operating systems provide none of the required security management capabilities. adherence to security configuration. Computers in an untrusted domain. Computers that run Windows NT cannot be classified as trustworthy in the context of server and domain isolation because this operating system does not fully support a basic security infrastructure. Computers that run these particular versions of the Windows operating system cannot be classified as trustworthy because these operating systems do not support a basic security infrastructure. it does not support Group Policy or IPsec policies or provide a mechanism that ensures that local Administrator-level access can be obtained if needed. in an untrusted domain. the required security management capabilities are not fully supported when the computers are in a Windows NT domain. The following list includes a few examples of functional reasons that may lead to a classification of this state: • Computers that run Windows 9x or Windows Millennium Edition. any way to ensure the confidentiality and integrity of network communications. the required security management capabilities are unlikely to be achievable when the computer is not a part of a trusted domain. For example. Also. In fact. • • . For example. In addition. An untrusted domain is a domain that cannot provide the required security capabilities to its members. Note The discussion of Windows NT being untrustworthy is strictly in relation to the implementation of server and domain isolation and is not a reflection on its use as an operating system at large. Although these operating systems may fully support the minimum required basic security infrastructure. Nor does Windows NT provide any way to protect the confidentiality of data and ensure its integrity (such as the Windows 2000 Encrypting File System). and measures to effectively monitor compliance are not available.Chapter 3: Determining the Current State of Your IT Infrastructure 63 There can be a number of functional reasons for a host to remain in the known untrusted state. Computers that are members of a domain based on Windows NT cannot be classified as trusted. these operating systems have no security infrastructure by design. Also.

untrusted. whereas other data will need an architect or business sponsor-level input. What this means is that a risk has been identified that the solution cannot mitigate. you can quickly determine the level of effort and approximate cost of bringing a particular host or group of hosts into the scope of the project. Some of it may be gathered by help desk or field service technicians.64 Server and Domain Isolation Using IPsec and Group Policy However. specific guidance on these techniques cannot be provided. Designers of the solution should strive to minimize the impact that the computers in this state can have on their organizations. the goal of these mitigation techniques should be to minimize the risk posed by this host. You must make several key decisions during the design phase of the project that require answers to the following questions: • • • • Does the computer meet the minimum hardware requirements necessary for isolation? Does the computer meet the minimum software requirements necessary for isolation? What configuration changes need to be made to integrate this computer into the isolation solution? What is the projected cost or impact of making the proposed changes to allow the computer to achieve a trusted state? By answering these questions. you should assign it a status of known. After you decide whether to place a computer in a trusted state. . Unknown. you can assign no trust to them. and that by performing the listed remedial actions you can change the state of a computer from untrusted to trusted. if the host has to be accounted for in the design because it provides a required role for the organization. Because of the varied nature of hosts in this category. Untrusted State The unknown. It is important to remember that the state of a computer is transitive. You must use additional techniques to mitigate this known threat. Capturing Upgrade Costs for Current Hosts The final step in this chapter is the process of recording the approximate cost of upgrading the computers to a point that they are capable of participating in the solution. Because hosts in this state have a configuration that is unknown. you are ready to begin planning and designing the isolation groups. which Chapter 4. However. untrusted state should be considered the default state for all hosts. "Designing and Planning Isolation Groups" in this guide discusses. It is very likely that no single person—or even several people within one role—will gather all of this data. All planning for hosts in this state should assume that the host has been or is capable of being compromised and is therefore an unacceptable risk to the organization.

Details. Describes why the computer is not currently in a trusted state. However. Software requirements met. or Windows Server 2003. Old hardware is not compatible with Windows XP SP2. • • • In the previous table.3: • • • Host name. Reflects whether a computer meets the minimum software requirements to participate in the solution. Projected cost $XXX. and support. The name of the host device on the network. if a large number of computers require the same upgrades. This information will help determine whether it is practical or worthwhile from a business perspective to add a particular computer into the solution as a trusted computer. In addition.0 to Windows 2000 SP4 or later." However. the overall cost of the solution would be considerably higher. the minimum was Windows 2000 SP4. hardware. At Woodgrove Bank. Reflects whether a computer meets the minimum hardware requirements to participate in the solution. Current operating system is Windows NT 4. SERVERLON-001 Yes No Join trusted domain and upgrade from Windows NT 4. . it could be considered trustworthy if the required upgrades are possible. a domain centrally managed by Woodgrove Bank IT staff) and must specifically provide IT administrators with access to the computer. No antivirus software present. $XXX. This cost should include estimates for software. Indicates what action must be taken for the computer to achieve a trusted state. Projected cost. Hardware requirements met. as well as all critical security patches for each operating system. Configuration required. lost productivity.3 Sample Host Collection Data Host name HOSTNYC-001 Hardware reqs met No Software reqs met No Configuration Details required Upgrade hardware and software. The following list explains each column from Table 3. untrusted. computers had to be in a trusted domain (that is. Indicates the projected cost for the device to achieve a trusted state. the host HOST-NYC-001 is currently "known.Chapter 3: Determining the Current State of Your IT Infrastructure 65 The following table is an example of a data sheet that you could use to help capture the current state of a host and what would be required for the host to achieve a trusted state.0. Table 3. Windows XP SP2.

test. Summary This chapter provided an overview of the information that is required to conduct a server and domain isolation project.3.66 Server and Domain Isolation Using IPsec and Group Policy The host SERVER-LON-001 meets the hardware requirements but needs to be upgraded to an operating system that is capable of consuming IPsec policy and is domain-joined. You can use the guidance in this chapter to perform the following tasks: • • • • • • • • Identify assets on the network Gather network information Gather host information Determine current traffic information Examine the current Active Directory architecture and obtain pertinent information from it Examine IPsec capacity considerations View the pre-deployment considerations for IPsec Explain what trustworthy and untrustworthy devices are and how they were categorized in the Woodgrove Bank scenario Accomplishment of these tasks will gather all of the information that you need to begin the isolation group design in the following chapter. This information will be the foundation of the efforts undertaken in Chapter 4. and training costs that will be associated with the project. It should be noted that the costs identified in this section only capture the projected cost of the host upgrades. The projected cost is the amount of effort that is required to upgrade the operating system and install antivirus software combined with the cost of the operating system and antivirus software licenses. including impact considerations. save it with the other information that you have gathered in this chapter so that it the architect or architecture team can use it. It also requires antivirus software. which focuses on designing the isolation groups. These additional costs will need to be accounted for in the overall project plan if an accurate project cost is to be identified. There are many additional design. After obtaining the information described in Table 3. support. .

You can download the Microsoft Windows Script 5. .6 Documentation from the Microsoft Download Center.6 for Windows 2000 and XP from the Microsoft Download Center. Windows Millennium Edition and Windows NT 4. For more information about WMI. see Configuring Firewalls.0) from the Microsoft Download Center. You can download the Windows Script 5. You can download the Windows Management Instrumentation (WMI) CORE 1.6 for Windows 98.Chapter 3: Determining the Current State of Your IT Infrastructure 67 More Information This section provides links to areas of additional information that may prove helpful in implementing this solution: • • • • • • For more information about configuring firewalls to support IPsec for Windows Server 2003.0 from the Microsoft Download Center. You can download the Microsoft Windows Script 5. see Windows Management Instrumentation.5 (Windows 95/98/NT 4.

and filter lists Active Directory concepts (including Active Directory structure and tools. and the models in this guidance are not designed to be followed without question or modification. this guide shows how IPsec can be combined with other security settings to build a detailed. IPsec filters. Every business will have unique requirements for their solution. Chapter Prerequisites This section contains information that will help you determine your organization's approach to the server and domain isolation solution. and the application of security templates using Group Policy or command-line tools) • • Before proceeding with this chapter. Successful completion of such a solution for your organization is dependent on the prerequisites identified in this section. Organizations that use this guide will need to determine what is required and achievable for their own environments and make appropriate changes to the isolation group model design to ensure the best fit for their own business requirements. auditing. you should also have read the previous chapters in this guide. manipulating users. Familiarity with Microsoft Windows Server™ 2003 is also required in the following areas: • • IPsec policy. Knowledge Prerequisites You should be familiar with concepts and terminology of IPsec. the use of security templates. groups. and scalable server and domain isolation solution. and access control lists [ACL]. filter actions.Chapter 4: Designing and Planning Isolation Groups This chapter provides complete guidance for defining isolation groups that fulfill the business security requirements discussed in Chapter 2. . and use of Group Policy) Authentication concepts including the Windows logon process and the Kerberos version 5 protocol Windows system security (including security concepts such as users. Through the Woodgrove Bank scenario. groups. and other Active Directory objects. manageable. and Microsoft® Windows® security policy to define and enforce isolation groups. Information technology (IT) administrators can use the concept of isolation groups to manage network traffic within their internal networks in a secure manner that is transparent to applications. IPsec policy to authenticate this identity. This capability can significantly reduce the threat of damage from networkborne infections and attacks. In addition." This solution uses a combination of the computer identity in the Active Directory® directory service domain. "Understanding Server and Domain Isolation. name resolution services. this guide shows the essential details of how an organization can turn its security requirements into deployed isolation groups.

Web server. administration. it is important to identify a single point of contact to help coordinate the efforts of cross-organization teams through the various phases of the project. these roles may be filled by a number of people. see New features for IPSec. "Determining the Current State of Your IT Infrastructure. This chapter uses the information and requirements gathered for Woodgrove Bank. such as the current network architecture and host assets in the network. including the following people: • • • • Executive sponsors Security and audit personnel Active Directory engineering.Organizational Prerequisites You should consult with other members of your organization who may need to be involved in the implementation of this solution. terminologies. Finally. "Understanding Server and Domain Isolation. There is nothing specific to this solution that would keep it from working with Windows 2000. This solution uses universal groups for Group Policy object application. However." and Chapter 3. However. or fewer people may span several roles. You should reference this file during the design process that . it is still possible to apply the Group Policy object (GPO) through the use of standard global and local group configurations. Chapter 2. and technologies of this solution. and Active Directory." and Appendix D: "IT Threat Categories. this solution does not use it. This chapter also assumes that you have met the requirements listed in Chapter 2. Chapter 3." explain the components of the solution and identify the threats that it can and cannot address. which are recorded in detail in Business_Requirements." provides information about how to gather planning data." These requirements include gathering information from the hosts. "Determining the Current State of Your IT Infrastructure. However. running in mixed or native mode. IT Infrastructure Prerequisites This chapter assumes that a Windows Server 2003 Active Directory domain infrastructure exists. and operations personnel Domain Name System (DNS). "Understanding Server and Domain Isolation.xls (available in the Tools and Templates folder). Because this solution will affect many areas of the organization. you must have in place a plan to train the various help desk and support staff on the new concepts. and network engineering administration and operations personnel Note Depending on the structure of your IT organization. Note Windows Server 2003 introduced a number of improvements that affect IPsec policies. For more information about the enhancements made to IPsec in Windows Server 2003. Creating the Server and Domain Isolation Design The design of the solution depends heavily on the business requirements and the information gathered in the previous chapters. The gathering of business requirements and obtaining business sponsorship are also discussed. because this option is more complex to manage. support staff should be trained to handle any issues that arise during deployment. the network. If the organization is not running in mixed or native mode. the solution was only tested using Windows Server 2003 Active Directory.

the best place to start is with those computers that are not owned. The following figure presents the two initial isolation groups that you should consider. Figure 4. Modeling Foundational Groups For most implementations. or even known to exist by the organization's IT department.1 Foundation isolation groups Foundational groups provide logical containers that are an excellent starting point for the isolation group design.Chapter 4: Designing and Planning Isolation Groups 71 this chapter details to better understand the reasons behind the solution's design. These computers will not be part of the solution because they will not be able to use the domain-assigned IPsec policies. The solution design process involves the following primary tasks: • • • • • Modeling foundational groups Planning computer and network access groups (NAG) Creating additional isolation groups Modeling network traffic requirements Assigning computer group and NAG memberships The following sections explain each of these tasks. . managed. it is recommended that you have a common starting point for the initial isolation groups. These computers are generally referred to as untrusted or unmanaged hosts and are the first systems to identify in the design. Untrusted Systems Conceptually.

Isolation Domain The isolation domain provides the first logical container for trusted hosts. Other non-Microsoft remote access or VPN clients. for most implementations. If the native Windows IPsec service cannot be used. it is possible to create an exemption for all IP subnets used for remote access. • • • Even if the native Windows IPsec service is running. The aim for the communications characteristics of the isolation domain is to provide the "normal" or standard rules for the majority of the organization's computers. However. the VPN client should permit IKE and IPsec communication end-to-end through the VPN tunnel connection. The term domain is used in this context to illustrate boundary of trust rather than to suggest a Windows domain. many Windows domains (or forests) may be linked with trust relationships to provide a single logical isolation domain. When this remote client reconnects to the internal network. the reality is that not all operating systems provide the same degree of support for IPsec. the host will not be able to participate in this solution. and the design would be simpler. business requirements exist for these computers to communicate with trusted hosts in the isolation domain. The hosts in this group use IPsec policy to control the communications that are allowed to and from themselves. Boundary Group In almost all organizations. In this solution the two constructs are very similar because Windows domain authentication (Kerberos) is required for accepting inbound connections from trusted hosts.72 Server and Domain Isolation Using IPsec and Group Policy Examples of computers that would fall into this group include: • Non-Windows-based computers and devices. Other isolation groups can be created for the solution if their communication requirements are different from those of the isolation domain. If end-to-end IPsec does not work through the VPN connection. the installation likely disabled the native Windows IPsec service. Perhaps in an ideal world. However. that are unable to communicate using IPsec. . A computer that is joined to a domain that is not trusted by the forest being used as the trust boundary for IKE authentication will not be able to participate in the domain or server isolation solution. In this way. Windows-based computers not joined to a trusted domain. an isolation domain will contain the largest number of computers. it can again participate in the isolation solution. Older versions of the Windows operating system. there will be a number of workstations. Often. If a non-Microsoft IPsec virtual private network (VPN) client is being used for an organization's remote access solution. Conceptually. These hosts will be exposed to a higher level of risk because they are able to receive incoming communications directly from untrusted computers. Macintosh and UNIX workstations and personal digital assistants (PDA) may not have readily available IPsec capabilities. So they should not be considered one and the same. all hosts on the internal network would be able to be trusted to the same level and to use IPsec. Stand-alone computers will not be able to authenticate using Kerberos domain trust in Internet key exchange (IKE). Computers running Microsoft Windows NT® version 4. computers such as Mac or UNIX workstations are unlikely to be able to communicate using IPsec. or servers. The recommended way to deal with this situation is to create an isolation group (referred to as the Boundary group in this guide) that contains hosts that will be allowed to communicate with untrusted systems. an isolation domain is just a type of isolation group. For example.0 and Windows 9x cannot use Group Policy-based IPsec.

Because these Boundary group hosts will be allowed to communicate with trusted hosts that use IPsecsecured network communications and untrusted hosts that use fall back to clear. Boundary hosts will attempt to communicate using IPsec by initiating an IKE negotiation to the originating computer. they must be highly secured in other ways. The following figure illustrates a sample process that can help make such a decision. Figure 4.Chapter 4: Designing and Planning Isolation Groups 73 Computers in the Boundary group are trusted hosts that can communicate both with other trusted hosts and with untrusted hosts.2 Boundary Group Membership Justification Process . the host will Fall back to clear and begin attempting to establish communications in plaintext without IPsec. If no IKE response is received within three seconds. Understanding and mitigating this additional risk should be an important part of the process of deciding whether to place a computer in the Boundary group. setting up a formal business justification process for each computer before agreeing to place it in this group can help ensure that all interested parties understand why the additional risk is required. For example.

not dynamic (or stateful) filtering. it should be assumed that if the risk cannot be mitigated. If the host supports many thousands of clients simultaneously and it has been found that IPsec cannot be used due to the performance impact. If the host supports trusted hosts from different isolation domains that do not trust each other. they are not candidates for the Boundary group. a host-based stateful filtering firewall. Thus. • The IPsec implementation in Windows supports only static filtering. The list of computers that cannot be secured with IPsec is called an exemption list and is implemented in each IPsec policy designed. Because a three-second fall back to clear for access to these services. which would be added to the exemption list. Key infrastructure servers such as domain controllers. Also. membership should be denied. Therefore. particularly DNS. However. If the host supports trusted and untrusted hosts but will never use IPsec to secure communications to trusted hosts. If the host is used for an application that is adversely affected by the three-second fall back to clear delay or by IPsec encapsulation of application traffic. Clearly they must be secured to the maximum extent possible from network attacks. Windows Firewall also supports the capability . If the host provides services for untrusted hosts and trusted hosts but does not meet the criteria for membership of the Boundary isolation group. If the host is a domain controller. the number of exempted hosts must be kept to a minimum. To help mitigate the risk of inbound attacks from hosts in the exemption list. a static exemption for outbound traffic is also a static exemption for inbound traffic. DNS servers. can be used. and Dynamic Host Configuration Protocol (DHCP) servers are usually available to all systems on the internal network. trusted or untrusted. There may also be other servers on the network that trusted hosts cannot use IPsec to access. Windows XP Service Pack (SP) 2 provided domain-based Group Policy controls for configuring the firewall. However. would severely impact the performance of all internal network access.74 Server and Domain Isolation Using IPsec and Group Policy The goal of this process is to determine whether the risk of adding a host to the Boundary group can be mitigated to a level that makes it acceptable to the organization. not just to domain members. and these hosts must be closely managed and hardened as much as possible against attacks and infections. a list of servers and services (protocols) that are exempt from using IPsec will be required for IPsec to work properly. as well as to permit all internal hosts to share the common internal network infrastructure. nor can they take advantage of using IPsec transport mode protection for all of their traffic. trusted hosts require access to the DHCP server to get an Internet Protocol (IP) address during computer startup or when the network cable or card is plugged in to a mobile computer. because IPsec between a domain controller and a domain member is currently not supported. domain controllers meet other criteria in this list and also provide the IPsec policy to the domain members and the Kerberos authentication that the isolation concept is based on. Generally. Exempted hosts therefore have unauthenticated inbound access to every host. these server services cannot require IPsec for inbound access. Ultimately. Creating Exemptions Lists The server and domain isolation security models all run into a few constraints when they are implemented in a live environment. such as Windows Firewall. because they are available to all systems on the network. Therefore. Trusted hosts also require DNS to locate the domain controllers so they can log on to the domain and receive Kerberos credentials. the following conditions might cause a host to be in the exemptions list: • • • • • • If the host is a computer to which trusted hosts require access but it does not have a compatible IPsec implementation.

and increases the time require to download and apply the policy Do not use an exemption if communications can withstand the three-second delay of Fall back to clear. other environments may find it necessary to achieve high levels of security within an isolated domain or group. Carefully consider the communications requirements of each isolation group. Consolidate exempted hosts on the same subnet. see the “Deploying Windows Firewall Settings for Microsoft Windows XP with Service Pack 2” white paper on the Microsoft Download Center. For large organizations. The Business_Requirements. For example. They might not be required to communicate with every exemption in the domain-level policy for clients. including the following: • • • Reduces the overall effectiveness of isolation Creates a greater management burden (because of frequent updates) Increases the size of the policy.Chapter 4: Designing and Planning Isolation Groups 75 to authorize only certain computers through the firewall when protected by IPsec.xls spreadsheet in the Tools and Templates folder provides a model for how requirements can be documented. If several exempt services can be hosted at one IP address. several options exist: • • • • As with defining the Boundary group. which means that it consumes more memory and CPU resources. To keep the number of exemptions as small as possible. A large list has a number of unwanted effects on every computer that receives the policy. you will need to create up to three specialized Active Directory groups. rather than using exemptions for each IP address. For more information. After inbound and outbound requirements are documented. Computer Groups Each isolation group will require a computer group to be created that will be used to contain the members of the isolation group. this solution uses security group filtering on the GPOs to deliver an IPsec policy to the computers in a particular isolation group. Refer to the decision flow in the previous figure as a model for processing requests for exemptions. you should start a record of the new Active Directory groups that will be required to support the isolation group requirements. using the policy setting "Windows Firewall: Allow authenticated IPsec bypass. the number of filters will be reduced." Woodgrove Bank chose not to implement Windows Firewall capability. However. particularly server-only groups. Where network traffic volume will permit. the servers might be able to reside on a subnet that is exempted. Consolidate server functions. Planning the Computer and Network Access Groups The isolation domain and each isolation group must have clear and complete specifications of network security requirements. At this point in the process. you can design the mechanisms for implementing access controls. . there should be a formal process to approve hosts being added to the exemption list. This is required because the security requirements for an isolation group are met by several types of security settings in GPOs assigned in the domain. slows down network throughput. the list of exemptions may grow quite large if all the exemptions are implemented by one IPsec policy for the entire domain or for all trusted forests. The following section explains the role of each of these groups. This option does not apply to domain controllers. For each isolation group.

the DENY user right will have computers that are not supposed to have IPsec-protected inbound access. Although it is possible to use a single group to contain user and computer accounts. this guide refers to these groups as network access groups (NAG). You achieve this control by using either the "Deny access to this computer from the network" (DENY) or the "Access this computer from the network" (ALLOW) user right in Group Policy. During the implementation of this solution. For more information. This approach improves the ability to manage and support these policies and groups on an ongoing basis. To help differentiate this group from any other groups. Likewise. Finally. The "account" may be a computer. There are two types of NAGs that you can create: Allow and Deny. which allows all authenticated users and computers to access the computer. see the Network Access Control Layers diagram in Chapter 2. service-specific permissions (such as file share access control lists) are evaluated using the user identity. the network logon right now controls the ability for a remote computer to make any IPsec protected connections. the Everyone group will be replaced through Group Policy user rights assignment with NAGs that contain specific computers or users and groups. Table 4. This is to avoid having to change or create a new organizational unit (OU) structure based on isolation group membership to apply the proper GPOs. This group will contain all computers that are allowed to accept communication from untrusted systems. It is these groups that control the ability of other trusted systems to either explicitly allow or deny access. depending on organizational requirements. or service account.1 Woodgrove Bank Computer Groups Computer group name CG_IsolationDomain_Computers Description This universal group will contain all computers that are part of the isolation domain. this user right access control only applies to network services that receive logon credentials to authenticate the account for network logon. After this IP-level access control has been checked. If the OU structure can be changed to reflect the isolation group membership. Technically. The configuration of user rights assignment for ALLOW and DENY supplements the guidance that earlier Windows platform security guides provide. which again evaluates the network logon rights of the user identity. . one for users and one for computers. CG_BoundaryIG_Computers Network Access Groups Using IPsec and Kerberos alone provides a trusted and untrusted authentication boundary. When IPsec policy is configured to protect all traffic IKE will confirm that the remote computer has a network logon right. Therefore. user. because those guides did not specifically accommodate computer authentication that IPsec requires." By default. file sharing) typically authenticate the user. "Understanding Server and Domain Isolation. the normal upper layer protocols (for example. these computer security groups are not needed to control the application of Group Policy. the ALLOW user right contains the Everyone group.76 Server and Domain Isolation Using IPsec and Group Policy Computer accounts that are members of the computer group will be assigned the associated IPsec policy when the GPO is processed. Microsoft recommends using separate groups.

if your organization requires some trusted hosts to have different inbound or outbound network access controls or traffic protection. it created a boundary server as a print server. Woodgrove decided that one server could fulfill the needs of both types of hosts. one business requirement set a limit on the amount of new computer hardware that could be purchased for the year. because each new group will increase the complexity of the overall design and therefore make it more difficult to support and manage. you can continue to the next section of this chapter to define the traffic models for these two isolation groups. Trusted hosts would still be able to make outbound connections to the print server when needed. The goal should be to keep the number of these hosts to a minimum. Accordingly. The designers at Woodgrove Bank identified a need for the following NAG: Table 4. The first thing to do is to identify which computers have specific isolation or traffic protection requirements that are not met by the settings for the isolation domain.Chapter 4: Designing and Planning Isolation Groups 77 The requirements that NAGs might implement include the following: • • • Blocking network access to sensitive servers from boundary hosts or trusted hosts located in public areas Limiting access to servers dedicated to senior executives to just the client computers that those executives use Isolating trusted hosts in a research and development project from all other trusted hosts in the domain In the Woodgrove Bank scenario. However. Therefore. there are two isolation groups: the isolation domain and the boundary group. the rest of the trusted hosts would need to block inbound access from that server. Woodgrove determined that it needed a DENY NAG to implement this requirement. . isolation groups will be required for each different set of requirements. Although Woodgrove would have preferred to purchase a new server that only untrusted computers would use for printing. At this stage of the design process. Because the print server was at higher risk of being infected and attacked from untrusted computers. Creating Additional Isolation Groups At this point in the design process.2 Woodgrove Bank Network Access Groups NAG Name DNAG_IsolationDomain_Computers Description This group includes any domain computer account that is denied from making inbound IPsec-protected connections to all trusted hosts in the isolation domain. The purpose of this section is to help you understand when additional groups will be required. assigning NAG membership is not required. All that is needed is to identify and document the NAGs that the design will require. A print server was needed to allow both trusted hosts and untrusted hosts to print. If your organization's business requirements can be satisfied by this design.

For example.3 Final Woodgrove Bank group design . trusted hosts might not be allowed to make outbound connections to untrusted computers to reduce the risk of information leakage or to enforce regulatory compliance for the protection of network traffic. All network traffic with these servers required 128-bit-level encryption that was compliant with U. These additional groups brought the total group count for Woodgrove Bank to four. these two different sets of requirements indicated that two additional isolation groups were required. Lastly. the designers identified requirements that could only be fulfilled by the creation of the following two additional isolation groups: • The Encryption group contained a small group of application servers with high value data that required the highest level of protection. The No Fallback group was required for a number of trusted hosts in the isolation domain that needed to be restricted from network communications to untrusted systems. In other cases. Thus. the inbound access requirements of servers that contain high value data are to allow only a subset of trusted hosts in the domain to connect. federal regulations for financial data privacy. Only a specific subset of trusted clients would be allowed to connect inbound to these servers. these servers were not allowed to make outbound connections to untrusted hosts or to receive inbound connections from boundary hosts.78 Server and Domain Isolation Using IPsec and Group Policy Typical examples of requirements that might lead to creation of a new group include the following: • • • Encryption requirements Limited host or user access required at the network level Outgoing or incoming network traffic flow or protection requirements that differ from the isolation domain In many cases.S. in the Woodgrove Bank scenario. The following figure shows how these groups looked in the final Woodgrove Bank isolation group design: Figure 4. • Although the second group has a no fallback requirement. they do not have the full set of requirements that the applications servers do.

3 Additional Woodgrove Bank Computer Groups Computer Group Name CG_NoFallbackIG_Computers CG_EncryptionIG_Computers Description This group contains all computers that are not allowed to Fall back to clear. Accordingly. DNAG_EncryptionIG_Computers This group includes groups of computer accounts that are to be denied access to hosts in the Encryption isolation group. as part of their own experience. However. This group contains all computers that are required to use encryption. along with the form that the communications will take. This group is assigned for computers that require the ability for untrusted hosts to access them.4 Woodgrove Bank Network Access Groups NAG name ANAG_EncryptedResourceAccess_Users Description This group includes all users who are authorized to have access to the Encryption isolation group servers. ANAG_EncryptedResourceAccess_Computers This group contains all computers that are authorized to have inbound network access to the Encryption isolation group servers. Gathering Network Traffic Requirements At this point in the design process. This is the default group for all trusted computers. This group contains computers that have a higher security requirement that dictates that they not be allowed the ability to initiate communications to untrusted hosts directly. No Fallback Isolation Group. it defined additional computer groups to control the application of these newly identified policies. you should document the communications traffic requirements that will be allowed to pass between the groups. Because Woodgrove Bank identified two additional groups that require IPsec policies. Boundary Isolation Group. the creation of a diagram was the best method for communication of the exact requirements. Table 4. There are many ways to record the traffic requirements for the groups.Chapter 4: Designing and Planning Isolation Groups 79 The following four groups require policy to achieve the design requirements: • • • • Isolation Domain. encrypted communications path. the Microsoft IT support team found. The designers at Woodgrove Bank created the following NAGs: Table 4. . Encryption Isolation Group. Woodgrove determined that it would need NAGs to authorize inbound access for the subset of trusted hosts. This group only allows communications through a trusted.

In the figure. Groups that are depicted with a bold dashed line have an IPsec policy assigned to the computers in that group. the untrusted hosts. Figure 4. the dotted lines indicate communications that are allowed to occur without IPsec. isolation groups may have a business requirement to exempt specific computers just for the computers in that group. all arrows shown in a solid black line use IPsec for their communications. the isolation group would then contain an additional exemption list of the computers that are to be exempted in addition to the common exemptions. and the exemptions lists. the exemptions lists are shown as a single grouping. In these cases.80 Server and Domain Isolation Using IPsec and Group Policy The following figure depicts the communications paths that are typically allowed between the foundational groups. Microsoft recommends keeping the exemption list entries to a minimum because they explicitly exempt systems from participating in the IPsec infrastructure.4 Typical allowed communications paths for foundational isolation groups . However. This is typically the case for infrastructure services. such as Domain Controllers or DNS servers. To simplify this model.

The traffic may require encryption. The specific exemptions may be different for isolation groups. In the example shown in the previous figure. the communications will fail because there is no Fall back to clear option when IKE fails a negotiation. Indicates whether this path attempts to use IPsec to secure the communications. it is possible to create a very concise representation of the solution's communications. and the exemptions lists: Table 4. Fall back. If the IKE negotiation fails. By using this form of documentation. unsecured systems. Indicates whether the path allows the roles of the initiator and responder to be reversed so that traffic can start from either the From or the To group. The group that contains the responders that will be contacted through the allowed communication path. The following list explains each column: • • • • Path. • • • The short forms of the group names were used to keep the information in the table as concise as possible. The number assigned to the communications path illustrated in the group diagram. The group that contains the initiators of the traffic. This path attempts to use IPsec to protect the traffic. 4. By assuming that all network traffic is disallowed unless specifically identified in this table. Try IKE/IPsec. Indicates whether this path requires the communications to be encrypted by using an encryption algorithm set in the IPsec policy. Bidirectional. To. and 7 illustrate the network communications that are specifically permitted for all hosts listed by the Exemptions lists in their IPsec policy. From. Indicates whether the communications can revert to not using IPsec if the IKE negotiation fails to complete. • . Encrypt. Traffic path 2 shows the communications between the isolation domain and Boundary groups.Chapter 4: Designing and Planning Isolation Groups 81 The following table records the allowed communications paths for the traffic among the foundational groups. the process of identifying the traffic that will be protected as part of the solution becomes much clearer.5 Allowed Communication Options for Foundational Isolation Groups Path 1 2 3 4 5 6 7 From ID ID ID BO BO UN UN To EX BO UN EX UN BO EX Bidirectional Try IKE/IPsec Yes Yes No Yes No No Yes No Yes Yes No Yes No No Fall back No No Yes No Yes No No Encrypt No No No No No No No The previous table records the communications requirements for each allowed communications path in the initial isolation group design. each of the following allowed paths is explained: • Traffic paths 1. depending on the security requirements.

82

Server and Domain Isolation Using IPsec and Group Policy

Traffic path 3 shows that the hosts in the isolation domain are able to initiate communications with untrusted hosts. This is possible because the policy for this group will allow the isolation domain hosts to Fall back to clear if there is no reply to the initial IKE negotiation request. Untrusted systems that attempt to initiate nonIPsec connections with trusted hosts are blocked by the IPsec inbound filters. Traffic paths 5 and 6 document the allowed communications between the Boundary group and untrusted systems. Path 4 shows that the Boundary group is allowed to communicate outbound with untrusted hosts in the clear. If the IKE negotiation is not responded to, the host will Fall back to clear text communications. Path 5 covers the traffic initiated from the untrusted hosts to the Boundary group. Although this arrow looks similar to path 4, the details in the table illustrate that the untrusted hosts are not attempting IKE negotiation with the Boundary group. They are connecting with clear text TCP/IP connections.

After the foundational communications have been documented, additional groups can be added to the overall plan and their communications requirements recorded in the same way. For example, the two additional groups required by the Woodgrove Bank scenario led to a more complex communications diagram, as shown in the following figure.

Figure 4.5 Woodgrove Bank-allowed communications paths for isolation groups The following table records the allowed communications paths for the traffic in the additional groups for the Woodgrove Bank scenario.

Chapter 4: Designing and Planning Isolation Groups

83

Table 4.6 Allowed Communication Options for Additional Isolation Groups Path 8 9 10 11 12 13 14 From EN EN EN EN NF NF NF To EX ID NC BO ID EX BO Bidirectional Try IKE/IPsec Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes No Yes Fall back No No No No No No No Encrypt No Yes Yes Yes No No No

In the example shown in the previous figure and described in the previous table, each of the following additional allowed paths is explained: • • Paths 8 and 13 are clear communications for all exempted traffic. Paths 9 and 10 show the IPsec Encapsulating Security Payload (ESP)-encrypted communications that is required between the Encryption, No Fallback, and isolation domain groups. If the IKE negotiation fails to secure the communication using encryption, the communication attempt fails. The traffic for path 11 is slightly different because it only allows communications to be initiated from the Encryption group to the Boundary group and not the reverse. This is because Woodgrove Bank placed high value data in the Encryption group and does not want that data to be exposed to computers that are accessed directly by untrusted resources. The traffic paths for 12 and 14 could be implemented by either IPsec AH transport mode, or IPsec ESP transport mode that is authenticated but without encryption (ESP-Null).

As this example illustrates, adding groups can have an exponential impact on the complexity of the solution. For this reason, it is recommended that the number of groups be kept to a minimum, especially during the early stages of a deployment when the most change is being introduced.

Assigning Computer Group and Network Access Group Memberships
After the traffic requirements are detailed and documented in the design, the next task is to identify which hosts will be members of which computer group or NAG. As mentioned previously, computer groups are used in this solution to apply the GPO that contains the associated IPsec policy. After determining that a computer must belong to a particular isolation group, that computer's account is added to the computer group for that isolation group. For the isolation domain, this step is not required, because all domain computers implicitly belong in the isolation domain computer group. Membership in a NAG will be based on the inbound authorization that the NAG implements. For example, if a NAG exists to restrict communication for a particular server to a known set of clients, the client computer accounts need to be placed in the appropriate NAG. NAGs are only created when needed, and therefore they have no default configuration.

84

Server and Domain Isolation Using IPsec and Group Policy

Computer Group Membership
It is important that a host should not be represented in more than one computer group, because the computer group is used to control which GPOs apply. Although it might be theoretically possible to modify the policies to allow a host to belong to more than one computer group, the complexity of such an approach would rapidly make the solution unsupportable. Generally, this task of determining computer group membership is not complicated, but it can be time consuming. You should use the information generated from an audit such as the one performed in Chapter 3, "Determining the Current State of Your IT Infrastructure," of this guide to place each host into one computer group based on the isolation group membership of that host. You can determine this placement by adding a Group column to record the computer group membership for the final design, as shown in the following table: Table 4.7 Sample Host Collection Data Host name HOSTNYC-001 Hardware Software reqs reqs met? met? No No Configuration Details required Upgrade both hardware and software Projected Group cost ID

Current $XXX. operating system is Windows NT 4.0. Old hardware not compatible with Windows XP. No antivirus software present. $XXX.

SERVER- Yes LON-001

No

Join Trusted domain, upgrade from NT 4 to Windows 2000 or later

EN

Network Access Group Membership
The final step in this design process is to populate the membership of the NAGs identified earlier in this chapter. Although a trusted host should only belong to one computer group, it is possible for a trusted host to be a member of multiple NAGs. Try to use as few NAGs as possible to limit the complexity of the solution. When assigning membership to a NAG for user accounts, decide how tightly you want to control the access. For a resource that is already using standard share and file permissions to ensure the correct level of control, the simplest way to assign membership would be to assign the user's NAG membership to Domain Users from each trusted domain in the forest that requires access to the resource. This approach almost restores the behavior of the original default value of Authenticated Users but does not include local user accounts. If local user or service accounts are required, then a domain-based GPO may not be the best approach to configure ALLOW and DENY network logon rights. The ALLOW and DENY user rights assignment do not merge settings among several

Chapter 4: Designing and Planning Isolation Groups

85

GPOs. Therefore, this computer should be prevented from having the domain-based GPO assigned for ALLOW and DENY and should use a customized local GPO. If the domain-based GPO that delivers IPsec policy assignment is different from the GPO used to deliver network logon rights, the domain-based GPO for IPsec policy assignment can still be used. In addition, decide how to implement the inbound access requirements using either an ALLOW NAG or a DENY NAG or both. Deciding which type of NAG to create is based solely on what the intended behavior is and what minimizes administrative effort. It may be helpful to have a pre-existing but empty DENY NAG for users and a DENY NAG for computers already populated in the GPO "Deny access to this computer from the network" right. For high-security scenarios, the membership of user NAGs can be assigned to specific users or groups. If this method is used, it should be understood that users who are not members of this group will be blocked from accessing the computer over the network, even if they are members of the local administrators group and have full control on all share and file permissions. For the Woodgrove Bank scenario, NAG_EncryptedResourceAccess_Users membership was assigned to Domain Users and recorded as shown in the following table: Table 4.8 Woodgrove Bank Network Access Groups with Membership Assigned NAG Name Membership Description This group is for all users that are authorized to make inbound IPsecprotected connections to the Encryption isolation group computers. This group contains all computers that are authorized to make inbound IPsec-protected connections to the Encryption isolation group computers. This group contains all computers that are not authorized to make inbound IPsec-protected connections to the Encryption isolation group computers.

ANAG_EncryptedResource User7 Access_Users

ANAG_EncryptedResource IPS-SQL-DFS-01 Access_Computers IPS-SQL-DFS-02 IPS-ST-XP-05 DNAG_EncryptionIG_ Computers

Note Membership in a NAG does not control the level of IPsec traffic protection. IPsec policy settings control the security methods used for protecting traffic and are independent of the identity being authenticated by IKE. The IKE negotiation is only aware of whether the Kerberos computer identity passed or failed the authentication process. It cannot implement a policy of "encrypt if user3 connects" or "encrypt if trusted host IPS-SQL-DFS-01 or IPS-SQL-DFS-02". The administrator must achieve the intended behavior by using an IPsec policy for the servers in the Encryption isolation group that requires "encryption for any inbound trusted host connection" and likewise requires "encryption for any outbound connection to a trusted host."

So far, this design process has not reviewed the details of the IPsec policy design. Chapter 5 will provide details of the IPsec policy design for Woodgrove. At this point in the design process, you have completed the tasks that are required to turn your requirements into a draft design. This section has helped you develop both the design and the documentation that will be required for the creation of the IPsec policies.

86

Server and Domain Isolation Using IPsec and Group Policy

Limitations That Might Affect Your Design
The following issues might affect your design and therefore must be considered before your design can be considered complete: • Maximum number of concurrent connections by unique hosts to servers using IPsec. The number of concurrent connections is a key factor in whether an IPsec implementation on high-use servers will go smoothly or will overload the CPU with IKE or IPsec processing. Each successful IKE negotiation establishes SAs that occupy approximately 5 kilobytes of user-mode memory. CPU resources are needed to maintain current IKE SA states with all concurrently connected peers. For more information about scaling, see the "Improving Security with Domain Isolation" white paper. Maximum Kerberos token size limitation for hosts using IPsec. There is a practical limit of approximately 1,000 groups per user, and if that value is exceeded, GPO application may fail to occur. For more information on this subject, see Microsoft Knowledge Base articles 327825 "New Resolution for Problems That Occur When Users Belong to Many Groups" and 306259 "Members of an Extremely Large Number of Groups Cannot Log On to the Domain".

Although these articles mention users specifically, the issue also exists for computer accounts, because the Kerberos MaxTokenSize also applies to computer accounts. Although this limit should rarely be reached in most implementations, you need to be aware of this issue if you decide to put one computer (perhaps a client) in a large number of NAGs. If your design will be affected by these issues, you will need to revisit the design process to address them. For example, you could address the maximum number of concurrent connections issue by moving a very heavily loaded server into the Exemptions lists. You could address the maximum Kerberos token size limitation by reducing the number of NAGs your design will use. If these limitations will not affect your design, the next task is to consider how the design will be deployed into the organization.

Group Implementation Methods
After you have created the initial design, you must carefully consider the process of deploying IPsec. Only in the smallest environments is it possible to simply deploy the policies to all computers and expect IPsec to work smoothly with an acceptably small impact on users. In large organizations, complexity and risk necessitate a phased deployment strategy. By using such an approach, the organization can help mitigate the risk associated with such a fundamental change to the environment. Without careful planning, help desk calls and lost productivity will quickly increase the cost of deployment. There are a number of ways to deploy IPsec in an organization. Some of the factors that help determine the deployment method include: • • • • • The environment's beginning and end states The complexity of group configuration The complexity of the domain structure The organization's risk tolerance Security requirements

including the specific sequence and timing of rollout steps and policy changes. . the computers that will receive the policies are identified and their computer accounts are placed in the appropriate computer groups with read access to the GPO. It is particularly important to analyze the impact of IPsec policy changes. Each IPsec policy has all of the exemptions and secure subnets defined with the appropriate filter actions enabled. For deployment in a production environment. The IPsec Standard policy is its default policy that requires incoming traffic to use IPsec but will allow the systems to Fall back to clear if they initiate to a non-IPsecbased computer. You should implement a formal change control process with rollback strategies and rollback criteria to ensure that all affected IT organizations are aware of the change and its impact and know how to coordinate feedback to decision makers. none of the computers in the environment should receive the policy. Group Policy polling intervals. Deploy by Group The Deploy by Group method uses fully-defined IPsec policies but controls the application of the policies through the use of groups and ACLs on the GPOs that deliver the policies. organizations should not deploy IPsec policies that restrict or block communication initially to ensure that adequate time is available to troubleshoot problems and to reduce the management coordination for complex environments. Note ACLs that restrict domain computers from reading the IPsec policy objects or the IPsec policy container in Active Directory are not recommended. Regardless of which approach is used to deploy IPsec. use a filter action that requests IPsec functionality but will accept plaintext communication by using Fall back to clear. the GPO.Chapter 4: Designing and Planning Isolation Groups 87 The following deployment methods are not all inclusive but provide examples of possible approaches that you could take. and IPsec policy polling intervals. computer groups are created in the domain to manage and apply these newly created GPOs. Finally. will apply at the next Group Policy polling interval. the IPsec policies are created in Active Directory in their final configuration. Next. The IPsec Encryption policy requires encrypted IPsec to be negotiated at all times. After the computer's Kerberos tickets are updated with the group membership information. because they will take effect in the production environment as a result of Kerberos ticket lifetimes. along with the corresponding IPsec policy. the appropriate IPsec policies are assigned to their corresponding GPO. an initial pilot is strongly recommended for each major phase of the rollout. In addition. In addition. This approach will help minimize the impact of the initial rollout. Generally. IPsec Standard and IPsec Encryption. Consider an organization that has two IPsec policies defined. the GPO is linked to the appropriate object in Active Directory. The ACLs on the GPOs are modified so that members of Authenticated Users no longer have the "Apply" right. it is highly recommended that the deployment scenario be thoroughly tested in a lab environment. The appropriate administrator user groups for management and application of the policy are also granted rights to the GPO. At this point in time. After the roll out is complete. Then the IPsec policy administrators create GPOs for each IPsec policy. In addition. because the ACLs that control the assignment of the GPO (the ability to read the GPO) are empty. In the Deploy by group method. move towards more secure modes of operation that require the traffic to be protected by IPsec. You can also use a combination of these approaches.

88

Server and Domain Isolation Using IPsec and Group Policy

In this example, the organization's administration creates two GPOs in Active Directory: Standard IPsec GPO and Encrypted IPsec GPO. In addition, they identify the groups in the following table: Table 4.9 IPsec Administration Groups Group name IPsecSTD IPsecENC Group type Universal Universal Description Controls application of the IPsec Standard policy Controls application of the IPsec Encryption policy

The ACLs on the two newly created GPOs are updated so that they do not automatically get applied to the Authenticated Users group and so that the appropriate application groups and management groups are given the correct rights. The administration modified the ACLs for the two GPOs according to the information in the following table: Table 4.10 Group Rights on GPOs Group Authenticated Users IPsecENC IPsecSTD Standard IPsec GPO Read None Read Apply Group Policy
Note This table only shows permissions that are added or modified. There will be some additional groups with permissions, as well.

Encrypted IPsec GPO Read Read Apply Group Policy None

The administrators linked the two GPOs to the domain in Active Directory. This approach ensures that the policy will apply on any computer in the domain without modifying its location (unless the computer is located in an OU that blocks policies). As computers are identified, their computer accounts are added to either the IPsecSTD group or the IPsecEnc group. After a period of time, the corresponding IPsec policy will apply and be in effect. This method requires careful planning to ensure that communications are not disrupted. For example, if a server was placed in the IPsecEnc group, but multiple clients that depended on that server could not negotiate IPsec, communications between those clients and server would be disrupted.

Deploy by Policy Buildup
This deployment method uses a technique in which the IPsec policies can be built from scratch during the deployment. The advantage to this method is that IPsec is negotiated only for a small percentage of the total TCP/IP traffic, instead of for all internal subnets in the deploy-by-group method. It also allows the testing of all network paths in the internal network to this target subnet to be sure there are no problems with the network passing IKE negotiation and IPsec protected traffic. A further advantage is that the polling interval for IPsec can more quickly deliver IPsec policy updates (including rollback) instead of having to depend on group membership changes in the Kerberos Ticket Granting Ticket (TGT) or service tickets. The disadvantage to this method is that it applies to all computers in the isolation domain or group, not to specific computers as in the Deploy by

Chapter 4: Designing and Planning Isolation Groups

89

Group Method. Also, all computers will have a three-second delay for Fall back to clear at some point when communicating to the specified subnets. In this deployment method, IPsec policies only include exceptions initially; no rules exist for computers to negotiate security in the IPsec policy. This method first tests and ensures that any previously existing local IPsec policies that might be in use are removed. The administration team should be able to identify hosts that were using locally defined IPsec policies in advance to manage those systems with a special process. If a local IPsec policy is overridden by a domain policy, there could be interruptions in communications and a loss of security for the affected computers. Unlike local policies that are overwritten by the domain policy application, persistent policies on Windows Server 2003 merge with the result of the application of the domain policy. A system that contains a persistent policy might appear to work, but the configuration of the persistent policy can change the behavior or actually lessen the security that the domain policy provides or can disrupt the communications after secure subnet rules are added to the policy. Next, you can create a security rule with a filter that affects only a single subnet within the organization's network, for example "From Any IP to Subnet A all traffic, negotiate." This rule would have a filter action to accept inbound plaintext and trigger negotiations for all outbound traffic to that subnet with Fall back to clear enabled. As the rollout in all domains of this IPsec policy takes effect, communications will gradually go from soft SAs to normal IPsec security associations for trusted hosts just on that subnet. Any IKE negotiation failures are investigated and resolved. Any application incompatibilities are identified and fixed. IPsec will secure communication between trusted hosts within that subnet. Communication with trusted hosts outside that subnet will Fall back to clear after the three-second delay. Additional subnets are added to the secure rule until the policy is built up to its final state. Consider an organization that has a single IPsec policy defined, which is called IPsec Standard policy, which requests IPsec negotiation but failing that will Fall back to clear text communication. The policy is created in Active Directory, and it contains only exemption rules. The Standard IPsec GPO is created and linked so that it applies to all computers in the environment. In addition, the IPsec Standard policy is assigned to this new GPO. All computers will eventually be assigned the IPsec policy. Any issues with local IPsec policies will be discovered because this domain policy will override the existing local policies. Issues are continually resolved until all subnets are listed in the Secure Subnets filter list.

Group Implementation for Woodgrove Bank
Woodgrove Bank chose to implement its production deployment by first moving all computers to the Boundary group using the buildup method. This approach allowed administrators to move forward slowly and resolve any outstanding issues without significantly affecting the communications between all systems. By first deploying a policy without any secure subnets, the administration team was able to identify any systems that had a local IPsec policy assigned and take that information for additional consideration. As subnets were added to the policy, any additional conflicts that were found were resolved. After the computers were operating under the Boundary group policy, the team implemented the Isolation domain, No fallback, and Encryption groups. Deployment of these groups used the Deploy by Group method. A set of computers were selected for a

90

Server and Domain Isolation Using IPsec and Group Policy

pilot and added to the appropriate groups that controlled the new policies. Issues were resolved as they were discovered, and additional computers were added to the groups until the groups were fully populated. The following table lists the computer groups and NAGs and their membership after the solution is fully deployed: Table 4.11 Computer and Network Access Group Membership Computer or network access group CG_IsolationDomain_Computers CG_BoundaryIG_Computers CG_NoFallbackIG_Computers CG_EncryptionIG_Computers ANAG_EncryptedResourceAccess_Users ANAG_EncryptedResourceAccess_Computers Members Domain computers IPS-PRINTS-01 IPS-LT-XP-01 IPS-SQL-DFS-01 IPS-SQL-DFS-02 User7 IPS-SQL-DFS-01 IPS-SQL-DFS-02 IPS-ST-XP-05 DNAG_EncryptionIG_Computers CG_BoundaryIG_Computers

Notice that the ANAG_EncryptedResourceAccess_Computers group contains the servers that are in the encryption isolation group. This is so they will be able to communicate with themselves and each other as required. If this communication is not required for these servers, you do not need to add them into this group.

Summary
This chapter described the design process for a server and domain isolation solution. Tasks included identifying the need for computer groups and NAGs, understanding the foundational isolation groups, adding additional isolation groups, completing a traffic model, assigning membership to the groups, and planning the deployment rollout method. This chapter also used the IPsec implementation at Woodgrove Bank, a fictitious organization, to help illustrate the design process in action and to prove the design in the Microsoft test labs. Group design was based on business requirements and the discovery information obtained from the previous two chapters and documented in the Business_Requirements.xls spreadsheet (available in the Tools and Templates folder). An appreciation for the impact of IPsec on a network was also an important consideration. After reading this chapter, you should have enough information to start planning isolation groups, documenting the communication requirements between those groups, and planning the high-level IPsec policy. These tasks will prepare you for Chapter 5, "Creating IPsec Policies for Isolation Groups."

Chapter 4: Designing and Planning Isolation Groups

91

More Information
This section provides links to additional information that may be helpful when implementing this solution.

IPsec
The following links provide a wide range of Windows-specific IPsec content: • The "Using Microsoft Windows IPSec to Help Secure an Internal Corporate Network Server” white paper presents the first model for using IPsec to secure network access to internal servers that process or store sensitive information. The Microsoft deployment of IPSec to protect all domain members is described in the technical white paper "Improving Security with Domain Isolation". The Internet Protocol Security for Windows 2000 Server page. The IPsec Web site.

• • •

Security
• The Microsoft IT security risk assessment approach is documented in the "Information Security at Microsoft Overview" white paper.

Windows Server 2003 Active Directory
For more information about Active Directory, see: • The Windows Server 2003 Active Directory page.

Chapter 5: Creating IPsec Policies for Isolation Groups
The objective of this chapter is to provide instructions for implementation of the server and domain isolation design. The previous chapters explain the design process and rationale behind the guidance that this chapter provides. If you have not already done so, it is strongly recommended that you read these chapters before continuing with this one. This chapter provides complete guidance for implementing the security requirements of domain isolation and the server isolation groups designed in Chapter 4, "Designing and Planning Isolation Groups." A combination of the following elements will implement these requirements: • Inbound and outbound access requirements for the isolation domain and isolation groups: • Internet Protocol security (IPsec) policy designed specifically for the isolation group that requires IPsec Internet Key Exchange (IKE) negotiation for inbound and outbound connections Domain-based security groups called network access groups to allow or deny network access when using IPsec-protected traffic IPsec policy filters designed to properly identify which traffic should be secured IPsec filter actions that negotiate the required level of authentication and encryption for the traffic that the filters identify IPsec filter action settings to control whether plaintext communication is allowed when trusted hosts initiate outbound connections

• •

Network traffic protection requirements for the Isolation domain and isolation groups: • • •

This guidance discusses the preparation of the solution using Group Policy and IPsec policies in the Active Directory® directory service using Microsoft® Windows Server™ 2003, and configuration of domain members using Windows Server 2003 and Microsoft Windows® XP. Additionally, this chapter discusses design alternatives and rollout options. Final check lists are provided to ensure that the design meets all of the business and security requirements.

Chapter Prerequisites
This section contains information that will help you determine your organization's readiness to implement the IPsec solution. (Readiness is meant in a logistical sense rather than a business sense—the business motivation for implementing this solution is discussed in Chapter 1, "Introduction to Server and Domain Isolation," of this guide.)

and other Active Directory objects. the use of security templates. and access control lists (ACL). . Chapter 3. or there may be fewer people spanning several roles. and network engineering administration and operations personnel Note Depending on the structure of your information technology (IT) organization. it is still possible to apply the GPO through the use of standard global and local group configurations. For more information about the enhancements made to IPsec in Windows Server 2003. "Determining the Current State of Your IT Infrastructure. and product keys. IT Infrastructure Prerequisites This chapter also assumes that the following IT infrastructure exists: • A Microsoft Windows Server 2003 Active Directory domain running in mixed or native mode. Organizational Prerequisites You should consult with other people in your organization who may need to be involved in the implementation of this solution. • Windows 2000 Server. and use of Group Policy Windows system security. Windows Server 2003 Standard Edition. There is nothing specific to this solution that would keep it from working with Windows 2000. this solution does not use it. because this option is more complex to manage. installation media. and the application of security templates using Group Policy or command-line tools An understanding of core networking and IPsec principles • Before proceeding with this chapter. Web server. groups. including Active Directory structure and tools. administration. auditing. However. including the following: • • • • Business sponsors Security and audit personnel Active Directory engineering. Familiarity with Windows Server 2003 is also required in the following areas: • • Active Directory concepts. This solution uses Universal groups for Group Policy object (GPO) application.Knowledge Prerequisites You should be familiar with general concepts of IPsec and the Microsoft implementation of IPsec in particular. see New features for IPSec. If the organization is not running in mixed or native mode. manipulating users. You also should have defined and documented the business requirements of the solution as part of the solution requirements matrix. you should also have read the planning guidance that the earlier chapters in this guide provide and have a thorough understanding of the architecture and design of the solution. Note Windows Server 2003 introduced a number of improvements that affect IPsec policies." describes the required information and how to obtain. and operations personnel DNS (Domain Name System). the solution was only tested using Windows Server 2003 Active Directory. groups. This chapter also requires a complete understanding of the existing IT infrastructure to ensure that the correct policies are deployed to the intended hosts in the environment. and Windows Server 2003 Enterprise Edition licenses. including security concepts such as users. However. these roles may be filled by a number of people.

• • • • Creating IPsec Policies in Active Directory The process of creating the necessary policies to support the required isolation groups consists of the following main tasks: • • • Create the filter lists. Create the filter actions. it is important to obtain the traffic model diagrams and tables from Chapter 4. Before undertaking the process of creating these components.Chapter 5: Creating IPsec Policies for Isolation Groups 95 You should not undertake the steps that this chapter describes until you have obtained at least the following information: • The isolation groups definition for the design. A high-level description of how the IPsec policies will change over time and a list of procedures that require IPsec policy changes. adding network components. These tables provide the necessary information to ensure that the policies provide the required functionality and are assigned to the correct isolation groups. A high-level summary of the impact of applying IPsec to enforce the isolation groups. This summary might be accompanied by a list of issues and workarounds. isolation group membership). This list would include such procedures as security incident responses. Each of the required isolation groups should have a clear statement that communicates security requirements and identifies assets to which these requirements apply (that is. ." and the host and network mapping tables. "Designing and Planning Isolation Groups. including a list of the different IPsec policies that are needed and how they will be assigned. and adding clients or servers in any isolation group. Create the IPsec policies to implement the isolation groups. An understanding of the organization's network topology and IP addressing scheme. A high-level description of how IPsec policies are used to implement the isolation groups.

Figure 5. Windows XP SP2. Compatibility of these platforms when secured using recommended hardening by the Microsoft Windows Security Guides.96 Server and Domain Isolation Using IPsec and Group Policy The following figure depicts the network configuration that was used to simulate the Woodgrove Bank scenario. The traffic maps for permitting and blocking traffic using IPsec filters were not integrated with this solution because the protection • . and Windows Server 2003 as domain members. this lab environment demonstrates the following required functionalities of Windows IPsec and test compatibility with other security technologies that would be used in real-world environments: • Compatibility of computers running Windows 2000 Service Pack (SP) 4 (with the network address translation traversal (NAT-T) update).1 Woodgrove Bank network configuration The Woodgrove Bank test lab configuration demonstrates the following key capabilities of the solution: • • Domain isolation using network access groups to block certain higher-risk but trusted hosts in the domain when using IPsec Server isolation using network access groups to limit which trusted host clients are authorized to connect using IPsec In addition.

Compatibility of IPsec encapsulated security payload (ESP) NAT-T using User Datagram Protocol (UDP)-ESP encapsulation for both of the following conditions: • • Outbound access from domain members behind NAT using IKE Kerberos authentication Inbound access to a domain member behind NAT using IKE Kerberos authentication • The lab scenario illustrated in figure 5. the No Fallback isolation group. the Isolation domain. SQL Server. and the Boundary isolation group. Additional reasons for not integrating traffic maps were to reduce complexity of the server isolation IPsec policies. IPsec Policy Component Overview An IPsec policy consists of a number of components that are used to enforce the IPsec security requirements of the organization. In total. The following figure depicts the various components of an IPsec policy and how they are associated with each other. Microsoft Operations Manager (MOM).Chapter 5: Creating IPsec Policies for Isolation Groups 97 requirements are different for isolation. four IPsec policies were created and assigned to the isolation groups shown in bold dashes in the figure (that is. and because Windows Firewall is better suited in many cases for permit/block filtering (independent of IPsec providing end-to-end security for each packet).) The following sections explain how these policies were created. Figure 5.1 was used to test that the correct functionality was achieved in all of the isolation groups for the solution. the Encryption isolation group. • IPsec application capability to secure Web (HTTP). Distributed File System (DFS). file and print sharing.2 IPsec policy components . and Microsoft Systems Management Server (SMS) servers and traffic.

If two filter lists have identical filters. Each filter in the filter list defines the following: • • • Source and destination networks or addresses Protocol(s) Source and destination Transmission Control Protocol (TCP) or UDP ports Filter lists and filter actions were designed to be shared among IPsec policies. because they are the fundamental building blocks that control which traffic is secured. Depending on the organization's business requirements and network design. Each of the rules consists of a filter list and an associated action. In addition. The following table describes some basic filter lists that might exist in a typical organization. The IPsec administrator should carefully avoid duplicate filters in an IPsec policy because the filters may have separate actions. The IPsec service may change the ordering of duplicate filters to packet processing and yield inconsistent results. filters that make up the filter lists cannot be shared between filter lists.98 Server and Domain Isolation Using IPsec and Group Policy The IPsec policy acts as a container for a set of rules that determine what and how network communications traffic will be allowed.1 Solution-Provided Filter Lists Filter list Secure Subnets List DNS Exemption List Domain Controllers Exemption List WINS Exemption List Description Contains all subnets in the organization that will be secured with IPsec Contains the IP addresses of the DNS servers that will be allowed to communicate without IPsec Contains the IP addresses of the domain controllers that will be allowed to communicate without IPsec Contains the IP addresses of the Windows Internet Naming Service (WINS) servers that will be allowed to communicate without IPsec Contains the filter that allows the Dynamic Host Configuration Protocol (DHCP) negotiation traffic across UDP 68 to occur DHCP. IPsec Filter Lists IPsec filter lists are collections of one or more filters that are used to match network traffic based on the criteria for each filter. The filter list contains a grouping of filters. additional filter lists may be required. Table 5. such as permit or block. In addition. This diagram depicts the policy components from the top down. This approach allows one filter list to be maintained for a certain type of exemption and used in the individual IPsec policy for each isolation group. once for each filter list. the filters will have to be created twice. However. the associated filter action is triggered. and performance is not affected. As traffic is matched to a specific filter. The network information gathered earlier is used to identify the various traffic patterns that the administrator wants to secure. Negotiation Traffic . the information is used to identify any traffic that might need to be exempted from the IPsec restrictions. Duplicate filters may be used if necessary when the filters have exactly the same action. the rules define which authentication methods are used between hosts. the most effective way to build policies is to start with the filters and filter lists. However.

the DNS name will be resolved and the corresponding IP address(es) will be placed in the filter. A Specific DNS Name. like the one for ICMP. When describing filters. It is important to remember that this approach means that no untrusted or non-IPsec hosts should be on those subnets. because other filters. There is no automatic way to create filters that will be kept current with the list of IP addresses for a given DNS name. the symbol "<->" is used to signify that the filter is mirrored. dialup or virtual private networking (VPN) interfaces. • • The DNS name option is useful when you need to create many filters. If the DNS server has an incorrect resource record for the DNS name specified in the filter. Any IP Address. This action is the broadest security action for all network traffic for that subnet (for example. During the creation of the initial filter. Mirroring ensures that traffic is matched when the exact opposite source and destination addresses are used. Using My IP Address causes the generic IPsec policy filters to be copied into a specific filter that contains each IP address used by the computer at the time the IPsec service prepares to enforce the policy. regardless of whether they have a static IP address or a DHCP-assigned address. This option causes the IPsec filters to match against any IP address. The result is the same as if the administrator had entered the IP address(es) into the filter(s). All Traffic Description Contains the filter that allows the Internet Control Message Protocol (ICMP) to function within the organization for troubleshooting purposes The Secure Subnets List contains all of the subnets within the organization's internal network. there will be two different IPsec-specific filters created using the two different IP addresses. the wrong IP address will be added to the filter. If a computer has one network card with two IP addresses configured for that card. such as for each domain controller in the domain. To support centralized policy assignment from the domain.Chapter 5: Creating IPsec Policies for Isolation Groups 99 Filter list ICMP. Windows XP and Windows Server 2003 have more options for addresses than Windows 2000. Note The DNS name is never evaluated after the filter(s) are created for the first time in policy. . they should be configured as mirrored. This option was designed so that a common IPsec policy in Active Directory can apply to many or all computers. IPsec does not support filter configuration for physical network interfaces. When defining the filters. This option causes IPsec to evaluate the IP address of the specified DNS name and then create the filters using that IP address or addresses. It also makes the IPsec service detect address changes or new network interfaces so that the right number of filters can be maintained. So you should use only Windows 2000 settings when this platform is a member of the domain. for example. only the type of interface such as LAN or WAN. The filters must implement both inbound and outbound security requirements. The Windows 2000 settings are explained as follows: • My IP Address. will be more specific to require a different action (such as permit). because the DNS name has many corresponding IP addresses. negotiate IPsec). Mirrored filters must be used whenever the filter action negotiates security methods for IPsec encapsulation. This filter list is associated with a filter action that implements the actions required for a particular isolation group. Source and Destination Addresses Each filter has a setting for both the source and destination addresses.

the administrator has the option to also configure source and destination ports. Because this solution assumes that communication between trusted computers is in fact trusted. Port filtering greatly increases the administrative overhead and complexity of the configuration of the IPsec filters and can require complex coordination between client and server policies for IKE to successfully negotiate security. By default the filter will match traffic on all protocols and all ports.100 Server and Domain Isolation Using IPsec and Group Policy • • A Specific Address. If a specific protocol that supports ports is selected as part of the filter criteria. locating a domain controller. In addition. Currently.xls for security requirements that are met by using the combination of IPsec and a hostbased firewall (such as Windows Firewall) positioned above the IPsec layer. each filter can be configured to match against a specific protocol or port. Computers running older versions of Windows such as Microsoft Windows NT® version 4. as well as a number of other options supported by those releases. because a rogue user spoofing an IP address from that subnet will also be exempted. or business reasons. a computer running Windows 2000 or later needs to have network connectivity and be able to obtain an IPsec policy from the domain before it can establish IPsec.0. Windows 95. see the "Best Practices" section of the IT Showcase white paper. This option allows the administrator to configure a specific subnet. Also. the filters allow all traffic (except ICMP) to be secured by IPsec. This option matches traffic to the IP address that is provided to the filter. computers that are not running the Windows operating system might not support IPsec or have IPsec easily deployed to them. this solution does not recommend creating port-specific filters. "Improving Security with Domain Isolation”. see Business_Requirements. To work around a number of the issues mentioned in Appendix A regarding the behavior of filters using "My IP Address. Exemption List Considerations Some traffic cannot be protected by IPsec for support. Protocols In addition to source and destination address configuration. technical. This approach allows the administrator to define the specific subnets that should be secured. Finally. and Windows 98 are unable to process Group Policy-based IPsec. the administrator must ensure that only Windows 2000 options are used in the policy design. Source and Destination Ports Although filters can be configured to match against the TCP or UDP ports. connecting to the network. For additional best practice recommendations in filter design. unmanaged computers that run Windows 2000 or later can only participate in the IPsec negotiation if the policy is manually rolled out to the individual computers and some form of authentication other than the Kerberos version 5 protocol (such as a preshared key or certificate) is used. Any IP address within the specified subnet will match against the filter. Any traffic outside the specified subnets will not match any IPsec filters and will be sent through in plaintext to the destination host. If the same IPsec policy will be applied to multiple platforms. A filter list is created that consists of multiple Any IP Address <-> A Specific Subnet filters in which all the organization's subnets are listed explicitly. A Specific Subnet. and retrieving the policy requires that the supporting infrastructure services be exempted from IPsec . Note Windows XP and Windows Server 2003 were enhanced to provide additional address options. especially if an exemption is created for a subnet." this solution uses Any <-> subnet filters for the Woodgrove Bank scenario. If port filtering on the trusted host is required. Care should be taken with this option.

Note that all domain controller IP addresses in all trusted forests must be exempted in order for clients in one domain or forest to be able to obtain Kerberos tickets for servers in another trusted domain or forest. Exemption lists are used to selectively opt out traffic from participating in the IPsec infrastructure by permitting traffic that matches the exemption lists' filters. These lists need to be carefully designed. Exemption lists are designed into the IPsec infrastructure to ensure that all required host communications can occur. Domain members require additional types of traffic with the domain controllers of their own domain such as server message block (SMB) TCP 445. Note Deciding whether to include servers that provide ADS. placing a server that is typically secured through the use of encryption (to protect proprietary information) in an exemption filter will allow guest computers to communicate directly with the server without using IPsec. remote procedure call (RPC). and Kerberos UDP 88 and TCP 88 traffic to both its own domain controllers and to domain controllers in other domains. RIS. If servers that provide these services exist on your network. you must implement some method to allow a communication path to be established.Chapter 5: Creating IPsec Policies for Isolation Groups 101 security. The exemption lists are implemented as filter lists to help minimize the list size for easier user interface (UI) configuration. For example. or other such services on exemption lists or make them members of the Boundary isolation group is based on factors of risk and manageability. thin clients or other bootstrap clients that need to download an image from Advanced Deployment Services (ADS) or Remote Installation Services (RIS) do not support IPsec. However. you should have a filter list that contains the filters for all domain controllers. or for domain controllers in each domain. These services include naming services such as DNS and WINS and the domain controllers themselves. there are tradeoffs in terms of the complexity and size of the IPsec policy as opposed to the security gained by using the most specific filters. Lightweight Directory Access Protocol (LDAP) UDP 389. Where security requirements are not extreme. The solution in this guide uses exemption filter lists to control these traffic requirements through permits. If a client is not able to participate in the IPsec infrastructure but has a business need to access a server that is using IPsec. other services that do not support IPsec might exist in the organization. and LDAP TCP 389. you want to permit the least amount of traffic necessary to be unprotected by IPsec. . In either case. you should examine them for inclusion in an exemption list or make them members of the Boundary isolation group so that they can accept network communications from hosts that are unable to use IPsec. the exemption is implemented for "all traffic" with the domain controller IP addresses for simplicity and to reduce the number of filters. When designing a rule to implement the exemption in IPsec policy. you should thoroughly test the approach that you have chosen. For example. The Windows Kerberos client requires ICMP. even if IPsec negotiations cannot be used. A second advantage in having several filter lists is that the permit rule can be disabled/enabled in IPsec policy easily for each filter list. because they bypass the security mechanisms that IPsec implements. In addition to these infrastructure services. For example.

src port 23.168.168.21 Any <-> 192. Placing system addresses in an exemption list effectively exempts those systems from participating as IPsec hosts for all IPsec policies that implement the exemption list. All Traffic Filters defined Any <-> 192. src port Any.21 Any <-> 192. but it would also allow an attacker to bypass IPsec authentication requirements and access any open port on the host. the servers that provide these services are usually implemented in this manner.1.168. See Microsoft Knowledge Base articles 811832 “IPSec Default Exemptions Can Be Used to Bypass IPsec Protection in Some Scenarios” and 810207 “IPSec default exemptions are removed in Windows Server 2003” for detailed discussion of the design and security impact of default exemptions. Organizations will have to carefully evaluate the security risk of such a potential attack before using this filter design.102 Server and Domain Isolation Using IPsec and Group Policy It is tempting to want to exempt particular application traffic by port instead of by destination address to avoid having to maintain a list of addresses. Woodgrove Bank Filter Lists After analyzing the traffic requirements output from Chapter 4. You should design IPsec policies with the assumption that no default exemptions are enabled. For example.22 Any <-> 192. TCP.1. consider the following outbound exemption: My IP Address to Any IP address. such as outbound Telnet using TCP port 23 to access UNIX systems.1. or WINS.168. TCP.22 Any < Any < My IP Address < My IP Address < All UDP source 68.168. DNS.2 Woodgrove Bank Filter List Examples Filter list Secure Subnets DNS Exemption List Domain Controllers Exemption List LOB Application Servers Exemption List WINS Exemption List DHCP. dest 67 ICMP only All All All All All . This same situation can exist if default exemption for the Kerberos authentication protocol is not disabled.0/24 Any <-> 192.1. The risk is certainly minimized if the destination IP address is specified.0/24 Any <-> 172. Because most clients in the organization (including guest clients) will typically need access to infrastructure services such as DHCP. mirrored The corresponding inbound filter would be as follows: Any IP Address to My IP address. dst port Any This inbound filter would allow responses from Telnet connection requests. dst port 23. Negotiation Traffic ICMP. Woodgrove Bank administrators mapped out the filter lists in the following table: Table 5.1.10.1.

The number of computers in the exemption filter lists should be kept to a minimum because all traffic is exempted to these computers.10 filter for the system hosting the LOB application. In addition. This list exempts DNS servers so that all clients in the network can perform domain name lookups. Woodgrove Bank had some line of business (LOB) applications that were running on servers that are unable to participate in the IPsec infrastructure. See the Business_Requirements. and all computers in the exemption list have full TCP/IP access to all trusted hosts in the Isolation domain. This filter list is comprised of a single mirrored filter (My IP Address <-> Any) that is configured for ICMP network traffic only.1. the Woodgrove Bank designers created a new filter for UDP port 68 to allow DHCP clients to receive traffic from the DHCP server. These filter lists are comprised of mirrored filters that define Any <-> Specific IP Address. will implement the isolation groups.0/24 Any <-> 172. regardless of whether an IPsec security association (SA) has been negotiated. the filter for DHCP was designed to match all DHCP client outbound traffic.10.1. Woodgrove took this approach because both filter lists will have the exact same action. The mirror of this filter will permit responses from DHCP servers. Domain Controllers. they created a new exemption filter list called LOB Application Servers Exemption List and added an Any <-> 192. Accordingly. and because its risk assessment did not rank highly the . WINS.168. For DHCP traffic. The first list—the Secure Subnets list—consists of two filters: • • Any <-> 192. The Woodgrove Bank designers then chose to implement an exemption list for ICMP network traffic. However. This list specifically allows host devices to look up NetBIOS names on a WINS server. Instead of having specific destination IP addresses. Woodgrove Bank chose to manage two separate lists for DNS servers and domain controllers. As discussed. the Woodgrove Bank design team identified its infrastructure services and created the corresponding exemption filter lists to allow all clients to communicate directly to the servers that provide these services. To accommodate those services. Finally. This filter list.168. which the DHCP client is using. the inbound attack is limited to destination port 68 on the client. Specific exemption lists were created for the following services: • • • DNS. this approach could present a larger attack surface than what otherwise might be present. the Woodgrove production network uses DNS servers in some areas that are not also domain controllers. when paired with the appropriate filter action. permit. This approach was also necessary because path maximum transmission unit (MTU) discovery is required for this solution to work correctly. This list allows the domain-joined systems to authenticate with a domain controller.xls spreadsheet (in the Tools and Templates folder) for details of requirements to mitigate the risk of inbound traffic from IP addresses that are exempted.0/24 These subnet filters are mirrored to match both inbound and outbound traffic and are configured to trigger on any protocol. Also. even though the IP addresses are the same.Chapter 5: Creating IPsec Policies for Isolation Groups 103 The Woodgrove Bank designers followed the guidance provided in this chapter to create these filter lists.1. regardless of the IPsec settings for the isolation groups. This approach allows administrators to use the Ping utility as a troubleshooting tool in the environment. and these filters are configured to trigger on any protocol. Woodgrove Bank used this design to avoid having filters for every DHCP server. it may also permit inbound attacks from any IP address using source port 67.

If the filter criteria are met by outbound packets. the packet will be allowed to pass the IPsec layer without any further processing by IPsec. For IP packets matching the associated filter. The following table lists the possible cryptographic options for each security method: Table 5. Security Methods The security methods that are implemented within the filter action are determined by the requirements for processing packets that match the associated filters in the filter list. not the information in the filter list. This section does not include the full description for each filter. Name Give the filter action a meaningful name that reflects what the filter action does. because the IPsec monitor and command-line tools display each filter's description. the packet is discarded or ignored. The following three options are available: • • • Block. description. Filter actions are the basis for implementing the various isolation groups. For IP packets matching the associated filter. whether to enable encryption. IPsec will attempt to negotiate one of the security methods that are in the filter action based on its relative order. Each security method can define whether to use integrity. the packet will be blocked. The higher the security method is in the list.104 Server and Domain Isolation Using IPsec and Group Policy risks of inbound attacks to the DHCP client port or the risk of unauthorized DHCP servers. Description Type a detailed description of the filter action behavior in the description field. and security method. Each filter action is comprised of a name. However.3 Security and Cryptographic Options Security method Authentication Header (AH) Cryptographic options MD5 SHA-1 Description Provides both IP payload (data) and IP header (address) integrity and authenticity without encryption. AH cannot traverse devices running NAT. but always respond with IPsec. Negotiate. . the higher preference it has. and which cryptographic algorithms provide the functionality. Although there are three default filter actions provided with the Windows operating system. it is recommended that you remove these and create new filter actions this approach allows you to ensure that only the actions that you create as part of your design are being used. IPsec Filter Actions The filter actions define how IP packets are handled after they are matched to a filter within a filter list. Permit. In other words. it is recommended that you use the filter description field to carefully define each filter. The handling of inbound packets that match a filter with a negotiate action is determined by the setting for Accept unsecured communication.

using three unique 56-bit keys.. IKE automatically detects when NAT exists in the path and uses UDP-ESP whenever ESP is allowed in the security method list. provides IP payload (data) encryption. With DES or 3DES. The following two cryptographic options are available: • DES.S. you can configure the security method to use AH with SHA-1 integrity and ESP – 3DES Encryption. Use of ESP without authentication is not recommended. 3DES. This hash algorithm uses a 128-bit cryptographic key to generate a digest of the packet contents. This hash algorithm uses a stronger 160-bit cryptographic key to generate a digest of the packet contents. The greater key length of SHA-1 provides stronger security. The NAT-T update is required for Windows 2000 SP4. . • You can combine AH and ESP protocols with one another if required to meet very high security requirements. it is strongly recommended that you use 3DES because it is more secure. This option uses a single 56-bit key and processes each block once. NAT-T techniques use a UDP header after the IP header to encapsulate ESP. in which case only data integrity and authenticity is enforced. Because of advances in the ability of attackers to compromise DES encryption. if there is a clear requirement for IP header integrity in addition to data encryption. then you can use ESP-null with SHA-1. MD5 is not an approved algorithm for U. federal security regulations. This option processes each block three times. 3DES is somewhat slower in performance and has higher processing overhead than DES. Although DES is supported. If only data integrity is required. ESP can also encrypt the data section of the IP packet. and it provides the ability to authenticate the hosts prior to establishing a communication connection and performing an integrity check on the data part of the IP packet carried within the ESP packet.S. However. This will change in future versions of Windows. • ESP can be configured to use no encryption algorithm. The Windows 2000 and Windows XP support for IPsec transport mode NAT-T is limited for Windows 2000 and Windows XP versions prior to SP2 because TCP Path MTU (PMTU) detection is not supported for IPsec-protected TCP traffic. Windows Server 2003 does have this support. Windows XP SP2. This configuration is commonly called ESP-Null. federal security scenarios.S. SHA-1 is an approved algorithm for U. The following cryptographic options are available for AH and ESP: • MD5. ESP – Encryption <None> 3DES DES The Windows 2000 SP4.Chapter 5: Creating IPsec Policies for Isolation Groups 105 Security method ESP – Integrity Cryptographic options <None> MD5 SHA-1 Description Provides data integrity and authenticity for the IP payload (data) only. SHA-1. For example. Can be used with a null encryption algorithm when encryption is not necessary. Also. using DES is not recommended. and Windows Server 2003 IPsec implementations now support NAT-T techniques for IPsec transport mode ESP. Federal government Advanced Encryption Standard (AES). in addition to supporting NAT-T for L2TP/IPsec VPN client tunnels. It can be used with or without encryption. it is worth noting that current Windows IPsec implementations do not support the U. although it does have higher processing overhead. DES is provided for Request for Comment (RFC) compliance.

Note If the policies assigned to the clients do not enable the default response rule. and you can write specific security requirements that define an appropriate level of protection against sophisticated cryptographic attacks. it can be used as a denial of service attack vector. . Using AH and ESP to provide integrity does not provide any additional integrity protection for packet data. it attempts to renegotiate well before either the lifetime for bytes or seconds expires. For example. Careful planning is required when designing the security methods of the filter actions. For two computers to successfully negotiate. To enable Inbound passthrough. it just increases the workload associated with processing the packet. but the filter action may also contain a negotiation method for ESP-3DES. ESP combined with AH will not overcome the NAT barrier issues that AH faces. You should not enable this option should on Internet-facing computers. Similarly. but always respond using IPsec in the Manage Filter Actions dialog box. The server's connection reply matches the outbound filter to trigger an IKE main mode negotiation request to the client. CPU usage may be increased for servers maintaining IPsec SAs with many clients. you should disable this option. they need to have at least one security method in common in their respective filter actions. This approach will allow the system to negotiate a 3DES encryption connection if required. With this option enabled. the IPsec SA lifetimes are configured to be when 100 megabytes (MB) of data are passed or after one hour has elapsed. more information is available to the attacker about the session key. because there will be no response for IPsec communication. then packets can be lost. IPsec must discard packets if the lifetime expires. Each filter action may contain more than one negotiation method to accommodate different types of negotiation. it is important to recognize that AH provides both data and address header integrity. In addition to selecting the security methods. The IKE quick mode process is called "rekeying" but does not actually just refresh keys for an existing IPsec SA pair. Renewing IPsec SAs ensures that an attacker is unable to decipher the entire communication even if they manage to determine one of the session keys. a system may typically only negotiate ESP-Null. a plaintext connection request that matches an inbound filter will be accepted. By spoofing source IP addresses. Therefore. If the lifetime is set too low. As the lifetime increases. using AH in addition to ESP is appropriate for only the highest security environments. because it allows inbound attacks to pass through the IPsec layer. In addition. Therefore. select Accept unsecured communication.106 Server and Domain Isolation Using IPsec and Group Policy Although you can select any combination of the security options. It also forces the server to attempt an IPsec SA negotiation to the source IP of any incoming packet. you should not change the lifetimes unless operational needs require it. Security Negotiation Options You can set the following security negotiation options for IPsec policies: • • • Inbound passthrough Fall back to clear Use session key perfect forward secrecy Inbound Passthrough The Inbound passthrough option was designed to be used in an internal server policy so that client policy could use the non-intrusive "default response" rule. In addition. therefore. the attacker can cause denial of service on the server as IKE tries to negotiate with hundreds or thousands of invalid IP addresses. you can set the session key settings for each security method if required. In its default setting. These settings control when a new pair of IPsec security associations is renegotiated by IKE quick mode.

but always respond using IPsec is not selected. If there is no response from the target computer in three seconds. a soft security association (soft SA) will be created. the master key can only be used once. Or the active IPsec policy may not be designed to negotiate with the source computer's IP address. but always respond using IPsec is selected). using this option also helps re-establish plaintext connectivity temporarily when the IPsec service is stopped for troubleshooting. select Allow unsecured communication with non-IPsec-aware computers in the Manage Filter Actions dialog box. it is recommended that this option be enabled so that clients can communicate with hosts that do not have IPsec enabled. Or the active IPsec policy might only have permit and block actions. If the system is running Windows 2000 or Windows XP without the appropriate service packs. effectively blocking the communication. network traffic that does not use IPsec is referred to as in plaintext. IPsec on the source computer will discard the outbound packets. the lack of an IKE main mode response does not necessarily mean that the computer is not capable of IPsec. Use Session Key Perfect Forward Secrecy The Use session key perfect forward secrecy (PFS) option determines whether the master key material can be used to generate all the session keys or just the first session key. For initial deployments. To enable the Fall back to clear option. This behavior occurs because when the Allow unsecured communication with non-IPsec-aware computers option is selected. and Session key and Master key PFS options. Windows XP SP1. it might be an IPsec-capable computer that does not have an active IPsec policy. see the "Security Negotiation Options" section in Using Microsoft Windows IPSec to Help Secure an Internal Corporate Network Server. IPsec processes the associated inbound filter as an Inbound passthrough filter (the same behavior that occurs when Accept unsecured communication. but always respond using IPsec option and the Allow unsecured communication with non-IPsec-aware computers option. Note The way this option functions has changed on computers that run Windows 2000 SP3 or later. Fall back to clear. Enabling this option is not recommended because of the additional overhead cost of performing a key exchange at each session key renewal interval. you must select both the Accept unsecured communication. the attacker cannot generate additional session keys to decrypt the traffic stream. or Windows Server 2003. This requirement ensures that if the master key is compromised. For more information about the Inbound passthrough. However. If the target computer does provide an IKE response and there is a failure during the IKE negotiation for any reason. If the system needs to respond to requests from and initiate communication to non-IPsec aware systems. the client will accept unsecured communication requests when Allow unsecured communication with non-IPsec-aware computers is selected. even when Accept unsecured communication. In IPsec terminology. These hosts are referred to as being "non-IPsec-aware" computers. and each additional session key renegotiation will require a new key exchange to be performed to generate a new master key before generating the session key. and communication will begin in plaintext. If this option is enabled.Chapter 5: Creating IPsec Policies for Isolation Groups 107 Fall Back to Clear The Fall back to clear option controls the ability for the (source) computer to allow traffic to be sent without IPsec protection if the initial IKE main mode negotiation does not get a response from a target destination computer. Also. the system will be able to initiate communication in the clear but will not accept any communication requests from non-IPsec aware systems. Hosts that do not support IPsec will not be able to reply (using IKE) to the IKE negotiation request. If only this option is enabled. .

4 are used to implement the isolation groups in the Woodgrove Bank scenario.5 Supported Security Methods Filter action IPsec – Request Mode (Accept Inbound. This filter action is used to configure the Boundary isolation group. The permit filter action will allow the traffic to occur for any associated filter list that has the matching filter. For outbound traffic. Woodgrove Bank administrators have four security isolation groups to implement. A host accepts inbound packets that are either IPsec or plaintext. The bank has determined that the four negotiated filter actions in the following table are sufficient to implement its environment: Table 5.108 Server and Domain Isolation Using IPsec and Group Policy Woodgrove Bank IPsec Filter Actions The following table provides the filter action names and descriptions that are used to implement the various isolation groups for the Woodgrove Bank scenario: Table 5. For outbound traffic. A host allows inbound TCP/IP access only when packets are secured by IPsec and ignores non-IPsec inbound packets. This filter action is used to implement the Isolation domain. To deploy this configuration. The block filter action will drop any traffic that matches a filter in a filter list associated with this action. This filter action is used to implement the No Fallback isolation group where all communications are protected by IPsec. Woodgrove Bank has no additional requirements for isolating computers from each other within a specific security isolation group. Disallow Outbound) The first two filter actions are straightforward. IPsec – Full Require Mode (Ignore Inbound. A host requires IPsec-secured communications for both inbound and outbound packets. A host allows inbound TCP/IP access only when packets are secured by IPsec ESP 3DES encryption and ignores non-IPsec inbound packets. Allow Outbound) IPsec – Secure Request Mode (Ignore Inbound. <None> ESP – SHA-1. For outbound traffic. you must define a minimum of three filter actions with custom security negotiation methods in addition to the permit and block filter actions. 3DES . it triggers an IKE negotiation. it triggers an IKE negotiation and allows Fall back to clear if no response. Allows the traffic that matches the filter. Allow Outbound) Description Blocks the traffic that matches the filter. The final four filter actions in Table 5.4 IPsec Filter Actions and Descriptions Filter action IPsec – Block IPsec – Permit IPsec – Request Mode (Accept Inbound. it triggers an IKE negotiation that requires IPsec ESP 3DES encryption. and allows Fall back to clear if no response. This filter action is used to implement the Encryption isolation group. Allow Outbound) Security methods supported ESP – SHA-1. Disallow Outbound) IPsec – Require Encryption Mode (Ignore Inbound. where outbound connections to untrusted hosts are allowed.

S. the group membership of a computer using a certain IP address is not known at the time the initial connection request is sent outbound. <None> ESP – SHA-1. you cannot use IP addresses to define or approximate isolation groups. Remember that IPsec policy uses filters and filter actions to intercept and control inbound and outbound IP packets. Isolation Domain Filter Action To implement the Isolation domain. Allow Outbound) filter action. Disallow Outbound) IPsec – Require Encryption Mode (Ignore Inbound. 3DES ESP – SHA-1. all policies needed the option to use encryption.5 and 4. Woodgrove Bank chose to use SHA-1 instead of MD5 for its stronger security. 3DES ESP – SHA-1. For ESP integrity. but also because it is required to meet U. Woodgrove also made this decision to avoid the performance impact on the computers caused by the key renegotiation. Therefore. Woodgrove Bank also had a requirement to implement encryption for some of the servers in the organization. Therefore. Also. <None> ESP – SHA-1. Disallow Outbound) Security methods supported ESP – SHA-1. computers that are members of these isolation groups may be located anywhere on the internal network. the Woodgrove Bank administrators created the IPSEC – Secure Request Mode (Ignore Inbound. Allow Outbound) IPsec – Full Require Mode (Ignore Inbound. government regulations on processing financial information that included using approved algorithms. Woodgrove Bank chose not to implement PFS on any of the filter actions because it did not have a specific security threat that required the use of PFS. IPsec and IKE cannot be configured specifically to initiate communications in a certain way with a particular identity or isolation group.6 of Chapter 4 of this guide: • • • • • • • Initiate communications with hosts in the No Fallback isolation group Accept communications from hosts in the No Fallback isolation group Initiate communications with hosts in the Encryption isolation group Accept communications from hosts in the Encryption isolation group Initiate communications with hosts in the Boundary isolation group Accept communications from hosts in the Boundary isolation group Initiate communication to untrusted systems Clients in the Isolation domain cannot accept communications from untrusted systems. 3DES Woodgrove uses IPsec ESP instead of AH because of the presence of network devices in the organization that use NAT. Woodgrove Bank has several business requirements for communication between hosts in the Isolation domain and other isolation groups. For these reasons. Therefore. . clients in the Isolation domain perform the following actions described in the tables 4. Woodgrove Bank chose to implement its security based solely on IPsec ESP. Although IKE does authenticate both computers. Accordingly.Chapter 5: Creating IPsec Policies for Isolation Groups 109 Filter action IPsec – Secure Request Mode (Ignore Inbound.

clients in the Boundary isolation group can perform the following actions described in tables 4. If a trusted host in the Isolation domain initiates an outbound connection to an untrusted host (or another non-IPsec-aware system). Allow Outbound) filter action is configured to allow the members of the Isolation domain to initiate communications with untrusted systems. such as stateful filtering. the security methods include the security methods (ESP–SHA-1–3DES algorithms) that are defined for the Encryption isolation group. Accordingly.5 and 4. IPsec filtering and soft SA support was not designed to provide connection-specific protections. Allow Outbound) filter action. like many firewalls do. This behavior is implemented by enabling the Allow unsecured communication with non-IPsec-aware computers option on the filter action. and including ESP–SHA-1–3DES. preferring ESP-null. For more information about this process. see the "IKE Main Mode SAs and IPsec SAs" section of Appendix A. Allow Fall back to clear if a target destination host does not respond with IKE. The IPSEC – Secure Request Security (Ignore Inbound. see the "Encryption Isolation Group Filter Action" section later in this chapter. Therefore. trigger IKE negotiation requests that attempt to secure traffic with IPsec ESP. Woodgrove Bank uses ESP-null with SHA-1. Soft SAs allow all traffic that matches the associated filter. that untrusted system is able to initiate new plaintext connections back into the trusted host during this time. ignore traffic if it is not already secured inside valid IPsec ESP packets. "Overview of IPsec Policy Concepts.110 Server and Domain Isolation Using IPsec and Group Policy In order to implement these requirements in an IPsec policy. for packets matching the corresponding filters (all internal subnets)." Boundary Isolation Group Filter Action To implement the Boundary isolation group. For more information about the encryption security negotiation methods. Woodgrove Bank has several business requirements regarding communication between hosts in the Boundary isolation group and other isolation groups. • To enable communications with the Encryption isolation group. the IPsec soft SA is established and remains active for five minutes after traffic stops flowing. You can reduce the preceding requirements into two basic behaviors: • Outbound. Inbound. but always respond with IPsec option on the filter action so that inbound plaintext will be ignored. the Woodgrove Bank administrators created the IPSEC – Request Mode (Accept Inbound. the trusted host will no longer accept plaintext traffic from that system.6 of Chapter 4: • • • • • • • Initiate communications with hosts in the No Fallback isolation group Accept communications from hosts in the No Fallback isolation group Initiate communications with hosts in the Isolation domain Accept communications from hosts in the Isolation domain Accept communications from hosts in the Encryption isolation group Initiate communications with untrusted systems Accept communications from untrusted systems . for packets matching the corresponding filters (all internal subnets). and with the Boundary and No Fallback isolation groups. After the soft SA times out. This approach ensures that hosts in the Isolation domain will not accept traffic from any computer that is not participating in the IPsec environment. you must design the filter action to work with the filter list that specifies all internal subnets. For traffic within the Isolation domain. You must disable the Accept unsecured communication.

No Fallback Isolation Group Filter Action To implement the No Fallback isolation group. It is possible to reduce the listed requirements into two basic behaviors: • Outbound. and including ESP–SHA-1–3DES. Hosts in the Boundary isolation group are allowed to communicate with untrusted systems. and including ESP–SHA-1–3DES. Do not allow Fall back to clear if a target destination host does not respond with IKE. Inbound. clients in the No Fallback isolation group can perform the following actions: • • • • • • Initiate communications with hosts in the Isolation domain Accept communications from hosts in the Isolation domain Initiate communications with hosts in the Encryption isolation group Accept communications from hosts in the Encryption isolation group Initiate communications with hosts in the Boundary isolation group Accept communications from hosts in the Boundary isolation group Clients in the No Fallback isolation group can neither initiate communications to nor accept communications from untrusted systems. the Woodgrove Bank administrators created the IPSEC – Full Require Mode (Ignore Inbound. Accordingly. preferring ESP-null. To facilitate this capability. Allow Fall back to clear if a target destination host does not respond with IKE. Disallow Outbound) filter action. ignore them. Woodgrove Bank ensured that the security negotiation methods used for the Isolation domain and No Fallback isolation group are present in the filter action. Woodgrove Bank has several business requirements for communication between hosts in the No Fallback isolation group and other isolation groups.Chapter 5: Creating IPsec Policies for Isolation Groups 111 In order to implement these requirements in an IPsec policy. the security negotiation methods include the encryption negotiation methods that are defined for the Encryption . accept plaintext packets that match the corresponding filters (all internal subnets). It is possible to reduce the listed requirements into two basic behaviors: • Outbound. for packets matching the corresponding filters (all internal subnets). • To meet the requirements for initiating and accepting traffic to or from the Isolation domain and the No Fallback isolation group. preferring ESP-null. for plaintext packets matching the corresponding filters (all internal subnets). Inbound. trigger IKE negotiation requests that attempt to secure traffic with IPsec ESP. The common security negotiation method that Woodgrove selected is ESP with SHA-1 for integrity. • To enable communications with the Encryption isolation group. for packets matching the corresponding filters (all internal subnets). Woodgrove Bank ensured that the hosts will accept inbound unsecured traffic and be able to Fall back to clear for outbound unsecured traffic. you must design the filter action to work with the filter list that specifies all internal subnets. By enabling both of these options. To implement these requirements in an IPsec policy. you must design the filter action to work with the filter list that specifies all internal subnets. trigger IKE negotiation requests that attempt to secure traffic with IPsec ESP. the Woodgrove Bank administration team enabled both the Allow unsecured communication with non-IPsec-aware computers and Accept unsecured communication. but always respond with IPsec options for this filter action.

Do not allow Fall back to clear if a target destination host does not respond with IKE. For more information about the encryption security negotiation methods. hosts in the Encryption isolation group need to communicate with the hosts in the Isolation domain and No Fallback isolation group. For traffic to the Boundary isolation group and No Fallback isolation group. The IPSEC – Full Require Mode (Ignore Inbound.112 Server and Domain Isolation Using IPsec and Group Policy isolation group. for packets matching the corresponding filters (all internal subnets). ignore them. the encryption security negotiation methods used in the IPSEC – Require Encryption Mode (Ignore Inbound. This approach ensures that hosts in the No Fallback isolation group must secure all inbound and outbound traffic with IPsec. see the following "Encryption Isolation Group Filter Action" section in this chapter. The filter action was configured to include the security negotiation methods that encrypt the information. the filter action must be designed to work with the filter list that specifies all internal subnets. Allow Outbound) and the IPSEC – Full Require Mode (Ignore Inbound. trigger IKE negotiation requests that attempt to secure traffic only with IPsec ESP–SHA-1– 3DES. It is possible to reduce the listed requirements into two basic behaviors: • Outbound. for plaintext packets matching the corresponding filters (all internal subnets). • Clients in the Encryption isolation group cannot accept communications from the Boundary isolation group and can neither initiate communications with nor accept communications from untrusted systems. Disallow Outbound) filter action will not allow a computer that uses this filter action to initiate communication to a computer that does not participate in the IPsec infrastructure. Inbound. Woodgrove Bank uses ESP with SHA-1 for the integrity security negotiation method. Encryption Isolation Group Filter Action Woodgrove Bank chose ESP as its base integrity protocol and SHA-1 as the cryptographic option. To implement these requirements in an IPsec policy. Woodgrove Bank has several business requirements regarding the communication between hosts in the Encryption isolation group and other isolation groups. To enable communications with the Isolation domain. Accept only IKE negotiation requests from trusted hosts that allow IPsec ESP–SHA-1–3DES. The Allow unsecured communication with non-IPsec-aware computers option was disabled to enforce this requirement. Disallow Outbound) are also present in the IPSEC – Secure Request (Ignore Inbound. Accordingly. clients in the Encryption isolation group can perform the following actions from table 4. Disallow Outbound) .6: • • • • • Initiate communications with hosts in the Isolation domain Accept communications from hosts in the Isolation domain Initiate communications with hosts in the No Fallback isolation group Accept communications from hosts in the No Fallback isolation group Initiate communications with hosts in the Boundary isolation group Computers in the Encryption isolation group are prohibited from accepting communications from hosts in the Boundary isolation group. but always respond with IPsec checkbox on the filter action is cleared to create the No Fallback isolation group. In addition. The Accept unsecured communication. They will not accept traffic from any computer that is not participating in the IPsec environment.

The dynamic policy will be discarded when the IPsec service stops. Persistent IPsec Policy. Stored and managed in registry on local computer. Stored in Active Directory. Active Directory IPsec policies are assigned to a GPO using the Group Policy Editor MMC snap-in or the Group Policy Management Console under Windows Settings. Managed by IPsec Policy Management MMC snap-in or command-line tool. Blocking IPsec communications from the Boundary isolation group computers required additional configuration. Active Directory Domain IPsec Policy. but always respond with IPsec option. Woodgrove Bank did not enable the Accept unsecured communication. The Woodgrove Bank administrators configured the GPO that delivers the IPsec policy for the Encryption isolation group computers with the "Deny access to this computer from the network" right. This right was applied to a group that had all of the computer accounts for the systems participating in the Boundary isolation group. which provides better security than DES but at the cost of additional overhead. Used to dynamically add to the existing policy. this guidance focuses on using Active Directory Domain IPsec Policy. Supported only in Windows XP SP2 or later. it is best to try to design a generic policy that will establish a foundation for the IPsec infrastructure for all computers. Applies in addition to persistent policy if no domain policy is assigned. Configured by IPsec Policy Management MMC (Microsoft Management Console) snap-in or command-line tool. Dynamic IPsec Policy. Because of the requirement to neither accept nor initiate communications with untrusted systems. Stored only in memory. Configured by command-line tool. This configuration ensures that hosts in the Encryption isolation group will not accept traffic from any computer that is not participating in the IPsec environment. An IPsec policy is a collection of rules against which traffic is matched. the IKE authentication would result in an authorization failure. Applies as soon as the computer obtains an IP address. Windows XP and Windows Server 2003 support the following additional policy types: • • • • For simplicity. Stored and managed in local registry. the Allow unsecured communication with non-IPsec-aware computers option was also disabled to prevent computers from attempting to initiate communications with any computer that was not participating in the IPsec environment. Replaced when service applies persistent policy. IPsec Policies. Configured by command-line tool. Then you can create additional policies to enforce more stringent settings on systems that need additional security . There are three different types of IPsec policy for Windows 2000: • • • • Local Policy Active Directory Domain Policy Dynamic Policy Startup IPsec Policy. If any of these computers attempted to initiate communications with a system in the Encryption isolation group. In addition. Replaces the startup policy. IPsec Policies IPsec policies configure Windows-based computers to function in an IPsec environment. Stored and managed on local computer. Applies first when the IPsec service starts. and the communication would be blocked. Overrides any local policy that may have been assigned. Security Settings. Local IPsec Policy. Woodgrove Bank uses the 3DES encryption method.Chapter 5: Creating IPsec Policies for Isolation Groups 113 filter actions. which can be before the IPsec service starts. When defining IPsec policies.

an associated filter action.041001. IPsec uses transport mode. The connection type defines the connections to which the IPsec policy applies. Rules An IPsec rule consists of a single filter list.114 Server and Domain Isolation Using IPsec and Group Policy configuration.1600) Description This policy defines the Isolation domain. This policy defines the Boundary isolation group. Keeping the total number of policies to a minimum will make it easier to troubleshoot and manage the policies. description.041001. You can configure the policy to apply to all connections. The tunnel type defines whether the IPsec policy defines an IPsec tunnel. set of rules. Name You should give policies. meaningful names to help in the management and troubleshooting of the solution during both the implementation and operations phases of the project. like filter actions. If negotiation fails between IPsec-aware clients.6 Woodgrove Bank IPsec Policies Policy name IPSEC – Isolation Domain IPsec Policy (1. It configures hosts to require IPsec communication. Each rule defines one or more authentication methods to use for establishing trust between the hosts. It configures hosts to request IPsec communications but allows them to Fall back to clear if communications need to occur with a non-IPsec-based host. and configuration settings for polling intervals. To support the security isolation groups identified earlier in this guide. and key exchange methods. the communication will fail.0. Woodgrove Bank implemented four IPsec policies. IPSEC – Boundary Isolation Group IPsec Policy (1. The following table shows the policies used in the Woodgrove Bank scenario: Table 5. key exchange settings. the connection type. the authentication methods used to establish the trust between computers.0. and did not define an IPsec tunnel. IPsec policies comprise a name. applied to all connections. The options are the Kerberos version 5 protocol. certificates from a specific certification authority. Description A detailed description of the policy will help administrators identify what the policy enforces without having to actually open the policy and study its rules. It configured all four policies so that they used the Kerberos version 5 authentication protocol. local area connections. If the tunnel type is disabled. You should design each additional policy to affect the greatest number of computers that need to meet the particular business or technical requirement. and whether the rule is a tunnel configuration.1600) . all of which the following section discusses in more detail. or remote accessbased connections. Hosts in this isolation group are able to Fall back to clear when initiating communications with non-IPsec hosts. and preshared keys.

0.1600) Filter list Woodgrove Bank Secure Subnets Filter action IPsec – Secure Request Mode (Ignore Inbound. the communication will fail.041001.7 Common Rules Defined in Woodgrove Bank IPsec Policies Filter list DNS Exemption List Domain Controllers Exemption List WINS Exemptions List DHCP. the default client response rule in each of the policies is disabled. It configures hosts to require IPsec communication and encryption. If negotiation fails or communication is attempted with a client that does not use IPsec. This policy defines the Encryption isolation group. If negotiation fails or communication is attempted with a client that does not use IPsec. because there are no requirements for exempting a special set of computers for one particular isolation group.Chapter 5: Creating IPsec Policies for Isolation Groups 115 Policy name IPSEC – No Fallback Isolation Group IPsec Policy (1.041001.0. The number associated with each policy name is a version number and is discussed later in the "Policy Versioning" section.8 Woodgrove Bank Base Rules for Implementing Isolation Groups Policy name IPSEC – Isolation Domain IPsec Policy (1. and the connection type is set to All. Negotiation Traffic ICMP. Each of the Woodgrove Bank policies contains the same exemption lists. the communication will fail.1600) IPSEC – Encryption Isolation Group IPsec Policy (1. the tunnel endpoint is set to None. It configures hosts to require IPsec communication. The following table shows the Woodgrove Bank rules for implementing the four isolation groups: Table 5.041001. All Traffic Filter action IPSEC – Permit IPSEC – Permit IPSEC – Permit IPSEC – Permit IPSEC – Permit Authentication methods None None None None None Tunnel endpoint None None None None None Connection type All All All All All In addition to the rules listed in this table. The following table shows the enabled rules that are the same across the four policies identified in the previous table: Table 5.1600) Description This policy defines the No Fallback isolation group. Allow Outbound) . the authentication method is set to Kerberos version 5 protocol. The four policies that Woodgrove Bank defined only differ in how they handle the remaining traffic that is not handled by any of the exemption filter lists. For each of these rules.0.

1600) IPSEC – Encryption Isolation Group IPsec Policy (1. Polling Intervals There are two polling intervals to consider: the Group Policy polling interval. Key Exchange Settings The following key exchange settings define how new keys are derived and how often they are renewed.0. all domain-joined computers are able to participate in the IPsec infrastructure because they can authenticate and obtain policy. This increased polling frequency introduced additional polling traffic in the form of LDAP queries from the client to check the timestamps on the IPsec policies.041001. Woodgrove did not choose certificates because the bank does not have a deployed public key infrastructure (PKI). Changes to the computer's OU membership and assignment of GPOs are detected through the Group Policy service polling. Disallow Outbound) IPsec – Require Encryption Mode (Ignore Inbound.041001. Woodgrove Bank treats non-domain computers as untrusted at the current time. The default setting for the IPsec service policy change polling interval is 180 minutes between consecutive polls for changes in Active Directorybased IPsec policies. in deployments with a large number of clients. if a UNIX system is configured properly to use Active Directory as its Kerberos realm and the IKE implementation supports Kerberos authentication. For example. Allow Outbound) IPsec – Full Require Mode (Ignore Inbound. policies could be updated and deployed within one hour to mitigate the risk. and the IPsec service polling interval. Woodgrove did not use preshared keys because the authentication key value can be read by local administrators in the registry. Although this did not introduce a significant overhead in the Woodgrove Bank scenario. If these computers meet trusted host criteria. and by any authenticated user and computer in the domain. you can configure an IPsec local policy using certificate authentication to enable them to communicate with other trusted hosts.041001.0.1600) Woodgrove Bank Secure Subnets Woodgrove Bank Secure Subnets Woodgrove Bank chose to use the Kerberos version 5 protocol as its only authentication protocol.1600) Secure Subnets IPSEC – No Fallback Isolation Group IPsec Policy (1. which occurs every 90 minutes by default. Computers that are not joined to the domain are not able to easily participate in the IPsec environment because of the lack of an authentication mechanism and policy distribution system. Disallow Outbound) IPSEC – Boundary Isolation Group Woodgrove Bank IPsec Policy (1. Note Using the Kerberos version 5 protocol for IKE authentication does not prevent nondomain-based computers from participating in the IPsec environment. Woodgrove Bank chose to set both polling intervals to 60 minutes so that in case a security response was needed.0. then it may be possible for it to participate in the Isolation domain. The term "master key" means the Diffie-Hellman shared secret key material generated in IKE main mode. such a configuration is beyond the scope of this document and has not been tested by Microsoft. By using the Kerberos version 5 protocol. The term "session key" refers to keys generated . they do not detect changes in domain or organizational unit (OU) membership or the assignment or removal of an IPsec policy in a GPO. However.116 Server and Domain Isolation Using IPsec and Group Policy Policy name Filter list Filter action IPsec – Request Mode (Accept Inbound. These polls check only for changes in the IPsec policy. this increase may become significant.

Similarly. This setting controls the maximum number of IKE quick modes allowed during the lifetime of one main mode SA. Session keys are derived from the master key. a new IKE main mode SA will be negotiated that will generate a new master key. Session key PFS generates a new master key during quick mode (without main mode authentication) and then derives new session keys from the new master key. which means that there is no limit. It controls how long the master key and trust relationship can be used before having to be renegotiated. the reverse can occur. If you reduce the lifetime. The default setting is 0. You should use Session key PFS only where IPsec protected traffic is at risk of sophisticated cryptographic attacks on the Diffie-Hellman master key. Unlike soft SAs. Woodgrove Bank administrators reduced the IKE main mode SA lifetime to 20 minutes to reduce the attack surface presented by resident main mode SAs that were negotiated with the Encrypted isolation group. This value sets the IKE main mode SA lifetime. This risk is reduced by forcing the main mode SAs on the boundary servers to be deleted more frequently. Main mode PFS is not recommended because the functionality is duplicated by other supported key exchange settings. . Session key PFS is supported as a checkbox in the security methods for a filter action. When this limit is reached. To achieve the same behavior as the Master key PFS setting. This additional protection comes at a cost of additional overhead. thereby limiting the number of session keys that can be generated from the same master key. The first TCP/IP connection from a unique client to the host causes a new IKE main mode SA to be created. IKE quick mode PFS was not used in the filter actions. There are two types of PFS in IKE. you can increase CPU load required to renegotiate main mode SAs for clients that communicate frequently. • Perfect forward secrecy (PFS). Main mode PFS requires that IKE should reauthenticate and negotiate a new master key each time a quick mode is performed to refresh session keys. • • Woodgrove Bank chose not to use Master key PFS because it did not have specific security requirements that required it.Chapter 5: Creating IPsec Policies for Isolation Groups 117 by IKE quick mode for use in the IPsec integrity and encryption algorithms. However. unless quick mode PFS is used. After the main mode SA has been established. and a smaller amount of encrypted data would be revealed if an attacker were able to discover the master key. the master key is only refreshed when the IKE main mode SA lifetime expires. this option is set to 1. Therefore. The number of sessions for which the master key can be used to generate a session key was left at the default setting of 0. Authenticate and generate a new key after every <number> sessions . Main mode SAs take approximately 5 kilobytes of memory each. This reduces memory utilization and saves IKE processing time to maintain fewer SAs. the two computers start from the beginning with a new IKE main mode and quick mode negotiation. By adjusting this value. Authenticate and generate a new key after every <number> minutes . This requirement ensures that every time any cryptographic key needs to be refreshed. Although hosts in the Boundary isolation group cannot initiate new IKE negotiations with hosts in the Encrypted isolation group. For the Boundary isolation group. which is 480 minutes by default. Master key PFS (which means main mode PFS). and Session key PFS (which means quick mode PFS). This functionality ensures that a large amount or all data in the communication is not protected by just one master key value. then a boundary host would be able to use this SA to negotiate quick mode SAs for the protection of inbound traffic to the corresponding system in the Encrypted isolation group until the main mode SA was deleted. the administrator can optimize CPU load and memory utilization required by IKE. you reduce the number of active main mode SAs on the server. The IKE main mode SA lifetime was changed from 480 minutes to 180 minutes to more quickly delete main mode SAs on busy servers in all but the Boundary isolation group. main mode SAs are not removed from the host after five minutes of inactivity.

which will ignore the 2048-bit option. Troubleshooting steps will require the ability to determine which version of the IPsec policy is active on the computer. This option corresponds to the IETF RFC 3526 specification for Diffie-Hellman group 14. and while in operational use. Low is provided for backward compatibility and should not be used because of its relative weakness. This key strength is required for 3DES to have maximum cryptographic strength. See IETF RFC 3526 for additional information. It is difficult to identify versions of IPsec policies within Active Directory by looking at the attributes on the policy. Medium (2) – 1024-bit keying strength. The cryptographic strength of keys used for the integrity and encryption of the IKE negotiation itself and for IPsec data protection depends on the strength of the DiffieHellman group upon which the prime numbers are based. such as the IPsec Policy Management MMC snap-in or the Netsh IPsec command-line utility. and the length of the base prime numbers used during the key exchange process. Note Computers running Windows 2000 must have the High Encryption Pack or SP2 (or later) installed to use 3DES. This policy can be assigned in the domain to Windows 2000 platforms. pilot deployments. The following table lists the key exchange security methods in order of preference that Woodgrove Bank chose to implement: Table 5. confidentiality or encryption (3DES and DES).118 Server and Domain Isolation Using IPsec and Group Policy Key Exchange Methods The key exchange methods control the security methods that are used during main mode IKE negotiation. you should manage these files with a version control system similar to Microsoft Visual SourceSafe®.and Windows Server 2003-based systems. Therefore. or other documents to create IPsec policies. • • The High configuration can only be used with Windows XP SP2.9 Default Key Exchange Security Methods Encryption Integrity Diffie-Hellman Group 3DES 3DES 3DES SHA-1 SHA-1 MD5 High (3) 2048 bits Medium (2) 1024 bits Medium (2) 1024 bits Note The use of the 2048-bit group in IPsec policy requires that the IPsec policy be configured using the Windows Server 2003 management tools. If you use scripts. it is recommended that you store some form of versioning information both within the name of the policy and within the policy rules. The Medium configuration provides interoperability with Windows 2000 and Windows XP SP1. . lab testing. There are three options for the Diffie-Hellman group: • High (3) – 2048-bit keying strength. Policy Versioning IPsec policy designs are likely to be changed many times throughout initial planning. The configuration options are for integrity (SHA-1 and MD5). Low (1) – 768-bit keying strength. spreadsheets.

Chapter 5: Creating IPsec Policies for Isolation Groups

119

A simple versioning method is to create a version ID based on the following formula: <Major Change>.<Minor Change>.<Date:yymmdd>.<Time:24 Hour> For example, 1.0.041001.1600 would be version 1.0 created on 10/01/04 at 4 P.M. You should then place this version ID at the end of the name of the policy that you are creating. For example, IPSEC – Boundary IPsec Policy (1.0.041001.1600). You can also append it to the name or description of Filter Lists, which frequently change. Group Policy retrieves the IPsec policy name, and it is stored in the local registry under HKEY_LOCAL_MACHINE \SOFTWARE\Policies\Microsoft\Windows\IPsec\GPTIPSECPolicy where it is stored as a string value under the DSIPSECPolicyName key. Although IPsec service polling checks for changes in all assigned policy objects, it does not update the name of the assigned policy that Group Policy has stored. Group Policy does not update the name in the local registry until the GPO assignment is changed. Microsoft IT found that an unused rule within the IPsec policy can be an effective means of storing the policy version information. For example, you can create a filter list with a filter that contains invalid addresses and is associated with a permit filter action, such as the following: Filter List Name: IPsec policy ver 1.0.041001.1600 Filter List Description: IPsec policy ver 1.0.041001.1600 1.1.1.1 <-> 1.1.1.2, ICMP, description = "IPsec policy ver ID 1.0.041001.1600" After you create this filter list in Active Directory, you can identify the version object distinguished name (DN) for the filter list by using the Active Directory Users and Computers MMC running in Advanced mode. You can find the filter list object under the <DomainName>\System\IP Security tree and identify it by its description. After you know the version object DN, you can compare it programmatically with the IPsec objects stored in the registry under HKEY_LOCAL_MACHINE \SOFTWARE\Policies\Microsoft\Windows\IPSec\Policy\Cache to determine whether it is located in the cache. If you find the version object DN in cache, you can compare the policy names for the object stored in Active Directory and the object stored on the local computer. If the names are the same, the local and domain policies are synchronized. The names and descriptions of each IPsec filter list are also stored in the IPsec policy cache, which will help identify which versions of these objects are currently assigned. The IPsec service retains the text description for each filter (not filter list) in memory so that the IPsec monitor MMC snap-in and command-line tools can report this information. A script can help automate the policy version check of a client, such as the example script Detect_IPsec_Policy.vbs, which is included in the Tools and Templates folder for this solution. As policies are edited over a period of time, you should update the corresponding filter names to reflect the changes.

How to Apply IPsec Policies to Individual Computers
The final step to enabling IPsec is to deploy the policies to the hosts. There are two methods of deploying the policies. One method is to apply them directly to the individual host computers, and the second is through the use of GPOs and Active Directory. Policy application through Active Directory is discussed in the "How to Apply IPsec Policies Using GPOs" section later in this chapter.

120

Server and Domain Isolation Using IPsec and Group Policy

You can accomplish the application of IPsec policies to individual computers in one of two ways: through the IPsec Security Policy Management MMC snap-in, or through the command line using Netsh (for Windows Server 2003), Ipseccmd.exe (for Windows XP), or Ipsecpol.exe (for Windows 2000). The MMC snap-in provides a graphical user interface (GUI) that the administrator can use to manually apply policy or import a previously defined IPsec policy that was exported from another computer. In addition to manipulating the policy on the local computer, administrators can use the snap-in to manage policy on a remote computer. Detailed information about the command-line tools is available from the following resources: • • • Windows Server 2003 Help and Support for Netsh Windows XP Service Pack 2 Support Tools documentation for Ipseccmd.exe Windows 2000 Server Resource Kit for Ipsecpol.exe

Microsoft provided updated IPseccmd and other support tools for Windows XP SP2. For more information see Microsoft Knowledge Base article 838079, "Windows XP Service Pack 2 Support Tools". Detailed information about the use of these tools is beyond the scope of this guidance. The examples in this guide are geared toward using Netsh on servers running Windows Server 2003.

How to Apply IPsec Policies Using GPOs
Active Directory Group Policy is used as the IPsec policy assignment and distribution mechanism for domain-joined computers. Before distributing the policies through the Group Policy distribution mechanisms in Active Directory, you must first configure the GPOs that will be used to apply the IPsec policies to the host computers.
Note Although the following section discusses loading IPsec policies directly into Active Directory, it is assumed that the policies were created and tested on a local system, in a test lab, and in small-scale pilot projects before being deployed to a production environment.

How to Load IPsec Policies into Active Directory
The first task in implementing IPsec policies through Active Directory is to create the filter lists, filter actions, and IPsec policies in the directory service. You can perform this task using the IPsec Security Policy Management MMC snap-in or a command-line tool such as Netsh. Regardless of which tool you select, you must perform the following three subtasks to implement the IPsec policies: 1. Create the filter lists and filters identified in the "IPsec Filter Lists" section of this chapter. 2. Create the filter actions identified in the "IPsec Filter Actions" section of this chapter. 3. Create the IPsec policies identified in the "IPsec Policies" section of this chapter.

Using the IPsec Security Policy Management MMC Snap-in
The IPsec Security Policy Management MMC snap-in is a GUI-based tool that allows administrators to create, configure, and edit IPsec policies on local computers, remote computers, or domains. Configuration of the IPsec components is a manual process that involves direct editing of the objects being created and is guided by wizards.

Chapter 5: Creating IPsec Policies for Isolation Groups

121

After the IPsec policies are defined locally or in Active Directory, the administrator can export the IPsec policies (including all filter lists and filter actions) to a file ending with an .ipsec file name extension. You can copy this file to other media for backup purposes. If a backup of the IPsec policies exists, you can use the tool to import the backed-up policies into Active Directory. You can use this approach for recovery purposes or to move IPsec policy files from a test forest into a production forest without having to recreate each filter list, filter action, and policy manually. Thoroughly review the design of policies that are restored from backup. Testing is recommended to assess the impact of applying old settings in the current environment. An old backup file may contain policy settings, such as filter lists, or filter actions, which are invalid and may cause communicate to fail if they were assigned to current domain members. For more information about using the IPsec Security Policy Management MMC snap-in, see the topic "Define IPSec Policies" in the Help and Support Center in Windows Server 2003.

Using Netsh
You can use Netsh as an alternative to the IPsec Security Policy Management MMC snap-in for configuring IPsec polices within a Windows Server 2003-based Active Directory. You can run this command-line tool in an interactive mode or in batch mode. When used in interactive mode, Netsh requires the administrator to type individual commands into the Netsh command shell. Before creating the filter lists, filter actions, and IPsec policies, you must configure the tool to point to the Active Directory. To point Netsh to the Active Directory, type the following command at the Netsh prompt: ipsec static set store location=domain The administrator then enters the filter lists, filters, filter actions, and IPsec policies manually through the Netsh command shell. Like the GUI tool, Netsh supports the export and import of IPsec policy files for backup and recovery scenarios. Running Netsh in batch mode requires creation of a script file of Netsh commands. This script file must contain the command to set the focus on the domain as well as all of the configuration commands for the filter lists, filters, filter actions, and IPsec policies. You can then create the IPsec policy information in Active Directory by launching Netsh and executing the script file. The command-line syntax for launching Netsh and executing a script file is as follows: netsh –f <scriptfile> For more information on using Netsh, see the "Netsh" topic in the "Administration and Scripting Tools" section of the Help and Support Center in Windows Server 2003.
Note Netsh only works with IPsec policies on computers running Windows Server 2003. Command-line manipulation of IPsec policies on computers running Windows 2000 or Windows XP requires Ipsecpol.exe or Ipseccmd.exe, respectively. Also, Netsh lists a dump command in the IPsec context of the tool. This function is not implemented, although it is listed in the help text. In addition, unlike the GUI tool, Netsh does not support remote connections.

Creating Group Policy Objects for IPsec Policy Distribution
GPOs are objects stored within Active Directory that define a set of settings to be applied to a computer. IPsec policies are not stored within GPOs directly. Instead, GPOs maintain an LDAP DN link to the IPsec policy. IPsec policies are stored in the cn=IP Security, cn=System, dc=<domain> location within Active Directory. GPOs are assigned to sites, domains, or OUs within Active Directory. Computers that are within those locations or containers receive the policy defined by the GPO unless it is otherwise blocked. The IPsec design team should consult with the Active Directory team

122

Server and Domain Isolation Using IPsec and Group Policy

to discuss the feasibility of using existing GPOs to deliver their IPsec policies. If this approach is not feasible or requires extensive modification of their management practices, new GPOs can be defined for each set of IPsec policies that will be deployed. The solution described in this guidance uses new GPOs for deployment of the IPsec policies. Although GPOs can be created through Active Directory Users and Computers or the Active Directory Sites and Services tools, it is recommended that you create the new GPOs by using the Group Policy Management Console (GPMC). Creation of a policy through the Active Directory tools automatically links the GPO to the object that is being browsed. By using the GPMC to create the GPOs, the administrator can ensure that the GPOs are created within Active Directly but not applied to computers until each GPO is explicitly linked to a site, domain, or OU. The GPMC is an add-on utility for computers running Windows XP Service Pack 1 (or later) or Windows Server 2003. The GPMC allows administrators to manage Group Policy for multiple domains and sites within one or more forests through a simplified user interface with drag-and-drop support. Key features include functionality such as backup, restore, import, copy, and reporting of GPOs. These operations are fully scriptable, which allows administrators to customize and automate management. Note that these GPO management techniques apply to managing IPsec policy objects themselves. You should develop a management strategy for managing IPsec policy in coordination with the GPOs that deliver the IPsec policy assignment. For additional information about using the GPMC, see the white paper "Administering Group Policy with the GPMC". You can download the Group Policy Management Console with Service Pack 1. Using the GPMC, the administrator creates a GPO for each IPsec policy by completing the following steps: To create a new GPO 1. Expand the domain tree, right-click the Group Policy Objects container, and then select New. 2. Type a new GPO name, and then click OK. As with IPsec filter actions and policies, you should develop a naming standard for the GPOs that includes a version number of the policy within the name—because version information of Active Directory objects is not easily obtainable. Inclusion of a version number in the policy name allows administrators to quickly identify the policy that is currently in effect. Microsoft recommends using the same naming convention that was described earlier in this chapter for filter actions and IPsec policies. For example, a GPO named "Isolation Domain IPsec GPO ver 1.0.040601.1600" would be version 1.0 created on 06/01/04 at 4 P.M. After the GPO has been created, the administrator needs to configure it to use the appropriate IPsec policy. To assign IPsec policy in a GPO 1. Launch the Group Policy Editor by right-clicking the name of the GPO and selecting Edit. 2. The available IPsec policies that can be assigned are located under Computer Configuration\Windows Settings\Security Settings\IP Security Policies in Active Directory.

Chapter 5: Creating IPsec Policies for Isolation Groups

123

3. To assign an IPsec policy, right-click the policy name in the right pane and then select Assign. Only one IPsec policy can be assigned per GPO. 4. To save the changes to the GPO, close the Group Policy Editor tool. IPsec policy is applied to host computers through the computer configuration settings on the GPO. If the GPO is only used to apply IPsec policies, it is recommended that you configure the GPO to disable the user configuration settings. Disabling these settings will help shorten the processing time of the GPO by bypassing the evaluation of user configuration options. To disable the user configuration on the GPO 1. Open the GPMC tool. 2. Right-click the GPO name in the GPMC. 3. Select GPO Status, and then User Configuration Settings Disabled. If the user configuration settings are configured in the GPO at a later date, the administrator will have to re-enable the processing of user configuration settings for them to apply.

Domain Security Groups
Domain security groups serve two purposes. The first is to identify domain computer accounts that are members of an isolation group, and the second is to identify domain computer accounts that are members of a network access group. All members of an isolation group are required to receive the same IPsec policy. Therefore, you can create domain security groups for application and management of the IPsec policies instead of using OU containers to control policy assignment. Universal groups are the best option to control policy assignment because they are applicable to the entire forest and reduce the number of groups that need to be managed. However, if universal groups are unavailable, you can use domain global groups instead. Domain local groups are used for network access groups discussed in a later section. The following table lists the groups that were created for the Woodgrove Bank scenario to manage the IPsec environment and control policy application: Table 5.10 IPsec Group Names Group name No IPsec Description A universal group of computer accounts that do not participate in the IPsec environment. Typically consists of infrastructure computer accounts.

CG_IsolationDomain_computers A universal group of computer accounts that are members of the Isolation domain. CG_BoundaryIG_computers A universal group of computer accounts that are members of the Boundary isolation group and thus allowed to communicate with untrusted systems.

Distributing IPsec Policies Through Active Directory You can control which GPOs are applied to computers in Active Directory in a combination of three ways: • • • Using OUs with linked GPOs Placing computer accounts in security groups that are referenced in ACLs that apply to the GPOs Using Windows Management Instrumentation (WMI) filters on the GPOs Controlling GPO application through OUs with linked GPOs is the most common form of policy application in Active Directory. assigned. then the policy is applied. A WMI SQL filter is created and linked to the policy. In addition.124 Server and Domain Isolation Using IPsec and Group Policy Group name Description CG_NoFallbackIG_computers A universal group of computer accounts that are part of the No Fallback isolation group who are not allowed to perform outbound unauthenticated communications. CG_EncryptionIG_computers A universal group of computer accounts that are members of the Encryption isolation group and thus require encryption for their communications. additional groups may have been created and used to restrict the policy application during the initial rollout. domains. The design of the boundary group policy is typically used to allow non-IPsec communication both inbound and outbound with computers that have not yet received their IPsec policy. it is not recommended to just create the GPOs and IPsec policies and assign them at the same time to all computers in the domain. the group is specifically denied permissions on policies that should not apply to the computers within the group. Unless the ACLs on the GPO are extremely large. If the condition that was queried is true. A domain security group can be used for precise control over which computers can read the GPOs and thus receive the corresponding IPsec policy. When deploying IPsec. WMI queries can slow GPO processing and should be used only when necessary. Woodgrove Bank chose to use security groups to control the policy application rather than linking to an OU directly. The GPOs that deliver IPsec policy can all be assigned to the entire domain. Computers receive policy based on their location within Active Directory. The second method uses security settings on the GPOs themselves. and retrieved on all nodes that expect to negotiate IPsec. The rollout process must carefully consider whether IPsec policy is properly designed. there is no additional overhead associated with this method over the first method because in both cases the ACLs have . This method creates OUs within Active Directory and links GPOs to sites. This group is then assigned Read and Apply Group Policy Permissions rights on the policy that should take effect on the computers within the group. otherwise it is ignored. A group is added to the ACL of the GPO in Active Directory. In addition to the listed groups. Computers running Windows 2000 ignore WMI filtering and will apply the policy. If a computer is moved from one OU to another. or OUs. The third method uses WMI filters on the policy to dynamically control the scope of the policy application. The policy is then linked at the domain level. Woodgrove selected this approach to easily introduce IPsec policies in the environment without having to force the policies into multiple locations or force a move of computers from one OU to another to receive the correct policy. the policy linked to the second OU will eventually take effect when Group Policy detects the change during polling.

If there is a business need for a system to accept communications from untrusted systems. Boundary Isolation Group Woodgrove Bank chose to link the Boundary isolation group policy to the domain level in each domain in the organization. Note that ACLs on IPsec policy objects themselves are not used and are not recommended. The Isolation Domain policy is the policy that all computers in the organization use as their default IPsec security policy. Woodgrove did not choose WMI filtering because there are Windows 2000 systems in the environment. The following table shows the final Group Policy ACL configuration. the Domain Computers group is granted Read access to the policy. and another temporary security group was used to control who could receive this policy. This approach allowed for a staged deployment of this policy. During the initial rollout. The Authenticated Users group's Apply Group Policy rights were removed from the policy ACL. The policy uses an ACL. Accordingly. . which prevents anyone who is not a member of the CG_BoundaryIG_computers group from applying the policy. the computer account of the system can be added to the CG_BoundaryIG_computers security group. Table 5. The Authenticated Users group's Apply Group Policy rights were removed from the policy ACL. The policy uses an ACL.Chapter 5: Creating IPsec Policies for Isolation Groups 125 to be evaluated. the Domain Computers group was removed from the ACL. The Authenticated Users group's Apply Group Policy rights were removed from the policy ACL.11 Woodgrove Bank Policy GPO Permissions GPO name IPSEC – Isolation Domain Policy Security group name No IPsec Rights assigned Deny Apply Group Policy CG_IsolationDomain_computers Allow Read and Apply Group Policy IPSEC – Boundary Group Policy No IPsec CG_BoundaryIG_computers IPSEC – No Fallback Isolation Group Policy No IPsec CG_NoFallbackIG_computers IPSEC – Encryption Isolation Group Policy No IPsec CG_EncryptionIG_computers Deny Apply Group Policy Allow Read and Apply Group Policy Deny Apply Group Policy Allow Read and Apply Group Policy Deny Apply Group Policy Allow Read and Apply Group Policy Isolation Domain Woodgrove Bank chose to link the Isolation Domain policy to the domain level in each domain in the organization. which prevents anyone who is not a member of the CG_IsolationDomain_computers group from applying the policy.

8 of Chapter 4. This policy uses an ACL. The policy uses an ACL. If the computers in the Encryption isolation group are members of separate trusted domains. are also added. the computer account of the system is added to the CG_EncryptionIG_computers group. Table 4. However. the computer account of the system can be added to the CG_NoFallbackIG_computers group. then these NAGs can be created as domain local security groups. If there is a business need for a system to communicate only with encrypted traffic. which prevents anyone who is not a member of the CG_NoFallbackIG_computers group from applying the policy. The accounts must be authorized to have the Access this computer from the network right in some manner.126 Server and Domain Isolation Using IPsec and Group Policy No Fallback Isolation Group Woodgrove Bank chose to link the No Fallback isolation group policy to the domain level in each domain in the organization. The Authenticated Users group's Apply Group Policy rights were removed from the policy ACL. If there is a business need for a system to be denied the ability to initiate communications with untrusted systems. The Woodgrove Bank scenario involves only one domain. IKE must obtain a Kerberos service ticket that contains the domain security group identifier that indicates whether the client computer is a member of the ANAG and/or possibly the DNAG. IPS-SQL-DFS-01 and IPS-SQL-DFS02. Encryption Isolation Group Woodgrove Bank chose to link the Encryption isolation group policy to the domain level in each domain in the organization. Authorizing Inbound Access to an Isolation Group The Woodgrove Bank requirements were to allow only a subset of trusted hosts to be authorized for inbound network access to the servers in the Encryption isolation group. If all the computers in the Encryption isolation group are members of the same domain. which prevents anyone who is not a member of the CG_EncryptionIG_computers group from applying the policy. The Authenticated Users group is removed from the Read right. The following additional GPO was created for the purpose of defining the Access this computer from the network rights that implement the inbound authorization: • EncryptionIG Inbound Network Authorization GPO The CG_EncryptionIG_Computers group is given Read and Apply rights for this GPO. by direct inclusion of their . which results in this GPO being applied only to computers that are members of the Encryption isolation group. The encryption server accounts themselves. then either one set of domain global groups can be used for the NAGs or domain local groups can be created in each domain. the authorized client domain computer account IPSST-XP-05 is added to the ANAG_EncryptedResourceAccess_Computers network access group. the same result could have been achieved more easily by using the CG_EncryptionIG_computers group to manage a larger list. The Authenticated Users group's Apply Group Policy rights were removed from the policy ACL. so it uses domain local groups for these NAGs. whether it is through the ANAG membership.8 in Chapter 4 defined the following network access groups (NAG) to implement these requirements: • • • ANAG_EncryptedResourceAccess_Users ANAG_EncryptedResourceAccess_Computers DNAG_EncryptedResourceAccess_Computers When a client initiates IKE to an encryption server. As shown in Table 4.

and SQL) authentication requests after successful IPsec ESP 3DES connectivity is established. Woodgrove Bank wanted to take advantage of the ability to define inbound network access restrictions for users as well as computers. To simulate a large domain environment. Deny Access to this computer from the network: • DNAG_EncryptedResourceAccess_Computers Assuming User7 cannot log on to the encryption servers directly. SMB. or through explicit listing of computer accounts in the right. Additional IPsec Considerations In addition to defining the IPsec policies. Access this computer from the network right: • • ANAG_EncryptedResourceAccess_Computers ANAG_EncryptedResourceAccess_Users The following group is configured for the same GPO. the following types of traffic are exempt from filter matches by default: • • • Broadcast Multicast Kerberos authentication protocol . Local Policies.xls spreadsheet in the Tools and Templates folder provides details of constraints when using IPsec. Note that the rest of the user and computers in the domain are effectively denied access. there are some additional considerations for a successful IPsec implementation. They reside in the GPO under Computer Configuration. Otherwise. it created a domain local security group called ANAG_EncryptedResourceAccess_Users and populated it with authorized application user accounts (such as User7) as well as the local administrators group and the domain administrator groups. The Business_Requirements. An explicit deny parameter overrides all forms of allow. the accounts will not be able to make IPsec -protected connections to each other. Authorized domain users must now be explicitly authorized. because they are purposely omitted from the Access this computer from the network right. However. the result is that User7 must use the IPS-ST-XP-05 client computer to access either IPS-SQL-DFS-01 or IPSSQL-DFS-02 servers. the ANAG groups are the only groups that authorize inbound access for the Woodgrove Bank scenario. The IPS-ST-XP-05 computer must have a valid domain computer account and an active IPsec policy that initiates IKE negotiation to the encryption servers' IP addresses. which their applications require. Default Exemptions In Windows 2000 and Windows XP. Windows Settings. Therefore. The following domain security groups were created for network logon rights for the encryption servers. it must be removed from the Access this computer from the network right. User Rights Assignment. Only the Boundary computers are explicitly denied access through the use of the DNAG as a defense-in-depth measure against future group membership changes that might include a boundary computer account in an ANAG. Because the Authenticated Users group includes all domain computers. These user-level network access restrictions occur during upper layer protocol (such as RPC.Chapter 5: Creating IPsec Policies for Isolation Groups 127 CG_EncryptionIG_computers group. which could be achieved by using the built-in group for Domain Users. Security Settings.

multicast. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC\NoDefaultExe mpt DWORD registry setting to a value of 1. A filter with a source address of Any IP Address will match multicast and broadcast addresses. A client on the NAT-configured network (Client 2) uses IPsec NAT-T to establish bidirectional security associations with Client 1. you should verify the behavior of IPsec policies designed for Windows 2000 or Windows XP and determine whether to configure explicit permit filters to permit specific traffic types. and IKE traffic from IPsec filtering by modifying the registry 1. Restart the computer. To restore the IPsec driver to the default Windows 2000 and Windows XP filtering behavior using Netsh 1. 3. To exempt all broadcast. . By default. Broadcast and multicast packets will be dropped if they match a filter with a filter action to negotiate security. You can use such a filter to block all traffic. clients may experience unexpected results when communicating with servers running Windows 2000 Server or Windows Server 2003 behind a network address translator that uses IPsec NAT-T. however. A network address translator is configured to map IKE and IPsec NAT-T traffic to a server on a NAT-configured network (Server 1). traffic from the broadcast. As a result of the change in default exemption behavior for the Windows Server 2003 family implementation of IPsec. A client from outside the NAT-configured network (Client 1) uses IPsec NAT-T to establish bidirectional security associations with Server 1. One-way filters that would be used to block or permit specific multicast or broadcast traffic. multicast. the Windows Server 2003 family provides limited support for filtering broadcast and multicast traffic. Restart the computer. are not supported. NAT-Traversal Because of the way that network address translators work. A filter with a source address of Any IP Address and a destination address of Any IP Address will match inbound and outbound multicast addresses. This change was made to avoid a perceived security risk for a situation in which the following sequence of events occurs: 1. RSVP. Type the following command at a Netsh prompt and then press ENTER: netsh ipsec dynamic set config ipsecexempt 0 2. Windows XP SP2 no longer supports IPsec NAT-T security associations to such servers. To restore the default Windows 2000 and Windows XP behavior for IPsec policies. The NoDefaultExempt key does not exist by default and must be created. you can use the Netsh command or modify the registry. By default.128 Server and Domain Isolation Using IPsec and Group Policy • • IKE Resource Reservation Protocol (RSVP) In the Windows Server 2003 family of operating systems. 2. 2. and Kerberos authentication protocols is not exempt from filter matches by default (only IKE traffic is exempt).

Click Start and Run. If there is a requirement for IPsec communication across NAT. Click OK. point to New. 4. After configuring AssumeUDPEncapsulationContextOnSendRule with a value of 1 or a value of 2. and then close the Registry Editor. A condition occurs that causes Client 1 to re-establish the security associations with Client 2 because of the static network address translator mappings that map IKE and IPsec NAT-T traffic to Server 1. 8. 2. Restart the computer. the default behavior on Windows XP SP2-based computers prevents any IPsec NAT-T-based security associations to servers that are located behind a network address translator to ensure that it never occurs. and then click DWORD Value. • • Note The configuration represented by value 2 exists in the original release version of Windows XP and in Windows XP Service Pack 1 (SP1). type regedit and then click OK. and then press ENTER: AssumeUDPEncapsulationContextOnSendRule Note This value name is case-sensitive. 7. type one of the following values: • 0 (default). it is recommended that you use public IP addresses for all servers that can be connected to directly from the Internet. A value of 1 configures Windows so that it can establish security associations with servers that are located behind network address translators. A value of 0 (zero) configures Windows so that it cannot establish security associations with servers that are located behind network address translators. If this configuration is not possible. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPSec 3. Although this is an unlikely situation. 5. 2. A value of 2 configures Windows so that it can establish security associations when both the server and the Windows XP SP2-based client computer are behind network address translators. type the following. and then click Modify. In the Value data box. This condition may cause the IPsec security association negotiation traffic that is sent by Client 1 and that is destined for Client 2 to be misrouted to Server 1.Chapter 5: Creating IPsec Policies for Isolation Groups 129 4. To create and configure the AssumeUDPEncapsulationContextOnSendRule registry value 1. 6. Windows XP SP2 can connect to a server that is located behind a network address translator. . In the New Value #1 box. Right-click AssumeUDPEncapsulationContextOnSendRule. then you can change the default behavior of Windows XP SP2 to enable IPsec NAT-T security associations to servers that are located behind a network address translator. 1. On the Edit menu.

see Knowledge Base article 885348. "L2TP/IPSec NAT-T update for Windows XP and Windows 2000". "IPSec NAT-T is not recommended for Windows Server 2003 computers that are behind network address translators". 6. see Knowledge Base article 885407. see Appendix A of the white paper. 5. such as the Windows XP Hardening Guide and the Windows Server 2003 Hardening Guide. ICF is not aware of traffic that is protected by IPsec AH or ESP in transport or . For additional information. An additional feature of IPsec support is that you can specify through Group Policy that all IPsec-protected traffic bypass Windows Firewall processing. Right-click EnablePMTUDiscovery. 4. IPsec and Internet Connection Firewall (ICF) For Windows XP-based computers not running SP2. 2. PMTU discovery must be enabled and functioning. IPsec and Windows Firewall Windows Firewall on computers running Windows XP SP2 provides additional protection against attacks by blocking incoming unsolicited traffic. and then close the Registry Editor. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\tcpip\ parameters 3. type regedit and then click OK. type EnablePMTUDiscovery and then press ENTER.130 Server and Domain Isolation Using IPsec and Group Policy Windows Server 2003-based servers also require an update to operate correctly with IPsec when placed behind a NAT device. Restart the computer. the Internet Connection Firewall (ICF) may better meet the security requirements for filtering traffic. the IPsec components of Windows XP SP2 instruct Windows Firewall to open UDP ports 500 and 4500 to allow IKE traffic. When there is an active IPsec policy. To enable PMTU Discovery 1. For more information. and then click Modify. IPsec in Windows XP SP2 is Windows Firewall-aware. On the Edit menu. In the New Value #1 box. Each customer must evaluate the benefits of using IPsec in this scenario against the security risks associated with it. For inbound NAT connections to successfully work. In the Value data box. recommend disabling PMTU discovery. Click OK. Some sources. Microsoft will support the scenario using the configuration documented in this solution. they provide policy templates that disable the PMTU discovery functionality. see the following URL for information on how to get Windows Server 2003 Service Pack 1. Note For more information. "Deploying Windows Firewall Settings for Microsoft Windows XP with Service Pack 2". type 1 7. point to New. 8. and then click DWORD Value. Although Microsoft does not recommend the scenario because of the associated security risks. However. "The default behavior of IPSec NAT traversal (NAT-T) is changed in Windows XP Service Pack 2". This article explains the security risks of using this scenario. In addition. and in some cases. Click Start and Run. ICF does provide filtering and can block inbound multicast and broadcast traffic in Windows XP SP1. hosts running Windows 2000 and participating in a NAT-T scenario require the application of the hotfix associated with Knowledge Base article 818043.

If you want to recreate this environment or study the deployment stages. . The next phase of the project is the deployment of a test or pilot environment to allow the design to be verified. Microsoft performed this verification in its test labs and used the Woodgrove Bank scenario as the pilot. Because IPsec operates in the network layer below ICF and IKE operates above ICF. This information was divided into the following seven basic tasks: • • • • • • • Identifying and creating filter lists Identifying and creating filter actions Identifying and creating rules Identifying and creating IPsec policies Defining a distribution mechanism for IPsec policies Defining a rollout deployment method for the IPsec policies Defining authorizations for inbound access control using Group Policy network logon rights configuration These tasks complete the design and planning phases of the server and domain isolation solution. the ICF dropped packet log will not contain the packets that IPsec discarded.Chapter 5: Creating IPsec Policies for Isolation Groups 131 tunnel mode. General Background Information on IPsec ● The IPsec Web site ● The Internet Protocol Security for Windows 2000 Server download provides a wide range of Windows 2000-specific IPsec content ● The Deploying IPsec chapter from the Windows Server 2003 Deployment Kit Additional Information ● The Windows Server 2003 Group Policy download provides a wide range of information about managing Windows systems with Group Policy in Active Directory. If IPsec blocks traffic. a static permit for IKE (UDP port 500) should be set for inbound traffic. More Information This section provides links to additional information about the technologies mentioned in this chapter. Summary This chapter provided the information for creating and deploying IPsec polices that are based on the isolation group design that was created in Chapter 4. ● The Cable Guy—October 2004: Problems with Using Network Address Translators provides additional information on the specific issues with NAT and IPsec. Appendix C of this guide contains step-by-step guidance for the deployment process that Microsoft used in its test labs to validate the design for the scenario.

The information provided in this chapter is designed to help you develop these solution management processes. and the application of security templates through the use of either Group Policy or command-line tools. Windows system security concepts. and access control lists. and use of Group Policy. . The key to success in managing a server and domain isolation solution is to plan ahead. incorrectly-set option can significantly restrict the functionality of hosts that consume the policies. Similarly. particularly with regard to IPsec and TCP/IP. An understanding of core networking concepts. Computer Management.Chapter 6: Managing a Ser ver and Domain Isolation Environment This chapter provides guidance to help manage a server and domain isolation solution after successful deployment into a production environment. groups. and NTBackup. the use of security templates. such as users. • • • Before proceeding with this chapter. a well-documented and well-communicated set of management processes and procedures should be in place for the support teams to use after the solution becomes operational. For these reasons. It is important for support staff to understand that the isolation solution adds an additional layer of security. the staff that is responsible for IPsec policies and Group Policy must understand that a single. you should read the rest of this guide and have a thorough understanding of the architecture and design of the solution. groups. Familiarity with Microsoft Windows® 2000 Server (or later) is also required in the following areas: • • Basic operations and maintenance of Microsoft Windows Server™ 2003. the information here should be customized as much as possible to reflect the needs of your own implementation. and other Active Directory objects. manipulating users. Chapter Prerequisites Before you use the information provided in this chapter. you should be familiar with the concepts used in the Microsoft® Operations Framework (MOF) and the concepts of IPsec. including Active Directory structure and tools. including the use of tools such as Event Viewer. and that more than a network connection and an IP address are needed for a host to successfully connect to a resource. Ideally. The Active Directory® directory service. auditing. An understanding of Windows Scripting Host and knowledge of Microsoft Visual Basic® Scripting Edition (VBScript) will help you obtain the most benefit from the supplied scripts but is not essential.

The formal initiation of a change through the submission of a request for change (RFC). but it should also include key integration. The consideration and approval or disapproval of the change by the change manager and the change approvals board (CAB). . Change request. Change development. Each change development procedure will have a companion change release procedure that describes how to deploy the change into production. the first concern is with the timing of the changes. Policy Application Delays When the assignment of IPsec policy in a Group Policy object (GPO) is changed to a new IPsec policy. The following section outlines the change development procedures for some of the key changes that will likely be required on a regular basis in your IPsec environment. and staging environments. As in the initial rollout. The release and deployment of the change into the production environment. the time required for the first impact and the complete rollout can be predicted. 2. 4. A post-implementation process that reviews whether the change has achieved the goals that were established for it and determines whether to keep the change in effect or back it out. Change release. The assignment of a priority and a category to the change that uses its urgency and its impact on the infrastructure or users as criteria. 5. All changes to an IPsec environment should follow the standard MOF change management process: 1. Change review. There is the Active Directory replication delay of the GPO attribute that contains the assignment in the domain. Microsoft recommends that you test and document these delays (minimum. 3. any changes made in one part of a system may have profound impacts on another. and also the polling delay of domain member Group Policy clients to detect the change in the GPO. 6. a process that can vary significantly in scope and includes reviews at key interim milestones. and median) for your particular environment so that when a change is introduced. maximum. Change classification. The planning and development of the change. a group of IT and business representatives. Because most systems are heavily interrelated. the target (or managed) environment is the production environment. testing. To mitigate or eliminate any adverse effects. change management attempts to identify all affected systems and processes before the change is deployed. These delays can range from less than a minute in a small location to a matter of hours in a global enterprise. because this timing affects the ability to implement a change and the timeframe for rolling back the change.Change Management A key goal of change management processes is to ensure that all parties affected by an impending change are aware of and understand the impact of the change. Changing IPsec Policy It is important to understand how changes in IPsec policy will affect communication. Change authorization. certain delays occur. This assignment affects the implementation speed and route. Typically.

or they may use only the default response rule (which is not used in Woodgrove's design). However. server policy change will not affect the ability of existing clients to rekey main mode. This discussion covers the impact of most types of changes on IPsec client-server functionality. which would cause clients to behave as if they have a domain-based IPsec policy assigned—but they would not to be able to retrieve that policy. ensure that you first create all the objects (GPO. and also the polling delay of the IPsec policy service on the member computers. The new main mode SAs will be generated when the next quick mode rekey occurs. Generally. Also. then traffic flow will stop as soon as the current IPsec security associations (SAs) idle or their lifetime in bytes or in seconds expires. In such a case.Chapter 6: Managing a Server and Domain Isolation Environment 135 When the contents of an already assigned IPsec policy are changed. There is an Active Directory replication delay of the IPsec policy objects. which will not affect established quick mode IPsec SAs. It does not assume the Woodgrove Bank IPsec policy designs. and so on) and then make the assignment of IPsec policy into the GPO. clients may have policy that is similar to the Woodgrove Bank design (in which the clients have filters to initiate IKE to the server). If a change causes Internet Key Exchange (IKE) main mode or quick mode to fail. To properly accommodate Active Directory replication delays. some changes on the server side that can cause IKE main mode negotiation with clients to fail include: • • • • Changing to a new authentication method (certificates only) without including the old authentication methods that the client can use Changing to 3DES/SHA1/DH1 or DH2 as the main mode security method when clients were configured only to use DES/SHA1/DH1 Activating main mode perfect forward secrecy (PFS) without updating both client and server policy so that both use main mode PFS Activating quick mode PFS without updating both client and server policy so that both use quick mode PFS The following server policy changes will not affect a client's ability to rekey main mode SAs: • • • • Polling interval for policy changes (because it is not a main mode IKE setting) Session keys that use the same master key (for example. similar delays occur. the number of IKE quick modes per main mode) Adding a new security method that clients do not know about Changing the IPsec policy advanced key exchange settings for Authenticate and generate a new key lifetime parameters for IKE main mode SA . IPsec policy. Windows 2000 and Windows XP hosts will simply fail to apply the domain-based policy. This section provides information on how common changes can affect IPsec connectivity from the perspective of changing a server policy when clients may not have the latest update. It is possible to create a condition where the replication of the policy assignment in the GPO happens before the replication of the IPsec policy. they will not apply any local policy that might be assigned. For this discussion. Main Mode Changes Changing an authentication method or main mode security method will cause the IKE to delete the existing main modes. Changes That Affect IPsec Connectivity There are many areas that can affect connectivity within the policies and groups that make up an IPsec solution.

that traffic will immediately begin to be permitted and may be dropped by the client with a more general IPsec default response filter. • Changing between incompatible security methods or encapsulation types. Server policy changes that will affect the ability of active IPsec clients to rekey quick mode include the following: • Changing from a more general filter to a more specific filter. For example. a new quick mode will be attempted if traffic is flowing. To avoid problems. • • • . receive a plaintext ICMP in reply. and drop the packet. You can avoid IKE quick mode negotiation failures caused by this type of change by including the old security methods or encapsulation types as the last choice in the new security method. This particular example would not affect any traffic other than ICMP traffic between the server and the client and is an expected design behavior that will always produce lost ICMP traffic after the server requests security for all traffic with the client. you can delete the old method at the bottom of the security method list. Entirely disabling a rule that the clients needed to establish either IKE main mode or quick mode." The more specific filter will be subject to outbound traffic on the server. For example. If the server adds a more specific filter that has a permit action. Therefore. After you see that all IPsec SAs are using the new encapsulation method. The "all traffic" filter on all the clients will eventually be deleted (after two hours) and can then be safely deleted in the server policy. Clients that cannot respond successfully to the IKE negotiation will fail to connect. for high-speed data transfers. if clients have default response policy and a server has policy is changing from "all traffic" to "TCP only. In quick mode. which will establish a new IPsec SA for TCP only when clients have default response. In this case. the client will secure its outbound ICMP. from only 3DES/SHA1 for ESP transport mode to only 3DES/MD5 for ESP transport mode.136 Server and Domain Isolation Using IPsec and Group Policy Quick Mode Changes Any change in a filter action that was being used for an IPsec SA will cause the existing IPsec SAs that were established under those policy settings to be deleted. will cause the connection to reset for the video application. This may be the intended behavior. the filter will get deleted so that a different filter or no filter will govern the IKE main mode and quick mode negotiation. because the current IPsec default response filter says all traffic must be secure. Some traffic may be lost in the process of this change. Clearing the Fall back to clear check box. This action will cause currently connected clients to have connectivity for as long as the soft SA lasts. Changing a filter action entirely from negotiate security to permit or block. This may or may not be a significant operational problem. For example. the impact of an immediate IPsec SA deletion is that outbound traffic will be dropped until the new quick mode can be established. an Internet Control Message Protocol (ICMP)-exempt filter gets added to a server. keep the more generic filter in place when you add the more specific filter. more server outbound traffic will cause IKE to attempt new main mode negotiation and recognize the new setting to not fall back. An example of this type of change would be when a server starts with an all traffic filter and then removes it. Traffic that is explicitly permitted or blocked will not require a rekey as the traffic will no longer participate in a communication channel protected by IPsec. but TCP connections should recover. For example a burst of packets from a video data stream. which TCP could not recover from. but the clients are already secure for all traffic to the server. After the SA expires or idles. However. leaving a TCP only filter.

either bytes or time. the IPsec Policy Management MMC snap-in is recommended to make small changes. they will still be able to negotiate a secure connection for that traffic. A change in the filter action IPsec SA lifetime. Netsh scripts should include initial steps that delete policy objects that already exist before new ones are added. Because of the chance for errors. such significant changes in an IPsec policy should be done through creation of a new IPsec policy version. whereupon they will not be able to reconnect. This action will cause clients to be disconnected if they do not have IPsec filters to trigger an outbound IKE main mode initiation.exe only supports the ability to create policy. Note Although the following section discusses how to modify IPsec policies directly in Active Directory. filter lists. The addition of a new object with the same name is not allowed in Netsh add commands. Because the Windows 2000 command-line tool Ipsecpol. the MMC snap-in can be used to manage changes in Windows 2000 Active Directory. Windows IPsec policy export and import is designed for backup and restore purposes. . Default response rule clients will stay connected until their dynamic default response filter idles out after two hours of no traffic to the server. you can use either scripts or the IPsec Policy Management MMC snap-in to make changes. Although the steps presented for each task use the IP Security Policy Microsoft Management Console (MMC) snap-in. Export is the recommended method to move all of the current domain policy into local storage for testing. Command line scripts are the recommended method for creating IPsec policy and making significant additions to existing objects. For this reason and because scripts are often run many times. For example. After IPsec policy has been created and is operational. Microsoft recommends the Windows Server 2003 platform as a policy management station because it provides the best capabilities exist for scripting and for monitoring. Generally. and filter actions) from a local store before using the export function. After the policy is created. Microsoft does not recommend the use of an exported local store to import into the domain because of the possibility that old object versions could overwrite more recent domain versions and break links between the objects. if permit filters are added to a server's policy for a new domain controller's IP address. such as adding filters to an existing filter list. be careful to delete every unwanted object (including policies. The following server policy changes will not affect a client's ability to rekey quick mode: • • • IPsec Policy Change Procedures The following sections provide steps for modifying IPsec policies that are delivered by using GPOs. Deletion of a nonexistent object will return an expected error message that will not stop the script from executing. each of these tasks can also be accomplished by using the Netsh command-line tool on a Windows Server 2003 system. Changing the filter action from Permit to Negotiate security. Deletion of a domain IPsec policy that is already assigned to a GPO will invalidate the GPO link.Chapter 6: Managing a Server and Domain Isolation Environment 137 • Clearing the Allow unsecured check box. The addition of a filter that does not match traffic that is already in current IPsec SAs will not affect that traffic. it is assumed that any changes have been tested on a local system or test environment before deployment in a production environment. If the clients can respond. The export function copies all IPsec policy objects in the storage location to ensure that all related objects are captured in the backup. The GPO must be edited to re-assign the latest version of the IPsec policy.

Remember that the order of a filter in a filter list has no effect on the order in which the IPsec driver will process packets. and then expand <domain name>. or modify a filter item. the MMC snap-in does not provide the ability to enter a different user ID and password when connecting to a remote computer or domain. 5. expand Windows Settings. complete the following steps. and then click Edit. 2. To make a change. right-click <IPsec policy name> and then click Assign. 7. Use the read-only mode of the IPsec Policy Management MMC snap-in in Windows Server 2003 whenever you do not intend to make changes. expand Domains. Changing an Existing IPsec Policy in the Domain Because the functionality of IPsec was extended in Windows XP and then again in Windows Server 2003. After identifying the new IPsec policy and the GPO that distributes the current policy. You can use the IP Security Policy Management MMC snap-in to perform this modification. you can replace the current IPsec policy with the new IPsec policy. Right-click GPO name. 3. To change the IPsec policy assigned to an isolation group 1. However. Ensure that <IPsec policy name> is assigned. The user must be logged on to the desktop as someone with appropriate permissions to make the intended changes. In the right pane. Be careful not to use an IPsec Policy Management MMC snap-in from an earlier release to view or edit policy that contains these extensions. Launch the GPMC. If you click OK when viewing a policy component. the MMC snap-in simply fails to save changes as if modify access was denied and use the error messages that existed at the time the product was released. Updates were made in Windows XP service packs and Windows 2000 Service Pack 4 (SP4) to detect newer versions of policy to help avoid this potential problem. The filters for all filter lists in all rules of an IPsec policy are ordered by using an internal algorithm for weighting. Changing an Existing Rule Filter List There are times when it is necessary to modify an existing rule filter list to add. and then click IP Security Policies on Active Directory <domain name>. Expand the Forest: <domain name>. the policy should be assigned locally on a computer so that the IPsec Monitor MMC snap-in or . then any changes will be lost when an access denied error occurs. As part of the change testing process.138 Server and Domain Isolation Using IPsec and Group Policy Changing IPsec Policy Assignment for an Isolation Group To change the IPsec policy assigned to an isolation group. if the user running the MMC snapin only has read permission to the IPsec policy objects. remove. Similarly. you will overwrite the existing settings with settings in current memory even if you made no changes. expand Security Settings. 6. Log on to a domain controller as a domain administrator. the IPsec policy storage format has been changed to include the settings for these extensions. 4. and then close the GPO Editor and then the GPMC. The Group Policy Management Console (GPMC) is used to change the IPsec policy that a particular GPO is distributing. Expand Computer Configuration. Finally. you must manually ensure that you are not creating a duplicate filter for any other filter used in the IPsec policy.

3. 4. Log on to a domain controller as a domain administrator. Log on to a domain controller as a domain administrator. In the IP Filters list. in the Manage IP filter lists and actions window. 12. . Right-click IP Security Policies on Active Directory. click the Exemptions filter list. 5. In the IP address text box. 2. To add a computer to a filter list 1. In the IP Filter List dialog box. In the Source address drop-down box. 4.Chapter 6: Managing a Server and Domain Isolation Environment 139 command line output can be used to view the exact filter ordering and detect duplicate filters. 9. and then click Edit. Close the IP Security Policy Management MMC snap-in. and then click Edit. type an appropriate description for the filter item. In the IP address text box. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. and then click Edit. and then click OK again. In the Destination address drop-down box. Right-click IP Security Policies on Active Directory. Log on to a domain controller as a domain administrator. Ensure that the Use Add Wizard check box is cleared. 13. and then click Manage IP filter lists and filter actions. 4. 2. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. To remove an entry from a filter list 1. type the specific IP address. On the Manage IP Filter Lists tab. To edit a computer entry in a filter list 1. change the entry to the new IP address. 8. click the Exemptions filter list. and then click OK again. Ensure that the Use Add Wizard check box is cleared. click Add. 10. 9. 3. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. 11. In the IP Filters list. click A specific IP Address. and then click Manage IP filter lists and filter actions. On the Manage IP Filter Lists tab. and then click Manage IP filter lists and filter actions. click the Exemptions filter list. 3. Click OK. Close the IP Security Policy Management MMC snap-in. Click OK. 5. Right-click IP Security Policies on Active Directory. and then click Edit. 6. 8. On the Description tab. 5. in the Manage IP filter lists and actions window. 7. in the Manage IP filter lists and actions window. click Any IP Address. Ensure that the Mirrored check box is selected. Note After the new system is added to an exemptions filter list. 2. On the Manage IP Filter Lists tab. the computer account should be added to the No IPsec security group. click the filter that corresponds with the <computer name> system. 7. click the filter that corresponds with the <computer name> system. 6.

Close the IP Security Policy Management MMC snap-in.140 Server and Domain Isolation Using IPsec and Group Policy 6. click Add. Close the IP Security Policy Management MMC snap-in. right-click <IPsec policy name>. Changing an Existing Rule Filter Action Each rule in an IPsec policy has a corresponding filter action that is performed when the rule is matched. Click OK. To add an option to an existing rule 1. and then click Edit. 2. 4. To change an existing rule filter action 1. In the right pane. 3. click <rule name>. Changing an Existing Rule Authentication Method The default authentication method in IPsec policies uses the Kerberos version 5 protocol. On the Authentication Methods tab. Log on to a domain controller as a domain administrator. 4. Although different information is needed for each authentication method that can be chosen. In the right pane. For example. Click Yes to remove the filter item. Click the button next to the new authentication option that is being chosen and then configure any options that are required. it may make more sense instead to change the filter action for a rule in an IPsec policy that already exists. right-click <IPsec policy name>. Click OK. to use a preshared key you must identify the key. To add a new authentication option to an existing IPsec rule. Note After the system is removed from an exemptions filter list. and then click Properties. the computer account should be removed from the No IPsec security group. click Remove. in the Filter Actions list. 6. and then click OK again. On the Filter Action tab. 7. You can use the IP Security Policy Management MMC snap-in to configure a rule in an IPsec policy to use a new filter action. and to use certificates the certificate authority (CA) must be known. For example. and then click Properties. Log on to a domain controller as a domain administrator. 9. . if a custom IPsec policy exists for a set of computers. In the IP Filter List dialog box. 3. it would make sense to change the filter action assigned to the rule rather than generate a new IPsec policy. In the IP Security Rules list. For example. and then click Edit. 5. and then click OK again. 2. complete the following steps. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. In the IP Security Rules list. 8. a public key infrastructure (PKI) could be rolled out so that certificates can be used to authenticate computers. 7. click <rule name>. 6. Over time it may become necessary to change an authentication method that is associated with an existing rule. Although it is possible to assign a new IPsec policy to the computers that have the new rule and filter action combination. 5. the general steps for adding an authentication method are similar. click <new filter action> to select the adjacent button.

Note To remove an authentication method. a shared authentication method must be decided upon. 2. use the Move up and Move down buttons to create the preferred order of the authentication methods. click Any IP Address. Ensure that the Mirrored check box is selected. On the Manage IP Filter Lists tab. In the IP Filter List dialog box. additional configuration must be done on the Protocol tab. In the Destination address drop-down box. 4. Log on to a domain controller as a domain administrator. 14. 8. click Add. and then configure the authentication mechanism to include the new authentication method that is chosen. Ensure that the Use Add Wizard check box is cleared.Chapter 6: Managing a Server and Domain Isolation Environment 141 7. In the Description text box. the non-managed IPsec host would need to have a policy applied to it that allows the communication to occur. Furthermore. click Add. Close the IP Security Policy Management MMC snap-in. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. Click OK. For example. type an appropriate description for the filter list. click it in the Authentication method preference order list. 9. associate the new filter list with the existing policy. In the Name text box. this procedure creates a rule that matches any traffic from any IP address to a specific IP address. 6. . In the IP address text box. Right-click IP Security Policies on Active Directory. 7. if an IPsec-capable system needs to communicate with systems in a particular isolation group but does not get its policy from the IPsec infrastructure. type an appropriate description for the filter item. In this example. Note By default. On the Description tab. 11. In the Source address drop-down box. and then click Manage IP filter lists and filter actions. click A specific IP Address. 5. A new rule could be created in the existing IPsec policy for the isolation group to allow the traffic to occur after the appropriate authentication method was agreed upon. 3. and then click OK again. and then click OK again. If matching needs to be done on a specific port or protocol basis. 12. 10. 8. type an appropriate filter list name. The necessary steps would be to create a new filter list in the directory. you can make changes to the isolation group policy to allow the communication. 13. Click OK. type the IP address of the specific computer. Click OK. 10. In the Authentication method preference order list. To create a new filter list to allow all traffic to occur to a specific computer 1. Adding a New Rule to an Existing IPsec Policy New rules are added to IPsec policies that already exist to further restrict or allow communication to occur between computers in the environment. either certificates or a preshared key could be used. 9. and then click Remove.

the host's name. the steps are specific to the role the host plays in the group. 6. To do so. 9. It is important to understand the implications of changes to group membership in terms of traffic communications. 8. Click the button next to the authentication method that is being chosen and configure any options that are required. Click OK. the policy that controls the update of the "Access this computer from the network" right must either be applied or removed. Ensure that the Use Add Wizard check box is cleared. Close the IP Security Policy Management MMC snap-in. If the system acts as both an initiator and responder. On the IP Filter List tab. click Add. and then click OK again. . Adding or Removing Hosts and Users in Existing Groups When you add a host to or remove a host from a network access group (NAG). 2. 10.142 Server and Domain Isolation Using IPsec and Group Policy To modify an IPsec policy to use a new filter list and filter action 1. follow the steps in the "Changing an Existing Rule Filter List" section earlier in this chapter. both steps must be taken. click the Filter Action option button. However. Note The authentication method that is chosen must be one that both the initiator and responder can negotiate. The exemption filter list. and its IP address must be known to complete this task. If the host acts only as an initiator. 11. Right-click <IPsec policy name> and then click Properties. if the host acts as a responder. If necessary. Adding or Removing a Host in the Exemption List You can add a host to or remove a host from the Exemption List by modifying the IPsec exemption filter lists and the No IPsec security group. it is sufficient to either add or remove it from the associated NAG. remove the Kerberos protocol from the list by selecting it and then clicking the Remove button. In the Authentication method preference order list. 7. On the Filter Action tab. hosts will periodically need to be moved from one group to another. in the Filter Actions list. Moving Hosts Between Isolation Groups For various reasons. such as a preshared key or certificates. Click OK. On the Authentication Methods tab. 5. click the New Filter List option button. 4. The following sections describe the steps for adding or removing hosts from groups. use the Move up and Move down buttons to select the preferred order of the authentication methods if more than one authentication method is listed. Click Add. 3. in the IP Filter list.

right-click the <NAG> and then click Properties. 4. and then launch Active Directory Users and Computers. Expand the domain.Chapter 6: Managing a Server and Domain Isolation Environment 143 Adding or Removing Initiators in an Existing Network Access Group You can add or remove an initiator in a network access group by using standard group management tools to modify the associated security group. Expand the domain. In the right pane. d. Click Yes to remove the <computer name> account. and then click Users. To modify a NAG relative to a specific computer 1. Click the Object Types button. In the Members list. Note There will be a delay between the time the host account is added to the group and when the host can access the restricted resource. 5. then this procedure is not necessary. In the Enter the object names to select text box. click <computer name> and then click Remove. To add a computer to the group: a. 3. Click the Members tab. If there is no requirement to restrict a resource in a manner that is similar to a NAG. Log on to a domain controller as a domain administrator. This delay is caused by replication delays and the time between updates of the session ticket on the server that is hosting the restricted resource (if the ticket is cached). You can add or remove a restricted user from a NAG Users group by using standard group management tools to modify the associated security group. and then click Users. 2. and then click OK. and then click Add. select the Computers check box. d. If there is a requirement to restrict a resource. Click OK. Click OK. 2. c. then the Domain Users group is granted the "Access this computer from the network" right on the responders. c. Click the Members tab. if the Domain Users group is being used. and then launch Active Directory Users and Computers. Adding or Removing Users in an Existing Network Access Group Although isolation groups were created to restrict which hosts can initiate communication to the restricted resource. To remove a computer from the group: a. they can also be used to help restrict which users have access to a resource. b. Log on to a domain controller as a domain administrator. type <computer name> and then click OK. To modify a NAG Users group relative to a specific user 1. then a NAG Users group is created. . b. This procedure is only required if a NAG Users group has been created and assigned to the NAG.

if two servers are in the same isolation group and will never initiate communications to each other. Note There will be a delay between the time the user account is added to the group and when the user can access the restricted resource. Click Yes to remove the <user name> account. 5. b. and then click Add. click the specific <user name>. . If a server acts as a responder within multiple NAGs. The GPO application can be controlled through any of the standard means of ensuring policy application with Active Directory. the approach used in this guide assigned the GPO to an organizational unit (OU) that was created to hold the domain computer accounts of responders. Adding New Network Access Groups Creation of a new NAG is a fairly straightforward process. In other words. In the right pane. c. and then click Remove. d. you can remove the GPO assignment that configures the "Access this computer from the network" right on the responder. (If the computer account was also a member of a domain local security group that consisted of network access groups. then it must also be removed from that group. you must create a domain local group to control the access to the resource and a GPO to update the "Access this computer from the network" right on the hosts that act as servers in the NAG. Click the Members tab. type <user name> and then click OK. This delay is caused by replication delays and the time between updates of the session ticket on the server that is hosting the restricted resource (if the ticket is cached). Adding or Removing Responders in an Existing Network Access Group To remove a responding host (the responder) from an existing NAG. and access will no longer be restricted. Only initiators need to be members of the NAG. Click the Members tab. Click OK. To remove a user from the NAG: a. Click OK. additional GPOs may be required so that specific computers meet this need. The computer would revert to the Isolation Domain policy. However. right-click the NAG Users security group. Simply moving the computer account out of the responders OU will cause it to no longer receive the assigned GPO. and identify the hosts that belong in the group. 4. all NAG security groups in which the server participates are present on that system's "Access this computer from the network" right. Then you must apply that GPO to the servers. and then click Properties. care must be taken to ensure that after the GPO is applied. To add a user to the NAG: a. In the Members list. If necessary. they do not need to be added to the NAG for their isolation group. c. they will need to be added to the NAG like all other initiators. In the Enter the object names to select text box. b. First.144 Server and Domain Isolation Using IPsec and Group Policy 3.) Care should be taken to ensure that hosts that are members of multiple NAGs are still able to communicate with other NAGs after they are removed from one of the NAGs. However if these two servers need to communicate.

Right-click the newly created group and then click Properties. and then click Group. and then click Add. 5. In the Group name text box. 4. 7. 2. To populate the new NAG for initiators with the initiator accounts 1. Log on to a domain controller as a domain administrator and then launch Active Directory Users and Computers. 5. and then click Properties. Adding Initiator Computer Accounts to a Network Access Group Complete the following steps to populate a new NAG with initiator accounts. In the Description text box. Click the Object Types button. type an appropriate name for the group. type an appropriate description for the group. 3. select the Computers check box. right-click the NAG initiators group. 3. Click OK. In the Group name text box. Otherwise. Right-click the newly created group and then click Properties. 2. Click the Domain local security group and then click OK. click New. Log on to a domain controller as a domain administrator and then launch Active Directory Users and Computers. click New. . Click the Members tab.Chapter 6: Managing a Server and Domain Isolation Environment 145 Creating a New Network Access Group for Initiator Computers Complete the following steps to create a new NAG. Expand the domain. 7. and then click Group. 4. a group for the restricted NAG users must be created. 4. To create a new NAG for initiator computers 1. Right-click the Users container. Click the Domain local security group and then click OK. 6. Right-click the Users container. 3. In the Enter the object names to select text box. If there is a requirement to further restrict which users in the domain are allowed to access the restricted resource. 2. 5. type an appropriate name for the group. the Domain Users group can be used instead. Log on to a domain controller as a domain administrator and then launch Active Directory Users and Computers. To create a new NAG for user accounts 1. 6. Creating a New Network Access Group for Restricted Users Complete the following steps to create a NAG for restricted users. and then click OK. In the right pane. and then click Users. Click OK. type the <initiator name> and then click OK.

Table 6. The administrator will need to determine whether there are any additional groups that should be granted this right. 3. 7. and then click Properties. and then click Users. Expand the domain. The Domain Users group is added as a default.146 Server and Domain Isolation Using IPsec and Group Policy 6. The following table provides an example of a GPO implementing a NAG and the associated group names that need to be granted the "Access this computer from the network" right. 5. <NAG Implementation Policy Name> <NAG name> To create a GPO to grant the Access this computer from the network right 1. To populate the new NAG with user accounts 1. In the Description text box. Log on to a domain controller as a domain administrator. Log on to a domain controller as a domain administrator and then launch Active Directory Users and Computers. Launch the GPMC. Adding Restricted User Accounts to a Network Access Group Complete the following steps to populate a new NAG with restricted users. . type <user name> and then click OK. Click the Members tab. Expand Forest: <domain name>. In the right pane. expand Domains. 4.1 Example NAG Policy Definition GPO name Group name Administrators Backup Operators NAG Users or Domain Users Note The listed groups are the minimum that should be added. If the administrator also wishes to restrict users as well as computers. 2. Click OK. Creating a GPO to Grant the "Access This Computer from the Network" Right A GPO is used to assign the "Access this computer from the network" right to the appropriate NAGs. right-click the NAG Users group. a NAG Users group will need to be created like the one for computer accounts that contains the selected user accounts. 2. In the Enter the object names to select text box. Click OK. and then click Add. and then expand <domain name>. 6. type an appropriate description for the group. 3.

Chapter 6: Managing a Server and Domain Isolation Environment

147

4. Right-click Group Policy Objects, and then click New. 5. In the Name text box, type <GPO name> and then click OK. 6. Right-click <GPO name>, and then click Edit. 7. Expand Computer Configuration, expand Windows Settings, expand Security Settings, expand Local Policies, and then click User Rights Assignment. 8. In the right pane, right-click Access this computer from the network, and then click Properties. 9. Select the Define these policy settings check box. 10. Click the Add User or Group button. 11. Click the Browse button. 12. In the Enter the object names to select text box, type the name of each group listed in the previous table, separated by semicolons. Then click OK. 13. Click OK. 14. Close the GPO editor and then the GPMC.

Deploying Network Access Group GPOs
To deploy NAG GPOs, they first need to be linked to a location within the domain environment so that they can be applied to the appropriate responders within the NAG. The GPO application can be controlled through any of the standard methods for ensuring policy application with Active Directory. It is beyond the scope of this guidance to provide specific steps, because they would be dependent on the OU structure and management methods employed within the organization.

Disabling IPsec in an Isolation Group
You can disable an IPsec policy by modifying the GPO that delivers the policy. To disable the IPsec policy, the GPO is configured so that the computer settings are disabled. To disable the computer settings of the GPO 1. Log on to a domain controller as a domain administrator. 2. Launch the GPMC. 3. Expand Forest: <domain name>, expand Domains, expand <domain name>, and then expand Group Policy Objects. 4. Right-click <GPO name>, click GPO Status, and then click Computer Configuration Settings Disabled. 5. Close the GPMC.

Re-Enabling IPsec in an Isolation Group
You can re-enable IPsec policies that have been disabled by modifying the GPO that delivers the policy. To re-enable a disabled IPsec policy, the GPO is configured so that the computer settings are enabled. To enable the computer settings of the GPO 1. Log on to a domain controller as a domain administrator. 2. Launch the GPMC.

148

Server and Domain Isolation Using IPsec and Group Policy

3. Expand Forest: <domain name>, expand Domains, expand <domain name>, and then expand Group Policy Objects. 4. Right-click <GPO name>, click GPO Status, and then click Enabled. 5. Close the GPMC.

Removing IPsec from an Isolation group
You can remove an IPsec policy by modifying the GPO that delivers the policy. To remove the IPsec policy, the GPO is configured so that the IPsec policy is no longer assigned. To unassign the IPsec policy of the GPO 1. Log on to a domain controller as a domain administrator. 2. Launch the GPMC. 3. Expand the Forest: <domain name>, expand Domains, and then expand <domain name>. 4. Right-click <GPO name>, and then click Edit. 5. Expand Computer Configuration, expand Windows Settings, expand Security Settings, and then click IP Security Policies on Active Directory <domain name>. 6. In the right pane, right-click <IPsec policy name>, and then click Un-assign. 7. Ensure that <IPsec policy name> is unassigned, and then close the GPO editor and then the GPMC.

Backup/Restore Considerations
This section provides information about how to evaluate the processes that specifically deal with backup and restoration of the server and domain isolation solution components.

Active Directory Backup
IPsec policies are not stored in the Group Policy objects that are used to deliver the policies. Group Policy backup and restore capabilities will only capture information about which IPsec policies are assigned to Group Policy objects, not the actual IPsec policy information. Although a full System State backup of a domain controller will capture the IPsec policy information, it is also possible to use the IP Security Policy Management MMC snap-in's Export Policies and Import Policies menu commands to back up and restore IPsec policies.
Note It is important to secure your IPsec policy backups. However, the backup is a file that inherits the NTFS file system permissions of the directory in which it is stored, and the data in the file is not encrypted or signed. You should protect the IPsec configuration information in these files by using appropriate permissions or security procedures. Only authorized IPsec administrators should have access to these backup files.

For more information about backing up System State data on a computer running Windows Server 2003, see the Back up System State data page.

Chapter 6: Managing a Server and Domain Isolation Environment

149

Host Restoration
On computers for which IPsec policy has been restored from backup (either a tape backup or an image-based backup), the IPsec policy that is applied might be a cached copy of the Active Directory-based IPsec policy or a local IPsec policy. If the computer is assigned Active Directory-based IPsec policy, the IPsec service attempts to retrieve the latest copy of the assigned IPsec policy from Active Directory before it applies the cached copy of the Active Directory-based policy. When doing so, the IPsec service first queries Domain Name System (DNS) for the current list of the IP addresses of all of the domain controllers. If the IPsec policy objects have been deleted from Active Directory, the cached copy of the Active Directory-based policy is applied instead. The list of domain controller IP addresses in the cached copy of the Active Directorybased IPsec policy might have changed substantially since the IPsec policy backup was created (for example, if new domain controllers were added). If so, communication might be blocked with current domain controllers—and therefore authentication that used the Kerberos protocol will fail when attempts are made to establish IPsec-secured connections remotely. In addition, the computer might not be able to receive Group Policy updates. This problem can be resolved as follows: 1. Access the computer locally, and stop the IPsec service on that computer. 2. Restart the computer in Safe Mode with Networking, and either configure the IPsec service to start manually or disable the IPsec service to allow IPsec-secured communication with the IP addresses of the new domain controllers.

Mitigation of Network-Based Infections
Some circumstances may require rapid disruption of communications to ensure the security of the environment, such as when a virus outbreak or security intrusion occurs. The following sections discuss various ways to isolate hosts that participate in authenticated communications. By design, these methods do not isolate the infrastructure or exempted servers, because care must be taken not to isolate the infrastructure servers so that the systems do not lose the ability to update their IPsec policies from the domain.
Note Although these methods of isolation are technically sound, they have not been tested in a lab environment. It is strongly suggested that you test these methods in a lab environment before relying on them.

Isolating the Isolation Domain
Hosts in the isolation domain are allowed to initiate communications with untrusted hosts. If there is a need to quickly block this type of traffic, the IPSEC – Secure Request Mode (Ignore Inbound, Allow Outbound) filter action can be modified so that the "Allow unsecured communication with non-IPsec-aware computers" right is disabled. After the IPsec polling period has elapsed, all hosts in the isolation domain should be blocked from communicating with systems that are not participating in the IPsec environment. To modify the IPSEC – Secure Request Mode (Ignore Inbound, Allow Outbound) filter action 1. Log on to a domain controller as a domain administrator. 2. Launch the IP Security Policy Management MMC snap-in and focus it on the domain. 3. Right-click IP Security Policies on Active Directory, and then click Manage IP filter lists and filter actions.

150

Server and Domain Isolation Using IPsec and Group Policy

4. On the Manage IP Filter Actions tab, click the IPSEC – Secure Request Mode (Ignore Inbound, Allow Outbound) filter action, and then click Edit. 5. Select or clear the Allow unsecured communication with non-IPsec-aware clients check box. 6. Click OK. 7. Click OK. After this option has been set, the policy will block all network traffic that is destined for untrusted hosts. After the issue has been resolved, the communication can be restored by re-enabling the option.

Blocking Ports
IPsec policies that are deployed to internal organizational local area network (LAN) computers are configured to allow all communication across all ports. This approach simplifies the configuration and management of the environment. However, if a host using IPsec becomes infected with malware such as a virus or worm, the host will likely spread the infection to other computers. Depending on the policies the computer is using, the infection could spread to both trusted and untrusted hosts. IPsec policies can be used to help reduce the spread of malware by explicitly blocking the ports that malware uses. The main limitation to this approach is the delay required for all computers to detect the policy change that adds the blocking filters. In addition, some worms have flooded the network and made it difficult for IPsec policy changes to be retrieved. And some worms have used ports that are also used by critical services such as DNS, which would make it difficult to update the policy after blocking filters were applied on the host. Blocking can be accomplished by creating a rule that blocks traffic from any IP address to the specific port that a certain form of malware uses. This rule is added to all policies in the environment. After the malware is removed, the rule can be removed from the policies. After you identify the ports and protocols that a certain form of malware uses, create a filter list that matches the criteria of the malware communication by following the steps in the "Adding a new rule to an existing IPsec policy" procedure in the "Changing IPsec Policy" section earlier in this chapter. The IPsec policy polling interval should be reduced right away as soon as the decision is made to use port blocking in domain policy. The polling interval can be increased again once the threat has subsided. However, instead of creating a filter that uses any IP address to a specific IP address, create a filter that uses "My IP Address" to any IP address. Typically, mirrored filters are not used. A filter list that contains two one-way filters is required, one for inbound to a well known port and one for outbound traffic to a well known port. For example, the following filters block the SQL port 1433 exploited by the SQLSlammer worm: From Any IP Address -> My IP address, TCP, src *, dst 1433, not mirrored From My IP Address -> Any IP address, TCP, src *, dst 1433, not mirrored Clearly, these filters will also block the SQL application connections and would be removed when the worm threat had subsided. Use caution not to block access to critical infrastructure ports such as DNS unless absolutely necessary. These filters are more specific than the Woodgrove Bank subnet filters that negotiate IPsec for all traffic on the internal network because they have a specific IP address defined. After the filter is created, add a rule to all isolation domain and group IPsec policies to associate the filter list with the IPsec – Block filter action. You may want to include in your policy design a rule that already associates an empty IPsec Filter List used for blocked ports with the block action. This empty filter list could be used by rules in all IPsec

Chapter 6: Managing a Server and Domain Isolation Environment

151

policies and enabled so that all domain members will check this filter list on each polling interval. Or the rule could be disabled, and the IPsec service polling would detect when the rule is enabled in each isolation group policy. If for some reason the port blocking prevents IPsec from accessing Active Directory to obtain an updated policy, then the IPsec service can be administratively stopped and restarted on the computer, or the computer can be restarted. When the IPsec service starts, it will attempt to download the latest version of the assigned IPsec policy before applying the older version in the cache.

Isolation to Within Child Domain Only
If an entire domain needs to be isolated from the rest of the domains in the forest, the policy for that domain can be configured to use a preshared key rather than the Kerberos protocol. This approach will allow computers within the child domain to maintain communications with other systems in the same domain, but it will block communication with systems outside of the domain to which they usually would have access. Each policy in the child domain would need to be modified so that it used only a preshared key for the IPsec – Secure Organization Subnets rule. Any existing authentication methods, such as the Kerberos protocol, will need to be removed. To configure the authentication methods, follow the steps in the "Changing an Existing Rule Authentication Method" section earlier in this chapter. If additional rules that perform authentication exist in the policy, they will also need to be configured to use a preshared key. All policies in the child domain that is to be isolated will need to be configured in this way. To minimize the chance of IKE main mode authentication failures as the policy is rolled out, the preshared key authentication method can be ordered first in the authentication method list, followed by the Kerberos method. After all computers have the updated policy, the Kerberos authentication method can be removed. A similar process is used to restore authentication for the Kerberos protocol and remove the preshared key after the threat has subsided.

Isolation to Predefined Groups
Although network access groups are one implementation that can be used to isolate a predefined group of computers, preshared keys or certificates can also be used to perform the same isolation. The primary difference from network access groups is that you will need to create separate policies for each group of computers to secure the traffic between the computers that have a preshared key or certificate. This solution requires additional traffic communication planning, particularly if a system belongs to more than one group. The major drawback to preshared keys is that they are stored in plaintext in the policy and thus are easily discovered (their secrecy is compromised) from a client within the domain. This drawback may not be a concern if the preshared key value is being used simply for temporary isolation during a worm outbreak. Because of a limitation in how IKE checks certificate constraints at the root CA rather than at the issuing CA, a unique root CA would need to be deployed for each group.

and procedures for managing. these processes and procedures should be designed to help ensure that no errors are introduced as a result of someone not understanding the ramifications of a policy change. processes. The processes and procedures should be well documented and communicated to all staff who are involved in the day-to-day management of the hosts in the environment. . maintaining. and modifying a server and domain isolation solution after it has been successfully deployed and made operational.152 Server and Domain Isolation Using IPsec and Group Policy Summary This chapter provided information. Because there is always the potential for a small change to an IPsec policy to disable a protected communications path.

These recommendations and tools should be adapted to meet the specific needs of your organization. Support Tiers and Escalation Within Microsoft. When IPsec is used to secure Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) traffic on the network. client issues were handled by the Tier 2 support teams.) Help desk staff members are trained to use a taxonomy and several flowcharts for classifying problems that relate to server and domain isolation. and the help desk is referred to as Tier 1 support. and Tier 3. Escalation procedures enable the help desk staff to escalate incidents that require the assistance of specialists. and is based on the experience and processes of the Microsoft Information Technology (IT) team. Where possible. . Initial Tier 1 guidance is provided to help an organization determine as quickly as possible if a problem is related to IPsec and. The help desk is the entry point for both domain-joined and nondomain-joined client issues. The procedures in this chapter refer to three levels of support: Tier 1. such as in server and domain isolation scenarios. The highly detailed and complex information that would be required to support Tier 3 troubleshooting efforts is outside the scope of this chapter. client issues were escalated to the Corporate IT Security department. this chapter refers to existing Microsoft troubleshooting procedures and related information. Many of the support procedures. most of the content is at the Tier 2 level. and scripts that are used by Microsoft are provided in this chapter for reference purposes. Microsoft IT support is based on a multi-tiered support model. it is important to plan for and develop IPsec-specific troubleshooting techniques that can be used if an issue arises between computers that use (or attempt to use) IPsec for their communications. server and domain isolation support is a standard offering and is defined in standard service level agreements (SLA). after the solution was deployed into production. to generate the required information to help Tier 2 support engineers troubleshoot the problem. Tier 2. (Other servers may be managed by line of business application teams or product groups. During the pilot phase of the Microsoft isolation solution. if it is. Isolation support is provided by the following tiers: • Tier 1: Help desk. However. For this reason. tools. The help desk also supports servers that are managed by the central IT organization.Chapter 7: Troubleshooting IPsec This chapter provides information about how to troubleshoot Internet Protocol security (IPsec). To ensure that the guidance is as practical and concise as possible. typical TCP/IP network troubleshooting procedures and tools can become ineffective. Microsoft recommends that you contact Microsoft® Product Support Services to obtain additional assistance. If the information provided in this chapter does not fix the IPsec problem.

and messaging/collaboration support . • The following section summarizes the troubleshooting techniques that can be used by the help desk staff in the Tier 1 support organization. Tier 3: Windows network and infrastructure services . If someone thought it might be related to IPsec. a flowchart should be defined and updated for each set of isolation changes being implemented. Each group has a subject matter expert for server and domain isolation. The goal of the help desk should be to categorize the problem so that known solutions can be attempted. these teams were the initial escalation point for help desk and Corporate IT Security for server-related issues and troubleshooting. Tier 3 works directly with the Windows Development teams until closure is reached. who provide Tier 1 support. Outside of Microsoft. After appropriate administrative information is obtained from the caller. Is IPsec the Problem? The help desk is likely to receive calls such as “I was able to connect to server x until IPsec was turned on" or "Everything worked yesterday. and network logon rights. . Within Microsoft. this level would engage with Microsoft Product Support Services when necessary. help desk staff should follow a defined troubleshooting process. Use ping <destination name> and nslookup. If these attempts do not resolve the problem. global network operations center. this group identified a team of people to be experts in troubleshooting the solution-related architectural components and technologies. computer accounts. Help desk management personnel must be involved in this planning process. but others do not when communicating with the same destination. Name resolution. today I can’t connect to anything!" In the experience of Microsoft IT. if further escalation is necessary. Tier 1 support personnel are phone-based help desk staff members who attempt to diagnose problems remotely. then help desk personnel can ensure the proper information is collected and escalate the problem to Tier 2 support. Because IPsec policy designs may vary in their impact on communications.• Tier 2: Data center operations. as well as detailed procedures for troubleshooting. A server and domain isolation implementation plan should include a call classification system so that help desk personnel can provide clear reports about the volume and nature of IPsec-related problems. Applications. Some applications work (for example. the rollout of IPsec increased call volumes for all types of network connectivity issues and "access denied" incidents because people were paying increased attention to application and network behaviors. During server and domain isolation pilots. TCP/IP packet processing. Tier 1 Troubleshooting This section presents the overall process for troubleshooting IPsec-related problems that is used by help desk staff. For server and domain isolation pilots. line of business application support. net view). and because the rollout process may take several days or weeks. Use ping and tracert Internet Control Message Protocol (ICMP) messages to test network paths. the help desk should be able to identify various types of problems in the following ways: • • • Network connectivity. For example. they called the help desk. such as IPsec. Typically. These groups are the day-to-day operations teams that monitor and manage IT services and related assets.

it is likely that all TCP and UDP traffic will be encapsulated. some computers may experience problems with connectivity and others not. This section reviews the different types of problems and common issues for each that Tier 1 support must handle. or are there more extensive problems such as logon or authentication failures across large parts of the network? Problems with connectivity can involve many different layers and technologies that are used in network communications. Therefore. Furthermore. • Network location and path-specific problems. the help desk may use Remote Assistance or Remote Desktop to connect to the caller's computer. Determine whether it can access any host or specific trusted host destination computers that are used for help desk testing and diagnosis. For example. how many and where they are. network devices along the path will only see only IKE. Computers that initiate communications and computers that respond to communications must have valid domain accounts and access to domain controllers for their domain. The target computer. Assign Scope and Severity One of the first questions that Tier 1 support must address is: who is affected by the problem? Support personnel need to understand if the problem is shared by other users and. The network connection is using a specific IP address or communicating using a particular network path. • Computer-specific problems.Chapter 7: Troubleshooting IPsec 155 • Services. The caller’s computer. In a server and domain isolation solution or other widespread deployment of IPsec. as well as specific issues related to the solution. In scenarios where server isolation is used without domain isolation. • • Depending on the organization. determine whether the server is running the Routing and Remote Access (RRAS) service. The support staff must then look at the extent of the problem. The computer has certain software installed or particular services running. The guidelines provided in this chapter do not require remote access. then communication may be blocked between the two computers. help desk personnel should be aware of which servers are members of the isolation group. Support engineers should be aware of how Windows TCP/IP network communications work in general. Because of these types of issues. Determine whether the caller's computer can access all help desk computers that are used for testing but cannot access a certain destination computer. IPsec policy assignment and network access controls depend upon computer accounts being in the correct domain groups. If there are any network problems in the transmission of these three protocols between the source and destination. Other computer-specific issues that may affect IPsec behavior include the following: • • • The operating system does not have the correct service pack. patch or registry key configuration. Note All of the IPsec troubleshooting tools discussed in this chapter require local administrator group privileges. which creates a conflicting automatic IPsec policy for L2TP. For example. does it affect connectivity to a single server. IPsec-protected communications require mutual Internet Key Exchange (IKE) computer authentication. although they may be useful tools for help desk personnel to use as an alternative to guiding the caller through the IPsec Monitor Microsoft Management Console (MMC) snap-in or the Event Log viewer. . if so. IPsec and ICMP protocols.

In addition to standard tools. a description of which is provided in the "Support Script Examples" section later in this chapter. If unsuccessful. Is it an application problem? If so. For example. two of the flowcharts refer to an IPsec policy refresh script. or only on certain computers. If unsuccessful. and the application of IPsec policies that are based on domain membership and group membership. go to Figure 7. Troubleshooting Flowcharts The call handling flowcharts in this section were developed by Microsoft IT to help classify Tier 1 IPsec support problems. escalate to Tier 2 support. . There may be differences in behavior between domain and local user or service accounts.156 Server and Domain Isolation Using IPsec and Group Policy • User-specific problems. escalate to Tier 2 support. Is it an IPsec problem with the target computer the caller is trying to reach? If so. go to Figure 7. Is it an IPsec problem with the caller's computer? If so. when using a certain IP address (for example. the problem may only affect users who are not in an authorized group for network access. escalate to Tier 2 support. can affect the network logon rights of domain users. connectivity problems may appear in only certain parts of the network.1 is used for initial diagnosis and to determine the type of problem: • • • • • Is it a network connectivity problem? If so.3. such as in a server and domain isolation scenario. if there is a problem with the design of the subnet filters or the network path used by that computer to reach other computers. Is it a name resolution problem? If so. Figure 7. The deployment of IPsec. regardless of where a computer is located on the internal network. a wireless address and not a LAN address).2. Consequently. attempt basic name resolution troubleshooting. or an authorized user may have problems obtaining Kerberos authentication credentials that contain the proper group memberships. attempt basic network troubleshooting. Two other features of the server and domain isolation solution that are also typically found in enterprise deployments of IPsec are the use of subnet filters to define the address ranges used on the internal network.

Note that in addition to diagnostics. Figure 7. escalate the issue to Tier 2 support. Is it none of the above? If IPsec policy refresh and/or creating a domain account do not solve the problem. Is it a domain account issue? If so.2 help determine the following potential problems with the caller's computer: • • • • Is it an RRAS issue? If so. The steps in Figure 7. try to refresh the Group Policy and the IPsec policy. create a domain account for the caller's computer. either stop the RRAS service (if RRAS is not required) or escalate the problem to Tier 2 support.Chapter 7: Troubleshooting IPsec 157 Figure 7. this flowchart references the use of an IPsec policy refresh script (see "Support Script Examples" later in this chapter).2 is designed to help identify problems with the caller's own computer. which may fix the problem without necessarily identifying it. Is it a policy issue? If so.1 Troubleshooting process for failure to communicate with a target computer Note This flowchart assumes the caller computer is running IPsec and that DNS reverse lookup zones are configured to allow the correct operation of the ping –a command. .

escalate to Tier 2 support. escalate to Tier 2 support.3 helps determine the following potential problems with the target computer (or the path to it): • • • • Is it a RRAS issue? If so. Then check network connectivity.3 is designed to help identify problems with a particular target computer. Figure 7. .158 Server and Domain Isolation Using IPsec and Group Policy Figure 7. Is it a network connectivity issue? If so.2 Troubleshooting caller computer IPsec-related problems Figure 7. Is it an IPsec policy issue? If so. try to refresh the Group Policy and the IPsec policy. Is it a logon right issue? If so. escalate to Tier 2 support. Note that this flowchart also references the use of an IPsec policy refresh script that may fix the problem without necessarily identifying it.

• . Fixed unclear. For example. This status means the problem is still outstanding but with likely problem issues identified as the issue is escalated to Tier 2 support.3 Troubleshooting target computer IPsec-related problems After the Tier 1 support staff has worked through the flowcharts. or no policy at all. an IPsec policy refresh may solve the problem but does not necessarily explain why an incorrect policy.Chapter 7: Troubleshooting IPsec 159 Figure 7. came to be applied. Not fixed. This status means the issue is resolved the issue but the reason for the problem is not fully understood. This status means the problem has been resolved and the reason for the problem may have been determined. the problem status will be one of the following: • • Fixed understood.

• • • • • Help desk personnel should also be trained to be wary if a caller asks them to connect to their computer IP address to see what it wrong—for example. For this reason. because in other security solutions such critical information is usually only available to higher-level support teams. thereby revealing logon names or e-mail addresses. The list of IP addresses in the exemption list filters is likely available to local administrators on all trusted hosts by using the IPsec Monitor MMC snap-in. or whether network intrusion detection is being used is very helpful for an attacker. attackers must scan the network to detect exempted computers. For example. help desk personnel should be trained in how to detect and resist social engineering attacks. Telnet. such as computers that are members of the exemption list. the attacker's computer can learn information about the password or (in some cases. help desk personnel may become aware of specific areas within the IT environment that are not protected by IPsec. If this information is known. The servers that contain the most sensitive information usually have the most security protections in place. which will be able to respond to TCP and UDP connection requests. such as with Telnet) steal the password. After domain isolation is fully implemented. or by viewing the domain IPsec policy cache in the local registry. In a social engineering attack. The following information should be carefully controlled by help desk personnel: • Members of the exemption list. an untrusted person attempts to gain information about how security is implemented and where security is weak. DHCP servers. often by simply taking advantage of the human tendency to trust other people. and domain controllers are easy to locate by using either a DNS query or a UDP Light Directory Access Protocol (LDAP) query. Computers that do use server isolation or require machine-based access control.160 Server and Domain Isolation Using IPsec and Group Policy Prevention of Social Engineering Attacks In an isolation solution. Remote Desktop. If a help desk person makes the connection without IPsec. knowledge of whether firewalls exist. For example. an attacker can then focus their attack on the most sensitive and valuable parts of the network. whether router filters permit certain traffic. Users who are administrators or have special roles in the IT organization . They may not be used to protecting sensitive information. if an attacker asks someone at the help desk to connect to their computer using file sharing. and WINS servers are easily identified from the DHCP configuration. the security settings used in the organization may provide nonadministrative users with read access to the cache. In some cases. Computers in the organization that are not participating in the isolation solution. or they do not require strong password protections before revealing user identity or password-related information. e-mail addresses are used as computer names or part of the computer name. Subnets that are being used for specific purposes or by certain organizations . . or other network protocol. Other network-based security measures that are being used . certain domains or server types may not be included in the solution. This situation can occur because some client network protocols do not first authenticate and establish a strong trust with the destination computer. Note that DNS servers. In addition.

This information may be found using various Windows tools. a number of scripts were developed to provide key information without requiring Tier 1 support staff to have detailed knowledge of tool operations and syntax.vbs Increases Group Policy logging Increases Kerberos version 5 authentication protocol logging Purges current Kerberos protocol tickets Refreshes Group Policy Enables IPsec logging Performs PING and SMB (Net View) tests Detects IPsec file versions Runs policy and network diagnostic tests Copies IPsec 547 events to a text file Disables IPsec logging Restores Kerberos protocol logging Restores Group Policy logging This script also enables all IPsec-related logs for troubleshooting by Tier 2 support. such as those referenced in the flowcharts. They are described in this chapter to illustrate how scripts can be used to support the troubleshooting process. These scripts are examples of the customized scripts used for the Woodgrove Bank environment that is detailed in this guide. To avoid loss of connectivity for remote desktop sessions. These scripts are available in the Tools and Templates folder of the download for this guide. They should be used as a basis for an organization's own customized solution. a solution can be quickly determined after the right information is identified.vbs The script performs the following functions: • • • • • • • • • • • • • • Discovers the operating system version Calls Detect_IPsec_Policy. Use the following syntax to run the script at a command prompt: cscript IPsec_Debug. this script may actually fix some problems. Note These scripts are tested examples but are not supported by Microsoft. help desk personnel can have them run one of three scripts provided with this solution. . forces a Group Policy refresh to reload the current domain-assigned IPsec policy from the Active Directory® directory service.Chapter 7: Troubleshooting IPsec 161 Support Script Examples For most troubleshooting scenarios. and updates the policy cache. In the Woodgrove Bank solution.vbs In addition to providing debug information. It stops and restarts the IPsec service (which deletes all current IKE and IPsec security associations). Scripts Available for Tier 1 Support If the user is a local administrator of their computer. the script should be downloaded to the caller's computer and run locally by an account that has administrative privileges. IPsec_Debug.

It refreshes computer Kerberos authentication protocol tickets and Group Policy. The caller's machine name so that the next support tier can identify the log file generated by the script.vbs script.vbs. so that escalation can be directed to the proper support group. as skilled network support engineers. logs only capture information. Tier 2 validates issues and reviews the steps taken by Tier 1 to ensure that no troubleshooting steps were missed.vbs Escalation When help desk personnel need to escalate a likely IPsec problem. Tier 2 support staff members should be able to use their skills and experience (listed in the following section) to successfully resolve the problem through log analysis without gaining administrative control of the computer.vbs Note This script is also called from IPsec_Debug. Use the following syntax to run the script at a command prompt: cscript Detect_IPsec_Policy. Tier 2 Troubleshooting Preparation Tier 2 support has two main roles. It is not expected that a Tier 2 support person should be a domain administrator or be able to make changes in domain-based IPsec policy or computer group memberships. and not a misdiagnosis. and therefore does not need to be run in addition to that script. as the recipient of all Tier 1 escalations. Refresh_IPsec_Policy. Second.vbs This script determines whether the computer is running the correct IPsec policy by checking the current local registry cache for policy version information for the domain IPsec policy. . and may fix the problem if it is caused by an incorrect IPsec policy assignment or a Group Policy download failure. First. Tier 2 should confirm that any escalated issue is really due to IPsec. Use the following syntax to run the script at a command prompt: cscript Refresh_IPsec_Policy. The destination computer to which access is denied. In this respect.vbs This script is the IPsec policy refresh script referenced in the troubleshooting flowcharts. and corrective actions require administrative access. However. the following information should be collected by Tier 1 and passed with the service request: • • • Log files generated with IPsec_Debug.162 Server and Domain Isolation Using IPsec and Group Policy Detect_IPsec_Policy. Server isolation scenarios often have their own support team to investigate membership of network access groups.

Unless the mistake is realized immediately. This type of mistake results in a situation in which communications between a client and a domain controller would be required to use IPsec. Understand and troubleshoot the information that is gathered in a network trace. an IPsec policy may block communication if the remote computer does not have an appropriate IPsec policy. Administrators must carefully plan any policy changes and ensure that procedural safeguards exist to mitigate this type of situation. Be able to perform the following tasks: • • • Issues Inherent with the Use of IPsec As indicated in the previous section. replication of the damaging policy is not easily stopped. In a worst case scenario. • IPsec configuration. Troubleshoot the network protocol stack on a host machine.Chapter 7: Troubleshooting IPsec 163 Tier 2 Support Skills Support staff that provide Tier 2 IPsec support should have skills and expertise in the following areas: • Group Policy. Verify IPsec filter configurations. how they are assigned. and be able to perform the following tasks: • • • • • Check access control lists (ACL) on Group Policy objects (GPO). or just the domain policy to use local policy for testing. it may not be immediately apparent whether it blocks network connectivity with one or more computers and causes any application warnings or errors. any client that inherits this policy would not be able to complete the logon process— because they would be unable to obtain the required Kerberos ticket to secure the communications. but they also must be able to isolate problems related to other technology components. Because the authentication used in this solution relies on the Kerberos protocol. Authentication failure identification. Troubleshoot network path problems. including TCP Path MTU discovery and virtual private network (VPN) remote access solutions. Experience with third-party software used by the organization . For successful IPsec communication between two computers. • Be able to verify that a domain computer account is OK by using the netdiag and nltest utilities. Disable IPsec entirely. Check GPO settings. Know what policies should be assigned. Be able to perform the following tasks: • • • • • Networking. Check group memberships for computers and users. Troubleshoot the IPsec IKE negotiation process and security protocols. with a correct assignment that quickly replicates after the original assignment. Tier 2 support personnel for a server and domain isolation solution will need to know the details of IPsec-protected communications. For example. Although this may be an intended or acceptable behavior during the rollout of a policy change. an administrator might accidentally assign an IPsec policy to all domain members that blocks all traffic. both computers usually require a compatible IPsec policy. Reload IPsec domain policy. .

some of the procedures documented in the background guides may not work at all. then most of these procedures and tools will probably become ineffective. Therefore. If the client Group Policy system experiences errors in detecting GPO changes or in downloading them. Therefore. many of the procedures referred to in these guides will only work while IPsec is providing successful connectivity. deleted. The remainder of this chapter uses these IPsec terms. The type of failure will depend on the tool and the IPsec environment. the information provided in the 547 events in the Windows audit and security logs generally provides invaluable guidance on the source of the problem. lack of connectivity will cause Kerberos authentication and IKE to fail. or recreated in the wrong OU. Group membership must be determined by using the information within the directory service. connectivity problems between computer A and computer B may be caused by blocked network connectivity between computer A and computer C. It is important for support personnel to have documented examples of the expected output of network troubleshooting tools that are obtained from a lab environment where server and domain isolation or some other IPsec deployment is functioning correctly. If IKE or IPsec is failing. which are caused by the inability of the Kerberos protocol to authenticate with a domain controller. Therefore. Group Policy and Group Memberships Domain-based IPsec policy depends upon Group Policy and the download of GPOs. Furthermore. or the small delays required for IKE initial negotiation of IPsec security associations (SA). Kerberos Authentication The server and domain isolation design uses the Kerberos version 5 protocol for IKE authentication. The Kerberos protocol makes it extremely difficult to determine if the Kerberos tickets for a computer contain the proper group memberships. where network access is deliberately denied by IPsec. In many cases.164 Server and Domain Isolation Using IPsec and Group Policy Background information on troubleshooting TCP/IP is provided in the troubleshooting guides listed in the "More Information" section at the end of this chapter. Because there are many different ways that IPsec policies can be deployed to control and help secure traffic. In situations like this. In the Tier 2 section it is preferable to use the IPsec terms initiator and responder to help make the more advanced troubleshooting processes clearer. administrators must plan for the time required for computers to receive new Kerberos TGT and service ticket credentials that contain group membership updates. A support organization should expect to update and customize troubleshooting tools and procedures to remain effective within a server and domain isolation environment. then IPsec connectivity may be affected. Note In the Tier 1 section the terms caller and target were used to help the support staff troubleshoot common problems. This difficulty is "by design. the tools will report failures.). network diagnostic tools do not expect three-second delays for Fall back to clear. If Group Policy assignment is controlled by organizational unit (OU) membership and computer accounts are inadvertently moved to a different OU. Because the Kerberos protocol requires successful network connectivity and available service from DNS and domain controllers. the tools may display one result when run initially but a different result when run a few seconds later. (IKE will also fail if Kerberos itself fails. not from the tickets themselves. then an inappropriate IPsec policy may become assigned." as all the information about group membership is stored in an encrypted form within the ticket. In a server and domain isolation scenario. Group membership is contained within Kerberos version 5 authentication protocol tickets (both TGT and service tickets) that have fairly long lifetimes. . it is unlikely that organizations will be able to rely solely on existing procedures and a generic toolkit. This solution uses domain security groups to control policy assignment and to control network access. even if IPsec is providing successful connectivity. However.

After successful IPsec communication is established. Stateful filtering by a network router or firewall can also block IKE rekey actions or IPsec traffic flow without affecting other diagnostic protocols such as ICMP. because such packets often exceed the path maximum transmission unit (PMTU) for the destination IP address. Troubleshooting is complicated by the fact that IPsec may be required to establish connectivity to a trusted host. or because a device that works above the IPsec layer (such as Windows Firewall or a network router) is blocking access. they will not be able to perform the monitoring role unless some specific exemptions are added to the design. then IKE main mode negotiations will succeed when one computer initiates and fail if the other computer initiates. Similarly. or in some cases to resend the last "good" message until the retry limit expires. Before IPsec service is stopped. TCP and UDP ports may not be accessible on one computer because a service is not running. then communications will stop and TCP connections will eventually time out. If fragmentation is not properly supported. then IKE will fail. This situation could happen if the Kerberos client from the initiating computer could not access a domain controller in the domain of the destination computer. such fragments may be dropped by network devices along a certain path. If these computers or devices are not able to join the "trusted" environment. If computers are members of domains that are not mutually trusted (two-way trust). inbound network logon rights may differ on two computers. stopping the IPsec service should be used only as a last option in the troubleshooting process. Communication Direction Issues One common troubleshooting scenario is successful communication in one direction but failed communication in the reverse direction. then the remote connection will experience a three or four second delay after the service is stopped on the remote computer. stopping the IPsec service on a remote computer will delete the IPsec SAs that are in use by all other currently connected computers. In addition.Chapter 7: Troubleshooting IPsec 165 IPsec-Protected Inbound Traffic Required This server and domain isolation solution specifies that IPsec-protected communication is required for inbound access. If these other computers are not able to Fall back to clear. Because sudden breaks in TCP communications can cause data corruption in applications. which means that an administrator may not be able to connect to a trusted host and then stop the IPsec service without losing connectivity. but also if the IPsec policy designs are not compatible on both sides. Some host-based firewalls intercept traffic below IPsec layer. IKE must be able to send fragmented UDP datagrams that contain the Kerberos tickets. Network Traces and Advanced Network Path Troubleshooting Failures in IKE negotiation often cause the computer that experiences the failure to stop responding to the IKE negotiation. However. IKE authentication typically requires mutual authentication between computers. the network may not pass IPsec protocol packets or fragments of IPsec packets . It is possible for IKE main mode and quick mode negotiation to fail in one direction not only for these reasons. IPsec-protected traffic is likely to be allowed in both directions for a period of time. Host-based firewalls that intercept traffic above the IPsec layer can enforce directionality on connections. the computer should be prepared to be shut down so that all connected users and applications can properly terminate communications. If one computer can not obtain a Kerberos ticket when it initiates IKE main mode for a remote computer. If the administrator's IPsec policy allows Fall back to clear. This requirement will cause remote monitoring tools that run on untrusted computers or dedicated network monitoring devices to report that a remote computer is not contactable.

Detailed tool information is only provided in this section if it is not readily found through the referenced Troubleshooting tools page or where it is useful to have summaries across operating system versions. The IP Security Policy Management MMC snap-in is included in Windows Server 2003. This will help in reading and understanding the network trace much better than going through the encrypted traffic capture. the TCP negotiation of the maximum segment size (MSS) during the TCP handshake does not take into account IPsec overhead. there are functionality and interface differences between the Windows XP and Windows Server 2003 versions. To obtain the same information in Windows XP you must use the IPseccmd command-line tool (described later in this section). It can also be used to modify IPsec policy on remote computers. troubleshooting connectivity failures may require network traces of one or both sides of the communication. filter lists. Windows Server 2003 adds support for parsing IPsec ESP-null. it is important to identify utilities that can abstract information to aid the troubleshooting process. the Windows Server 2003 version has the following additional features: • Provides details on the active IPsec policy. filters. OU. it is considered best practice to lower the ESP encryption level (if it’s currently at DES/3DES) to ESP-null. including the policy name. or Windows Server 2003 Help or that is accessible through the IPSec Troubleshooting Tools page. IP Security Monitor MMC Snap-In The IP Security Monitor MMC snap-in shows IPsec statistics and active SAs. Windows XP. and also understand the IKE negotiation. and parsing UDP-ESP encapsulation used for NAT traversal. parsing ESP when encryption is offloaded. Windows 2000 Server. Windows 2000 Network Monitor provides parsing of IPsec AH and IKE. as well as logs from both sides of the communication. and filter actions and to assign and un-assign IPsec policies. It is also used to view information about the following IPsec components: • • • IKE main mode and quick mode Local or domain IPsec policies IPsec filters that apply to the computer Although this snap-in is part of the Windows XP and Windows Server 2003 operating systems. This section does not attempt to duplicate information that is found in Windows 2000. Therefore. When troubleshooting IPsec and taking network traces between hosts. store. path. and Windows 2000 Professional operating systems and it can be used to view and edit IPsec policy details. description. . Windows XP. Also.166 Server and Domain Isolation Using IPsec and Group Policy correctly. and Group Policy object name. Servers should have the Windows Network Monitor software installed. However. IP Security Policy Management MMC Snap-In The IP Security Policy Management MMC snap-in is used to create and manage local IPsec policies or IPsec policies stored in Active Directory. The Troubleshooting Toolkit Before starting troubleshooting. there is an increased requirement for ICMP PMTU discovery in the network to ensure successful IPsec-protected TCP communication. date last modified. Technical support engineers should understand how to read network traces. Consequently. IPsec integration with TCP enables TCP to reduce the packet size to accommodate the overhead of IPsec headers.

DiffieHellman 2048 group information. This update makes it possible to view Windows Server 2003 computers from Windows XP.Chapter 7: Troubleshooting IPsec 167 • Statistics are provided separately for main mode or quick mode.”Windows XP Service Pack 2 Support Tools”. use the following syntax: ipseccmd show gpo Note This command only works with the SP2 version. The updated Ipseccmd utility has the following capabilities: • • • • Dynamically turns IKE logging on and off Displays information about a currently assigned policy Enables you to create a persistent IPsec policy Can display the currently assigned and active IPsec policy For more information on the updated Ipseccmd utility. certificate mappings. you can use Netsh either locally or remotely. use the following syntax: ipseccmd show all To display currently assigned and active IPsec policies (local or Active Directory). and the updated version does not work on pre-SP2 computers. and Windows XP Service Pack 2 provides additional functionality for this tool. This tool and how it can be used is described in Microsoft Knowledge Base article 257225. However. which must be installed from the Support Tools folder on the Windows XP SP2 CD. refer to Microsoft Knowledge Base article 838079. For more information see the referenced Knowledge Base article. An update to this snap-in is available for Windows XP as part of the update that is described in Microsoft Knowledge Base article 818043. The updated IP Security Monitor MMC snap-in can also read advanced features created in Windows Server 2003 (for example. use the following syntax (no IPsec service restart is required): ipseccmd set logike .exe) with its own graphical user interface (GUI). but cannot edit them. To enable debug logging in Windows XP SP2. the Windows Server 2003 version is enhanced to provide IPsec diagnostic and management functionality. Ipseccmd must be installed from the Support Tools folder on the Windows XP CD. "L2TP/IPSec NAT-T update for Windows XP and Windows 2000". "IPsec troubleshooting in Microsoft Windows 2000 Server". The Netsh commands for IPsec are only available for Windows Server 2003. It is only available for Windows XP. In addition. Netsh is available for Windows 2000. To display all IPsec policy settings and statistics for diagnostics. Netsh Netsh is a command-line scripting utility that allows you to display or modify the network configuration. The pre-SP2 version does not work on updated computers. Windows XP. Note In Windows 2000. they replace Ipseccmd in Windows XP and Netdiag as used in Windows 2000. IP Security Monitor is a stand-alone executable program (IPsecmon. Ipseccmd Ipseccmd is a command-line alternative to the IP Security Policy MMC snap-in. in folders under each mode rather than in one display. and dynamic filters). and Windows Server 2003. An updated version is available with Windows XP SP2.

Table 7. and main mode statistics Windows 2000? Yes Yes Windows XP? Yes Yes* Windows Server 2003? No** No** Yes Yes* No** * Provides network diagnostics. and basic network testing is also obtainable from Netsh. ** Provides network diagnostics. Note Netdiag is not updated when Windows XP SP2 is installed. use the following syntax (again. Netdiag must be installed from the Support Tools folder of whichever Windows operating system CD is used. and quick mode statistics Display the active IPsec policy. Netdiag is available in Windows 2000. Netdiag no longer includes IPsec functionality. but displays IPsec policy name only. Functionality differences are described in the following table. Additional IPsec information is available by using Ipseccmd. but its functionality changes with the operating system version. Windows XP. The relevance of Netdiag to IPsec troubleshooting depends on the operating system version.1 Netdiag IPsec Functionality in Different Operating Systems Command netdiag /test:ipsec netdiag /test:ipsec /debug netdiag /test:ipsec /v Description View the assigned IPsec policy Display the active IPsec policy. including IPsec information. it is important to make sure you are using the latest version by checking the Microsoft Download Center. Netdiag Netdiag is a command-line diagnostic tool that is used to test network connectivity and configuration. filters. the above commands do not work on pre-SP2 computers. instead. . use the following syntax: netsh ipsec dynamic show all. but does not display any IPsec information. In Windows Server 2003. you can use the netsh ipsec context. and Windows Server 2003. no IPsec service restart required): ipseccmd set dontlogike Note You can only use Ipseccmd to enable Oakley logging in Windows XP SP2. For all operating system versions.168 Server and Domain Isolation Using IPsec and Group Policy To turn off debug logging. Instead. filters.

Windows XP and Policy was Windows Server 2003 Help last applied Gpresult Windows 2000.Chapter 7: Troubleshooting IPsec 169 Other Useful Tools for Supporting IPsec In addition to the IPsec-specific tools noted earlier. Windows XP Windows 2000. Windows XP View IPsec Windows Server 2003 Help policy for a computer or for members of a Group Policy container Services. Windows Server 2003. and protocols information Windows Server 2003 Resource Kit Tools Help Srvinfo Windows 2000. Windows Server 2003. Windows Server 2003.com/kb/31 0099 .2 Miscellaneous Useful Tools for IPsec Troubleshooting Tool Supported How to operating obtain systems Windows 2000 Resource Kit Role More information Ipsecpol. Windows XP Windows 2000– Resource Kit. Table 7. it is part of the operating system Part of the operating system Resultant Set of Policy (RSoP) MMC snap-in Windows Server 2003.microsoft. device drivers.exe Windows 2000 only Configures Windows 2000 Resource Kit IPsec Tools Help policies in the directory or in a registry Check when Windows 2000 Resource Kit Group Tools Help. for Windows XP and Windows Server 2003. Windows XP Windows 2000 and Windows Server 2003 Resource Kits PortQry Windows Network Server port status 2003 reporting Resource Kit http://support. the following table lists other tools that may be useful in troubleshooting and should be included in your Tier 2 troubleshooting toolkit.

Calculations on packet loss for Ping may show packets lost during the time required for IKE to successfully negotiate an IPsec SA pair with the target. A known bug in Windows 2000 causes the Ping utility to not wait the proper amount of time before retrying the next echo request. Calculations on packet loss for each intermediate hop will not be available when ICMP traffic is encapsulated by IPsec. the message "Negotiating IP security" is displayed by the utility.170 Server and Domain Isolation Using IPsec and Group Policy Tool Supported How to operating obtain systems Windows 2000. Windows XP Windows 2000. Windows XP Windows 2000. which means that the command may complete immediately instead of waiting three seconds until the soft SA is established. Windows Server 2003. Windows Server 2003. . Pathping. Windows XP Support Tools Role More information NLTest Test trust Windows Server 2003 Support relationships Tools Help and Netlogon secure channels Windows Server 2003 Resource Kit Tools Help KList Windows Kerberos 2000 and ticket Windows reporting Server 2003 Resource Kits Part of the operating system Network connectivity and path testing Pathping Windows Help LDP Support Tools LDAP client for Active Directory testing Windows Server 2003 Support Tools Help Using ICMP-Based Tools with IPsec Windows XP and Windows Server 2003 Ping. The Ping utility in Windows XP and Windows Server 2003 waits the expected number of seconds before the next echo request is sent. When this happens. and therefore requested IKE to negotiate security. they would not be able to detect any intermediate hops (routers) between the client and the target destination. Windows XP Windows 2000. but may not function correctly until soft SAs are established (if Fall back to clear is allowed). Windows Server 2003. If IPsec SAs were negotiated successfully to encapsulate the ICMP traffic used by these utilities. These ICMP utilities are designed to detect whether the IPsec driver matched an IPsec filter to the outbound ICMP echo request packet. Windows Server 2003. and Tracert are aware of IPsec.

It is expected that administrative owners of servers will be able to perform basic network connectivity diagnostics and may skip Tier 1 support. then Tier 2 support will be able to quickly find the relevant troubleshooting procedure in the following sections. Output from the show all command in netsh.txt Output of the Detect_IPsec_Policy. If your organization is using the scripts that are provided as part of the troubleshooting process.txt <CompName>_gpresult. Copy of the lsass. Note Some tools that use ICMP may not be able to detect that IPsec is negotiating security and may produce inconsistent or erroneous results.Chapter 7: Troubleshooting IPsec 171 The "Negotiating IP security" message will not display under the following conditions: • • • If the IPsec driver drops the outbound ICMP packet because of a blocking filter.log <CompName>_netdiag. Descriptions of the files that the script generates are provided in the following table.txt <CompName>_ipsec_547_events. In this model. This file captures the output of the ipseccmd command. Output of the gpresult command. Tier 1 support primarily handles client-related access problems. .txt <CompName>_klist_purge_mt. Table 7. If the IPsec driver allows the ICMP packet to pass unsecured because of a permit filter or a soft SA. Shows the current policy version on the box and if it matches the Active Directory policy.txt <CompName>_netsh_show_all.3 Files Created from the IPsec_Debug. If the IPsec driver does not detect the outbound packet (for example.txt <CompName>_kerberos_events.vbs Script File name <CompName>_FileVer. each organization should adjust the model for their support environment. However. Output from KList while purging machine tickets. Tier 2 support should focus on identifying where the failure to communicate is happening.txt Description Lists the file versions of various IPsec-related DLLs.log file if present. Only on Server platforms. Output from running netdiag. Output of any Kerberos events in the System event log.txt Only on Windows XP. then investigate related possibilities in the architecture of the system. if it was dropped by layers above the IPsec driver). <CompName>_ipseccmd_show_all. Output of any IPSEC 547 errors in the Security event log. <CompName>_ipsec_policy_version.vbs script. The IPsec Troubleshooting Process If Tier 1 support has clearly identified the problem.txt <CompName>_lsass. you will have access to a number of text log files that can be used to help diagnose the problem.

log file.log file. but not being able to connect to a file share on that server. starting with network connectivity. Verify correct application of Group Policy and IPsec policy on both client and server. the server IKE likely failed the negotiation. this section addresses each architectural component in order. This is the only server the client cannot access.log <CompName>_OSInfo. <CompName>_userenv. which contains the IP address of the server. it will cause connectivity interruptions with those remote computers that currently have IPsec security associations established. Copy of the userenv. in a domain isolation environment. and client-server path connectivity for IPsec-related protocols.log <CompName>_<SrvName>_netview. Tier 2 support would then review the MOM event database for 547 events that are collected from the specified server. will indicate that the client has an IPsec policy and that IKE is being initiated.log file. Output of current operating system information. A quick review of the Security Log for event 547 (IKE negotiation failure). it will also disable the protection that IPsec provides. <CompName>_<SrvName>_ping. IKE sends delete notifications for all IPsec SAs and for the IKE SA to all actively connected computers.log Output of the ping command against <SrvName>. Procedures are defined that will help you perform the following tasks: • • • • Verify IP network configuration. Warning: Starting and Stopping the IPsec Service The Windows Server 2003 TCP/IP Troubleshooting document and other references describe how to determine if IPsec is causing a connectivity problem by stopping the IPsec service.txt <CompName>_oakley. Although this will stop IPsec filtering on the computer. Because there are many potential points of failure. When the IPsec service is stopped. if required. Copy of the winlogon. If IPsec is disabled on one computer.txt Output of the net view command against <SrvName>. and disable packet protection. Remote computers with IPsec policy that allowed Fall back to clear will re-establish connectivity after a three second delay. Remote . Identify the cause of a problem for Tier 3 escalation. Consider the following example scenario: a client reports being able to ping a server.txt <CompName>_RegDefault. Can be used to manually reset the registry to previous values if the script fails for some reason. network connectivity and service with domain controllers.txt Description Only on Server platforms. If the client event 547 indicates that the client IKE negotiation timed out.txt <CompName>_winlogon. Investigate issues with IKE negotiation and IPsec-protected communication. if present. Also. expose the computer to untrusted network access. if present. if present. Output of the original registry key values prior to changing. which will contain the current client IP address.172 Server and Domain Isolation Using IPsec and Group Policy File name <CompName>_netsh_show_gpo. Copy of the Oakley. TCP and UDP traffic that is not protected by IPsec will be dropped by other isolation domain members. Output from the show gpo command in netsh.

Therefore. the IPsec driver is loaded at computer startup time with the TCP/IP driver. to remove the IPsec driver failsafe behavior. Network IP Address Configuration Problems If dynamic IP configuration is not successful. and that name resolution services are working. you can query the current state from the command line by using the following syntax: netsh ipsec dynamic show config For Windows Server 2003. the computers that are members of the exemption list) In Windows 2000. The IPsec service must be disabled and the computer restarted to avoid loading the IPsec driver. then the first step is to determine if basic network connectivity exists. Windows Server 2003 resorts to failsafe behavior if the IPsec service cannot successfully start or cannot apply the assigned policy. IKE logging to the Oakley. or if communications are blocked after restarting the computer (or even during normal operation). Consequently. see "Understanding IPSec Protection During Computer Startup".Chapter 7: Troubleshooting IPsec 173 computers with IPsec policy that does not allow Fall back to clear will be unable to communicate.log file requires a restart of the IPsec service. In Windows Server 2003. stopping the IPsec service will delete all filters from the IPsec driver and set the driver mode to PERMIT. Failsafe only applies when an IPsec policy is assigned to the computer and when the IPsec service is not disabled.log file. The IPsec service should be disabled only as a last resort to rule out IPsec-related problems for the following situations: • • Broadcast and multicast traffic environments Connections to remote computers that do not require IPsec for inbound access (for example. In Windows 2000 and Windows XP SP1. Therefore.log file in Windows XP SP2 and Windows Server 2003. After you determine what traffic is allowed and blocked by bootmode and persistent configurations. stopping the IPsec service will unbind the IPsec driver from TCP/IP and unload the IPsec driver from memory. Note For details about Windows Server 2003 failsafe behavior. it is important to use the techniques discussed in the following sections to troubleshoot isolation scenarios without stopping the IPsec service. Stopping the service is not required to enable and disable IKE logging to the Oakley. a communications failure may be easy to explain. Verifying Network Connectivity If Tier 1 support identifies possible network connectivity issues. IPsec may be the cause. connectivity to or from a computer could fail during normal operation because the IPsec driver is not enforcing the domain-based IPsec policy. such problems may be related to IPsec failsafe behavior (for example. Windows Server 2003 IKE logging can be enabled dynamically using the Netsh commands described in online Help. In Windows XP and Windows Server 2003. This determination involves verifying that the proper IP configuration is being used. The latest update to Ipseccmd for Windows XP SP2 provides the syntax ipseccmd set logike and ipseccmd set dontlogike to dynamically enable and disable IKE logging to the Oakley. the IPsec service must be disabled and the computer restarted. To obtain alternative or additional information. if the computer is started in Safe Mode or Active Directory Recovery Mode). The previously referenced IPsec deployment chapter includes recommended configuration of boot exemptions to exempt . that there is a valid network path between the initiator and the responder computer. It does not unload the IPsec driver from memory.

For example. Fall back to clear used for DNS servers. in the Woodgrove Bank scenario. broadcast traffic and traffic to the DHCP servers is exempt to ensure that dynamic IP configuration works properly. Name Resolution Problems The IPsec policy design used in the server and domain isolation scenarios should not interfere with typical procedures that are used to determine if name resolution is working. Answer the following questions to confirm that name resolution is working properly while the IPsec service is running: • • • • Can the client ping the DNS server IP address listed in its IP configuration? Can nslookup find a DNS server? Can the client ping the fully qualified DNS name of the target? Can the client ping the shortened DNS or NetBIOS name of the target? Potential sources of name resolution problems include an active and misconfigured HOSTS file. WINS proxy failures. refer to the Active Directory Operations Overview: Troubleshooting Active Directory-Related DNS Problems page. For example. use Ipconfig to confirm there are no problems obtaining an address. open a command window and enter the following: ipconfig /release ipconfig /renew If the address configuration problems only happen during computer startup for Windows XP SP2 and Windows Server 2003. one filter may negotiate security to the DNS server and one may exempt the domain controllers. incorrect WINS record. Some high-security environments may require DNS and WINS servers to be protected with IPsec.x. incorrect DNS record registrations. If a computer cannot obtain proper DHCP configuration (for example.174 Server and Domain Isolation Using IPsec and Group Policy inbound Remote Desktop Protocol connections. and network timeouts reaching the WINS server. For DHCP clients. For procedures to help troubleshoot Active Directory-integrated DNS. and DHCP auto-update issues. WINS server unavailability. Possible reasons for NetBIOS name resolution failures include an active and misconfigured LMHOSTS file. which can result in name resolution problems. Active Directory replication issues. For more information.x) or has problems renewing the lease. zone file update problems. However the exemption list must be maintained manually and may become outdated. which will ensure that remote access to the server is available when other traffic is blocked. a misconfigured DNS server entry in IP properties. the IPsec policy design exempts all traffic to DNS and WINS servers. WINS replication problems. With the IPsec service running. if it uses an Auto-configuration IP address of 169. then the IPsec policy should be examined for proper exemptions. it is possible that DNS and WINS servers could be configured to not respond to Ping requests. the configuration for exemptions (default exemptions and boot exemptions) should be inspected. In a server and domain isolation solution. if DNS is integrated within Active Directory and there are duplicate filters for the same IP address in the IPsec policy. . a misconfigured WINS server entry in IP properties.254. see the "Troubleshooting IPsec Policy" section later in this chapter. However.

In a server and domain isolation scenario there should be a quick mode-specific filter with a negotiation policy (filter action) of permit for each of these addresses. To troubleshoot access to network services in Active Directory • • Check that the client can ping each domain controller IP address. tests for network connectivity and successful operation of authentication services should be performed before IPsec-specific troubleshooting steps (described in the next section) are performed. if any of these connections trigger IKE negotiations and the authentication fails because IKE is unable to locate a domain controller for Kerberos authentication. the IPsec service should be stopped briefly (if possible) while name resolution tests are repeated. In a scenario such as Woodgrove Bank. • • The Windows Support tool klist. the list of domain controller IP addresses in the exemption list must be maintained manually. When connected to the internal network. as discussed later in this section. investigation should continue to determine which IPsec policy is being applied. IKE authentication. Verifying Connectivity and Authentication with Domain Controllers Because IPsec policy delivery. so network connectivity tests to the domain controllers should not be affected by IPsec. If IKE negotiation is seen to a domain controller IP address. the Event 547 failure error may be logged in the security log. “HOW TO: Use Portqry to Troubleshoot Active Directory Connectivity Issues”. refer to the network connectivity steps above. Use nslookup <domain name> to return the full list of IP addresses. Netdiag uses multiple network connections and protocols to perform testing. IPsec policy design exempts all traffic to all domain controllers. and RPC ports.txt to perform many DNS-related and domain controller-related connectivity tests. If name resolution tests only fail when the IPsec service is running. These steps are explained in Microsoft Knowledge Base article 816103. The UDP protocol messages used by portqry do not usually require upper layer protocol authentication. then the IPsec policy may be incorrectly assigned or not updated.exe tool or the PortQueryUI tool to test access to the domain controller UDP. You can use the following command line options to view the filter lists for this task: Ipseccmd show filters Netsh ipsec static show all If the name resolution problems still persist. use netdiag /v >outputfile. If not.0 or later of the portqry.exe can be used to verify successful Kerberos login and authentication. To view Kerberos service tickets for the logged in domain user • Open a command prompt and type the following: klist tickets . LDAP. you can get the filter list from the initiator and check for duplicate filters. Identify which IP addresses are used for the domain member's domain controllers.Chapter 7: Troubleshooting IPsec 175 If name resolution problems persist. so they can verify service availability even if authentication is not available. Klist must be run in the local system context to view the Kerberos tickets for the computer. and most upper layer protocols depend on access to domain controllers. Use version 2. However.

this information is encrypted so that the groups cannot be viewed. Additional detailed procedures for troubleshooting Kerberos are published in the following white papers: • • Troubleshooting Kerberos Errors. type ldp. If this procedure is part of troubleshooting IKE negotiation failure.exe. Verifying Permissions and Integrity of IPsec Policy in Active Directory It may be necessary to verify information about the IPsec policy container in Active Directory. Select Connection. wait one minute for IKE negotiation to time out and then try to access the target server again with the application. chose a time one minute from the current system time (such as 4:38 pm) and type the following: at 4:38pm /interactive cmd 3. 4. Type klist tickets to confirm that no tickets exist. Troubleshooting Kerberos Delegation. Specify the name of the target domain.exe. Verify the Task Scheduler service is running and the logged on user is a member of the Local Administrators group 2. The following procedure uses the support tool ldp. To ensure that computers have the latest group membership information in their Kerberos tickets. 2. Select Connection. When IKE negotiation is attempted again. which will have the latest group information available. and then Bind. new Kerberos tickets will be obtained automatically. Although Kerberos tickets contain group information for the user or the computer. Click Start. chose a time one minute ahead of the current system time (such as 4:38 pm) and type the following: at 4:38pm /interactive cmd /k klist tickets Note that the command window title bar contains C:\Windows\System32\svchost. Run. and then Connect. use klist to purge the current Kerberos tickets. Type klist purge and press Y for each ticket type to delete all Kerberos tickets. Be careful not to execute the application from the command window that is running in Local System context. . At a command-line prompt. To purge the Kerberos tickets for the computer (Steps 1-4 must be run in Local System context) 1.176 Server and Domain Isolation Using IPsec and Group Policy To view domain computer tickets 1. Verify the Task Scheduler service is running and the logged on user is a member of the Local Administrators group. Specify logon credentials for the target domain. To verify information about the IPsec policy container 1. 3. New IKE main mode negotiations will request new Kerberos TGT and service tickets for the destination computer. At a command line prompt. 5. Therefore group membership must be confirmed manually by inspecting the group membership in Active Directory. 2.exe and press ENTER.

these objects should only be managed using the IPsec Policy Management MMC snap-in and the command-line tools that are available for each platform. when corruption prevents the IPsec Policy Management MMC snap-in or command-line tools from being used. and Microsoft may make changes in these formats at any time. The following table shows the relationship between the Active Directory object names and the IPsec policy component names that are configured in the IPsec Policy Management MMC snap-in: Table 7. For more information. Note The design details of these objects are considered an internal private data structure and are not published by Microsoft.Chapter 7: Troubleshooting IPsec 177 4. two versions of the same filter list). Windows XP. Differences exist within the format of these objects in different Windows releases. Select View. To control Read and Modify access for IPsec policy. or specify the LDAP DN for the IPsec policy container as a base location.4 IPsec Policy Component to Active Directory Object Name Mapping IPsec policy component name IPsec Policy IKE Key Exchange Security Methods IPsec Rule IPsec Filter List IPsec Filter Action Active Directory object name ipsecPolicy{GUID} ipsecISAKMPPolicy{GUID} ipsecNFA{GUID} ipsecFilter{GUID} ipsecNegotiationPolicy{GUID} Ldp. Therefore. Either specify no base DN and navigate to the IPsec policy container. For advanced troubleshooting of policy retrieval and corruption problems. However. Caution: It is strongly recommended that all objects in the IP Security container have the same permissions. Note Some domain users may not have Read access to the container because of the way permissions are configured. You should only delete objects by using LDP as a final option. 5. Microsoft does not recommend setting permissions on individual IPsec policy objects. all IPsec policy objects in the container will display. or identical names for objects cause improper IPsec policy design (for example. ldp. see Microsoft Knowledge Base article 329194. Windows 2000. "IPSec Policy Permissions in Windows 2000 and Windows Server 2003". If you have Read permissions on the container. It can be launched from a command window in the context of the local system to troubleshoot Read permission issues for the IPsec service. See the following "IPsec Service" troubleshooting section for more information about how to correct IPsec policy corruption. Corruption of the IPsec policy is the most common reason for situations in which an IPsec object contains a DN reference to an object that no longer exists. permissions should be managed on the IP Security container itself as explained by Knowledge Base article 329194.exe can be used to manually inspect the contents of the IP Security container and the relationship of among IPsec policy objects. "IPSec Policy Permissions in Windows 2000 and Windows Server 2003". which can help troubleshoot object version and replication issues. and then Tree. Click the plus sign (+) next to the container node in the tree view. and Windows Server 2003 use the same basic directory schema for IPsec policy that is shown in the IPsec Policy Structure diagram in the Windows Server 2003 How IPsec Works technical reference.exe provides the ability to identify the last time IPsec policy objects were modified. corruption may also occur if control characters become part of the name of an object. . individual objects are unable to be read due to permission problems.

then the IP configuration on the client may be a problem. Can the client ping an IP address in the exemption list. then an improper IPsec policy may have been assigned. including the need to use ICMP for network path testing by utilities such as Ping. but not when using IKE or IPsec protocols. Pathping and Tracert. Use other TCP/IP troubleshooting procedures to investigate. local or network filters may be blocking traffic. Use other TCP/IP troubleshooting procedures to investigate. then IPsec is not causing the problem or IPsec does not have a filter for that exempted IP address. In particular. destination port 500 and fragments IKE/IPsec NAT-T. the IPsec overhead for IKE main mode authentication packets that contain the Kerberos ticket is often larger than the PMTU for the destination IP address. UDP source port 4500. Can the client ping the IP address of the target destination? If yes.178 Server and Domain Isolation Using IPsec and Group Policy Network Path Connectivity Microsoft recommends that the ICMP protocol be exempted in server and domain isolation solutions. which requires fragmentation. Use other TCP/IP and core network troubleshooting procedures to investigate. and filters on the target host must be open to the following protocols and ports and support fragmentation: • • • • IKE. There are several reasons for this recommendation. AH. Therefore. network firewalls.1? If not. IKE is required to be able to rekey in either direction. such as a DC? If not. Can the client ping the DNS servers shown in its IP configuration? If not. • • • • Path connectivity tests may succeed for ICMP. If one side does not receive a delete message. filtering in routers. By definition.0. it may believe that an IPsec SA pair still exists when the peer no longer recognizes it and . If no. IP protocol 51 and fragments Stateful Filtering in the Path Is Not Recommended Stateful filtering may cause connectivity problems for IKE. then basic network connectivity exists between the client and the target without IPsec.0. or any of the possible issues mentioned previously may exist. a third-party firewall may be installed. destination port 4500 IPsec ESP. work properly and not display the "Negotiating IP security" message. Can the client ping the default gateway shown in its IP configuration? If not. IP protocol 50 and fragments IPsec AH. and ESP because the state is typically based on activity timeouts. See the following IPsec policy section later in this chapter. then try tracert to the target and other destination IP addresses to determine how far the network path is valid. the local interface may not be connected or may have limited connectivity. The latter can be confirmed by inspecting the filter configuration. then the DNS servers may not allow themselves to receive ICMP echo request messages. host-based firewalls. or the network path to the default gateway may be interrupted. or the IP configuration is invalid. Devices cannot inspect IKE traffic to determine when IPsec SAs are deleted because these messages are encrypted by IKE. Use other TCP/IP configuration troubleshooting procedures to investigate. UDP source port 500. If this message displays. which means delete messages may be sent in either direction. then there could be a problem with the TCP/IP configuration. the Ping utility is missing. These utilities should. therefore. To determine whether the problem is related to basic network configuration or path connectivity • Can the client ping its own IP address or the local loopback address 127. the IPsec policy may not be exempting the proper DNS server IP addresses.

Note There is a known issue that requires TCP PMTU detection to be enabled for IPsec to secure traffic in a NAT traversal scenario where IPsec UDP-ESP connections are initiated from a host outside of the NAT to a host behind a NAT. Windows Server 2003 does support this discovery processing. IPsec. IKE uses UDP packets and allows them to be fragmented as necessary. because the IPsec driver processes packets at a lower layer than the layer at which the firewall filtering is performed. IPsec is not integrated with UDP. Windows 2000 and Windows XP do not support ICMP PMTU discovery processing for IPsec transport mode packets that use the NAT traversal encapsulation (UDP port 4500). Windows Firewall does not filter IPsec packets. when IPsec AH or ESP is applied. confirm that TCP PMTU detection is enabled either by ensuring that the following registry key is not defined or set to 1 on both sides: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\ Parameters\EnablePMTUDiscovery=1 (This key may display on more than one line. See the "Troubleshooting Translational Bridging" page. However. the small offsets for rekey when the time-based lifetime expires. If this scenario is required. Therefore. because hardware acceleration typically does not process fragmented packets. This setting is preserved when either AH or ESP IPsec transport mode is used to secure the packet. for file sharing using SMB protocol over TCP port 445). the IKE ports should be configured open in the host firewall to receive incoming IKE negotiations for upper layer protocol connections that are allowed through the firewall (for example. Fragmented IPsec packets must be processed by the IPsec driver in software. The direction that IKE will rekey is based on the direction of traffic flow that expires the byte-based lifetime more quickly. and UDP applications do not have a method to detect if IPsec is protecting their traffic. it is a single line in the registry. UDP . The IPsec NAT traversal implementation added support for IKE fragmentation avoidance only when IKE authenticates with certificates (for example. AH. Both the client and the target computer must be able to receive ICMP PMTU messages for IPsec packets that are too big.Chapter 7: Troubleshooting IPsec 179 discards those packets that use it. IKE authentication that uses Kerberos does not support fragmentation avoidance and must be able to send and receive fragmented UDP packets that contain the Kerberos ticket. It is especially important for IPsec-protected traffic to avoid fragmentation. IPsec is integrated with TCP so that when TCP packets have the DF (Don’t Fragment) flag set (the default setting). Support Required for Fragmentation Network paths and filters must support passing fragments for the IKE. and ESP protocols. The network path must support passing fragments for AH and ESP because IPsec secures the entire original IP packet before outbound fragmentation at the IP layer. and the direction that packets flow after idle IPsec SAs are deleted. TCP will reduce its packet size to accommodate the additional bytes that are added by IPsec encapsulation. Consequently. Host-based stateful filtering of IKE traffic on clients that initiate connections (and thus IKE negotiations) through Windows Firewall usually does not cause a problem. Support for ICMP PMTU Required by TCP The default setting in Windows 2000 and later releases is for each TCP packet to have the Don't Fragment bit set in the IP header. This behavior is called TCP Path MTU Discovery. in L2TP/IPsec VPN scenarios).) The Microsoft Windows Server 2003 Member Server Baseline Security Policy template and other third-party configurations may configure this registry key in order to disable TCP PMTU. a packet that is too big will be dropped at the router and the router should return an ICMP Destination Unreachable message that specifies the maximum size allowed.

Windows 2000 and later releases automatically disable these TCP offload functions in the TCP/IP stack when the IPsec driver has filters. Similarly. Some NIC drivers also support the ability to disable offload by using the Advanced . Note The registry key can be set to control the default exemptions as necessary for all platforms. NIC drivers that accelerate non-IPsec functions may have problems with IPsec-protected traffic. 811832. However. For more information. “How to Enable IPSec Traffic Through a Firewall”. This configuration prevents known problems with Real Time Communications (RTC) clients and Windows Media Server. An extensive set of tests is used by WHQL to certify NIC drivers that are designed to support IPsec offload. NICs that accelerate TCP functions may be ones that support TCP checksum calculation and validation (checksum offload). Windows XP and Windows Server 2003 TCP/IP stack supports a registry key option to disable all forms of TCP/IP offload. see Microsoft Knowledge Base article 233256. the outbound filter Subnet -> Any will match outbound broadcast and multicast traffic sent from hosts using an internal subnet IP address. Diagnostics in Network Devices May Not Be Useful One of the impacts of using IPsec encapsulation is that applications which assume TCP/IP traffic is in plaintext can no longer inspect traffic within the network. Network card drivers that are not certified and signed by the Windows Hardware Quality Lab (WHQL) may cause problems when IPsec is used to protect traffic. the Windows 2000. if IPsec policy filters do not exempt ICMP. Windows XP SP2 uses the same default exemptions as Windows Server 2003. IPSec default exemptions are removed in Windows Server 2003. IPsec filtering does not support configuring destination addresses for specific broadcast or multicast addresses. Network Interface Card and Driver Issues IPsec packet loss can sometimes be caused by network interface cards (NIC) that perform special functions. even if IPsec is performing only permit and block functions. even if AH or ESP encryption is not used.180 Server and Domain Isolation Using IPsec and Group Policy packets that use the full MTU size will become fragmented by the host when transmitted. IPSec Default Exemptions Can Be Used to Bypass IPsec Protection in Some Scenarios. To assist troubleshooting. For details about the use and security implications of the NoDefaultExempt registry key. Support Required for Broadcast or Multicast Traffic The IPsec policy design for server and domain isolation uses filters from Any <-> Subnet. Therefore. If multicast or broadcast traffic is required. see the following Knowledge Base articles: • • 810207. both of which use multicast traffic. which allows multicast and broadcast traffic to bypass IPsec filtering in Windows XP and Windows Server 2003. use of the Ping utility may produce ICMP packets that appear as fragmented IPsec AH or ESP packets on the wire. Network diagnostic tools that inspect or provide reports based on TCP and UDP ports are unlikely to be able to interpret the IPsec-encapsulated packets. Updates to such tools may be required from vendors to inspect IPsec AH or ESP-null packets. as well as the ability to efficiently send large TCP data buffers (large send offload). it must discard such traffic if it matches the filter. Inbound multicast and most types of broadcast will not match the corresponding Any -> Subnet inbound filter. because IPsec cannot secure multicast or broadcast traffic. Cards that perform clustering or "teaming" should be tested for IPsec compatibility. then you can set the registry key to NoDefaultExempt=1.

exe graphical display. See the "Toolkit" section in this chapter for details on how to enable IPsec driver logging. or under certain conditions (such as when the device is congested. The IPsec service should be stopped only as a last resort to identify whether a problem is IPsec-related or not. or by an application that did not realize it was protected. and third-party packet processors occasionally corrupt packets of a certain size. which may affect application connectivity.168. It is important to start by trying to exclude sources of corruption in the local system. As previously mentioned. In Windows Server 2003. then this error may not warrant investigation. If a device modifies a part of the packet that is protected by an integrity hash. it is also important to correlate these observations with a network monitor trace so that the source of the corruption can be found. The computer may need to be restarted for driverlevel configuration changes to take effect. those with a certain number of fragments. This error may also represent an attack on the packet by a malicious application. this counter can be checked by using the IPsec counter in Performance Monitor. a network monitor trace is usually needed from both sides of the communication to identify and correlate which packets are being sent and which received. The event text will be similar to the following: Failed to authenticate the hash for 5 packet(s) received from 192. if it persists please stop and restart the IPSec Policy Agent service on this machine. Enable IPsec driver logging and look for event 4285 in the System Log from source IPsec. certain network devices may not allow IP protocol types 50 or 51 or UDP port 500 or 4500 to pass. or conditions in which the error occurs. then the receiving IKE or IPsec driver will discard this packet and cause the Hash Authentication Failure error. This could be a temporary glitch. In such cases. the source of most packet loss problems is not the IPsec system. Disable IPsec offload. This section reviews common cases in which packets are discarded by IPsec. Similarly. • Examine the IPsec Packets Not Authenticated counter. or by using the netdiag /test:ipsec /v command. Troubleshooting Packet Loss in IPsec Protocols Packets can be discarded or lost. try to disable • . or by looking at Statistics in the IPsec Security Monitor MMC snap-in. The error may also be an indicator of a denial of service attack.10. Restarting the service will not fix the problem and may cause more problems.0. Experience has shown that some devices. adapters. Although the event text suggests that a restart of the IPsec service may fix the problem. times of day. by using the netsh ipsec dynamic show stats command. To detect IPsec packet discards of corrupted packets. IPsec-encapsulated packets may cause some packets to fragment and not pass through the network. In Windows XP. monitors traffic. Windows 2000 shows this counter in the ipsecmon. or reboots). If the number of packets is small. Event Error 4285 Event title: Hash Authentication Failure IKE and IPsec provide protection against modification of packets while they transit the network. which is logged in the System Log as event 4285. It may be necessary to capture a trace of the typical protocol behavior without IPsec and then compare it with the protocol behavior of IPsec-protected traffic. network drivers.Chapter 7: Troubleshooting IPsec 181 properties of the network connection. Look for retransmissions indicated by the same size of packet appearing repeatedly. Resolution of this error requires investigation to identify a pattern of source IP addresses. the following techniques can be used. this counter can be checked by using the ipseccmd show stats command or by looking at Statistics in the IPsec Security Monitor MMC snap-in. However. those of certain protocol types.

" This error indicates that the receiving computer received IPsec-formatted packets when it did not have an IPsec SA with which to process them. use the following registry key for computers running Windows 2000. or Windows Server 2003: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC\ EnableOffload DWORD registry value to 0 (This key may display on more than one line. The following conditions are likely to generate more of these events: • • • • Transferring high volumes of IPsec traffic over 1 Gigabit or higher connections. The impact on upper layer protocol communications is usually negligible. although the packets are being discarded. This could be a temporary glitch. Windows XP does not log this event by default. Event Error 4268 Event title: Received Packets with Bad Security Parameters Index (SPI) The Windows 2000 and Windows XP (including SP1) implementation of IKE has a known issue that results in packet loss under particular conditions. which causes IKE to constantly rekey with many clients simultaneously.182 Server and Domain Isolation Using IPsec and Group Policy advanced or performance features of the driver using the configuration provided by Advanced Properties. it is called a "bad SPI. When there are heavily loaded (slow) servers and fast clients. When there are many active clients for a server. When there are slow clients communicating to a fast server. preferably those certified and signed by the Windows Hardware Quality Lab. Note To disable TCP/IP offload functionality. if it persists please stop and restart the IPSec Policy Agent service on this machine. Look for other evidence of packet corruption in TCP/IP packet discard statistics and on other computers that use the same configuration. and use the latest NIC drivers that are available from the vendor. the IPsec service should be stopped only as a last resort to identify whether a problem is IPsec-related or not. . it must discard the packets. driver logging must be enabled. This issue is fixed in Windows Server 2003 and Windows XP SP2. it is a single line in the registry. the source of this type of packet loss is part of the design of the IPsec system." Although the event text suggests that a restart of the IPsec service may fix the problem. The IP counter for Datagrams Received Discarded will increase each time IPsec discards a packet. Therefore. Restarting the service will not fix the problem and may cause more problems. System Log event 4268 messages (if enabled). An IPsec security parameter index (SPI) is a label in the packet that tells the responder which security association should be used to process the packet.) Then restart the computer. By default. If an SPI is not recognized. Windows 2000 logs these messages to the System Log as event 4268. Windows XP. This issue can be identified by the following issues: • • Slow but consistent increase in Bad SPI counter values. These are benign error messages. Event text is similar to the following: "Received <number> packet(s) with a bad Security Parameters Index from <ip address>. because protocols already expect some packet loss for a variety of other reasons. As noted earlier. The number of bad SPI events that are generated depends on how busy the computers are at the time and how fast IPsec-protected data is being transmitted at the time of rekey. Then investigate the characteristics of the network paths through which the packet would be transmitted.

the computer must be restarted to reload TCP/IP and IPsec drivers. bad SPI events may be seen periodically (although not necessarily at specific intervals). Reporting of these events can be enabled by using the IPsecDiagnostics configuration through the netsh ipsec command-line option. However. and the slower responder receives IPsec-protected traffic that it doesn't yet recognize. File copy. When the connection is reset. However. The error can also mean that an attacker is flooding a computer with spoofed or injected IPsec packets. The IPsec driver counts these events and logs them to keep a record of bad SPI activity. and therefore may contribute to the delay in response to other negotiations. when IKE establishes a new IPsec security association pair the initiator must use the new outbound IPsec SA to transmit data. Terminal Server. To avoid potential security weaknesses introduced by using the same key to encrypt too much data. As specified in the Internet Engineering Task Force (IETF) IPsec Requests for Comment (RFC). the socket is closed and applications are notified of a connection break. the initiator is able to send new IPsec SA secured packets sooner than the responder is ready to receive them. • • • .Chapter 7: Troubleshooting IPsec 183 The impact is that an IPsec-protected TCP connection will slow down for a few seconds to retransmit the lost data. or through the registry key directly. These cards substantially reduce the amount of CPU utilization that IPsec uses for high throughput data transfer. either setting will substantially increase the load on the computer that services many clients. Add CPU or other hardware to increase performance or reduce application loads. In Windows 2000. Telnet. Use main mode or quick mode perfect forward secrecy (PFS). Install an IPsec hardware acceleration NIC if one is not already installed. In third-party client interoperability scenarios. Increase the quick mode lifetimes (if security requirements will allow it) to 21 or 24 hrs (idle IPsec SAs are deleted in 5 minutes if they are not used). TCP may go into congestion avoidance mode for that connection. these events cannot be eliminated by any current registry key settings or patches. In Windows 2000. In Windows XP and Windows Server 2003. set it to the minimum value (every 60 seconds) so the events are registered quickly. and other TCP-based applications should not notice these few lost packets. Because IKE rekey is dependent on both the elapsed time and the amount of data sent under the protection of an IPsec SA. there have seen some cases in which TCP loses a burst of packets on a fast link and must reset the connection. If several packets are lost. The following techniques can help minimize these errors: • Adjust the IPsec policy settings. which will not cause this problem for the particular IPsec SA that is being negotiated. a bad SPI error may indicate that an IPsec peer did not accept and process a delete message or had problems completing the last step of IKE quick mode negotiation. The LogInterval registry key can be used to investigate and minimize these events. The most common cause of this error is a known issue with Windows 2000 that involves how IKE synchronizes IPsec SA keying. you can stop and restart the IPsec Policy Agent service to reload the IPsec driver. do not set a lifetime greater than 100 MB when using ESP encryption. When troubleshooting. The default setting in Windows XP and Windows Server 2003 is to not report these events. which may interrupt file transfers. In a few seconds the connection should resume full speed. When an IKE quick mode initiator is faster than the responder.

Because one peer has already deleted the filter while an active upper-level protocol communication is taking place. • If these options do not work. These products are typically a PCI card with Diffie-Hellman exponentiation offload capability that accelerate the Diffie-Hellman calculations. The most likely cause of the error is a policy configuration problem that causes one side to send traffic in the clear because of a more specific outbound permit filter. because it expects all traffic to and from the server to be secured inside the IPsec SA pair. the IKE delete message may not arrive and be processed by the other peer before the plaintext packets arrive. which causes the error." If possible. Event Error 4284 Event title: Packets in the clear that should be secured This event indicates that an IPsec security association was established at a time when packets were received in plaintext that should have been inside the IPsec security association. create a filter to permit certain high-speed traffic that does not need IPsec protection (for example. For example. server backup traffic over a dedicated LAN). if a client has a filter to secure all traffic with a server and the server policy has a more specific filter to permit plaintext HTTP replies. IPsec does not have a counter value that records packets dropped for this reason. Although the IP counter for Datagrams Received Discarded will be incremented. IPsec SAs may be negotiated for traffic that will be exempt after policy loading has completed. the amount of time it takes to process the delete message depends on the current load on the peer computer. This acceleration also benefits public and private key operations for certificates that use the Secure Sockets Layer (SSL) protocol. the server will secure all traffic to the client except outbound HTTP packets. .184 Server and Domain Isolation Using IPsec and Group Policy • If CPU utilization remains high. Verify with the vendor that their card specifically supports the "ModExpoOffload interface in CAPI for DiffieHellman calculations. The error message may also happen while a large policy is being loaded. This event can also occur during regular operations and during third-party client interoperability cases in which one peer deletes an IPsec security association or a filter in the IPsec driver while traffic is flowing between the computers. or may experience a policy update that deletes IPsec SAs and filters. and upgrading to Windows XP SP2 or Windows Server 2003 is not possible. If this situation occurs. because IPsec security associations may become established before the full filter set is applied to the IPsec driver. The client receives these packets and discards them for security reasons. one side may unassign IPsec policy. if it persists please stop and restart the IPsec Policy Agent service on this machine. It is unlikely that restarting the IPsec service will correct the error. This could be a temporary glitch. the event suggestion should not be followed. Also. This issue can only be identified from System Log error event 4284. For example." As with previous errors. which reads as follows: "Received <number> packet(s) in the clear from <IP address> which should have been secured. then contact Microsoft Product Support Services to see if there are other options currently available. These packets are discarded to prevent packet injection attacks on IPsecsecured connections. investigate use of a hardware accelerator product to speed up Diffie-Hellman calculations.

and IPsec policy interpretation. Common issues with Group Policy for IPsec policy management include the following: • • • • • • Replication delays of various configuration components in Active Directory Problems with the Group Policy polling and download process Confusion over which IPsec policy version is assigned IPsec service is not running IPsec policy in Active Directory cannot be retrieved. the IPsec policy design requires all traffic (except ICMP) to be secured by IPsec. and security group membership information. The retrieval of assigned GPOs by the domain member is what delivers the IPsec policy assignment to a host computer. attribute changes in GPO IPsec policy assignments and IPsec policy. References to information about these topics can be found in the "More Information" section at the end of this chapter. in Windows XP and Windows Server 2003 this service is called the IPsec Service. any problems with GPO retrieval will cause computers to not apply the proper IPsec policy.Chapter 7: Troubleshooting IPsec 185 The error message can also be an indication of an injection attack where plaintext traffic is being sent that matches (either deliberately or by chance) the traffic selectors for a particular active inbound security association. so a cached copy is used instead Delays because of IPsec policy polling for retrieval of currently assigned IPsec policy Replication may be delayed because of the number of IPsec-related objects in Active Directory. IPsec NAT-T Timeouts When Connecting Over Wireless Networks A recent problem was found that causes connections to time out when Windows Server 2003 or Windows XP-based client computers attempt to connect to a server on a wireless network that uses IPsec NAT-T. Support engineers must be familiar with the use of Group Policy by IPsec. see Microsoft Knowledge Base article 885267. Verifying the Correct IPsec Policy This section describes steps for detecting problems with IPsec policy assignment and interpretation. IPsec policy precedence. Troubleshooting Group Policy for IPsec Group Policy provides the mechanism for assigning a domain-based IPsec policy to a domain member. Therefore. This problem should be escalated to the IPsec policy designer. For more information. Appropriate filters must also be in place to guide IKE as a responder. as well as to trigger IKE to negotiate IPsec SAs with remote IP addresses to secure traffic. GPOs. . In this solution. The policy also contains filters for each IP address in the exemption list. “Connections time out when client computers that are running Windows Server 2003 or Windows XP try to connect to a server on a wireless network that uses IPsec NAT-T”. Careful planning must be done to assess the impact of an IPsec configuration change as it gradually takes effect on domain members. Filters from a properly interpreted IPsec policy must be in the IPsec driver for IPsec to permit and block packets. the IPsec service is called the IPsec Policy Agent. such as IPsec policies. Note In Windows 2000.

DC=Woodgrove. look under the following location in the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\ WinLogon\GPExtensions IPsec deployment-related GUIDs for Group Policy CSEs are as follows: • • • Security. If there are problems processing security policy.CN=Microsoft.186 Server and Domain Isolation Using IPsec and Group Policy For Group Policy troubleshooting procedures. the GPOs that contain IPsec policy assignments are stored in the registry under the GUID for the IPsec client side extension at the following location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\ Group Policy\History\{e437bc1c-aa7d-11d2-a382-00c04f991e27} The IPsec CSE is activated by the security policy CSE whenever an IPsec policy assignment exists in a GPO. see the following white papers: • • • • Troubleshooting Group Policy in Windows 2000 Troubleshooting Group Policy in Microsoft Windows Server The IPsec Policy Management MMC snap-in (an extension of the Security Policy MMC snap-in) for assignment of an IPsec policy in the GPO The Group Policy client side extension (CSE) for IPsec (implemented in gptext.DC=domain.dll) that processes the IPsec-related information in the GPO Domain-based IPsec policy assignment is implemented by two components: Domain-based IPsec policy assignment is implemented by two components: • • The IPsec Policy Management MMC snap-in assigns policy to a GPO by storing the selected IPsec policy information in the IPsec component of the GPO. Each time the IPsec CSE is called. To locate the GUID for each Group Policy extension.DC=com The LDAP DN of the assigned IPsec policy is stored in the GPO attribute ipsecOwnersReference.CN={GPOGUID}. GPO precedence rules apply to the order in which the IPsec CSE receives the GPOs to process. When Group Policy retrieves the list of GPOs that apply to the computer.CN=Windows. {42B5FAAE-6536-11D2-AE5A-0000F87571E3} The IPsec CSE copies the LDAP DN and related information about the assigned IPsec policy in the following registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\GPTIPSECPolicy If multiple GPOs contain IPsec policy assignments. the IPsec-related information in the GPO (including the DN) is .CN=System. {E437BC1C-AA7D-11D2-A382-00C04F991E27} Scripts. which is referenced as the LDAP Distinguished Name (DN): CN=IPSEC. {827D319E-6EAC-11D2-A4EA-00C04F79F83A} IP Security.dll) that processes the IPsec-related information in the GPO The IPsec Policy Management MMC snap-in (an extension of the Security Policy MMC snap-in) for assignment of an IPsec policy in the GPO The Group Policy client side extension (CSE) for IPsec (implemented in gptext. CN=Policies.CN=Machine. then there may also be problems processing IPsec policy. then the IPsec CSE is called for each GPO. The order may also be affected by settings in the GPOs themselves and by Read ACLs that prevent some assigned GPOs from being retrieved.

When all GPOs have been processed. The service maintains the cached IPsec policy as the latest domain policy. the Policy Assigned column will display an error. Note An invalid DNS entry can mean that Group Policies are not downloaded from Active Directory. . not the one that is in current use. It is not otherwise possible to easily detect version changes if the policy name remains the same. For more information on DNS problems. The load balancing logic of the domain controller locator may cause a GPO to be retrieved from one domain controller while the LDAP query for the assigned IPsec policy is retrieved from a different domain controller in the same site. use the following registry key: Troubleshooting the IPsec Service The IPsec service does not need to be running to use the IPsec Policy Management MMC snap-in. A known issue exists that allows the name of the assigned IPsec policy to become out of sync with the name of the IPsec policy that is actually in use (and which is cached). for more information see the "Policy Versioning" section in Chapter 5. The IPsec service continues to poll for changes in the assigned IPsec policy based on last update time of any of the assigned IPsec policy directory objects. refer to the earlier "Name Resolution Problems" section. "Creating Isolation Policies for Isolation Groups. If the proper IPsec policy is not assigned. There is no log specifically for the IPsec CSE. Group Policy logging must be enabled to investigate this issue in more detail. The log file will be created at %windir%\Security\Logs\Winlogon. There are many different types of Group Policy logs and logging levels. and the name of the IPsec policy only changes when the IPsec CSE is called. However. IPsec tools may report an IPsec policy name that was last processed by the CSE. Network monitor traces may be needed to capture traffic at the time of the Group Policy refresh to confirm which domain controller IP address is being used for the retrieval of each object. Problems may include: • • Replication problems or delays that lead to objects not being found. even though computer and user logon are successful. Note Microsoft recommends that IPsec policy naming conventions include a version number in the name. as well as any errors reported by the IPsec CSE. even after a forced refresh of Group Policy. To create a detailed log file for the Security CSE. Application Log errors from sources Userenv or SceCli will indicate Group Policy processing problems. Logs for the Security CSE are necessary to see any errors processing security policy. the IPsec CSE is not called unless there is a change in the IPsec policy assignment DN attribute in the GPO." in this guide. so that the currently applied version of the policy can be easily found. However.log. The IPsec service does not update the information in the GPTIPsecPolicy registry key. the CSE signals the IPsec service that a domain-based IPsec policy is assigned.Chapter 7: Troubleshooting IPsec 187 overwritten to this registry key. This registry value is used by the IPsec Monitor MMC snap-in and the command-line tools to report the name of the currently assigned IPsec policy. There are several workarounds for this bug. such as DSIPSECPolicyName. The IPsec service then reads the GPTIPsecPolicy\DSIPSECPolicyPath value to retrieve the proper IPsec policy. if an administrator then assigns a local policy. Therefore. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\ CurrentVersion\Winlogon\GpExtensions\{827d319e-6eac-11d2-a4ea00c04f79f83a}\ Set the ExtensionDebugLevel entry to a REG_DWORD value of 0x2.

the security model for Windows XP SP2 and Windows Server 2003 SP1 was fundamentally changed to limit remote RPC connections and to activate Windows Firewall by default. . the IPsec driver is loaded by default at the end of the startup process by the Policy Agent service. and Active Directory. In Windows XP and Windows Server 2003. the remote RPC interfaces for the IPsec service were disabled as a safety measure. Some Winsock Layered Service Providers (LSP) may be installed that are interfering with IPsec. Therefore the IPsec Monitor and IPsec Policy Management MMC snap-ins are not able to perform remote monitoring on these computers. These policies should permit remote access to the computer by other means in case they are the only policies applied because of to other failure conditions. the IPsec Policy Management MMC snap-in is not able to connect to remote computers that do not have the same operating system version installed. Because of these changes. IKE cannot obtain exclusive control of UDP port 500 and port 4500 . which execute the IPsec MMC snap-ins as local processes. which maintains the run-time state of IPsec policy for IPsec monitor queries and IKE queries. this design was improved so that the IPsec driver is loaded by the TCP/IP driver during the startup process. The IPsec driver is not part of IP packet processing until the first time the Policy Agent informs the IPsec driver of an active policy. the RPC interfaces were again significantly updated. remote computer. IPsec Policy corruption. The command portqry –local –v provides even greater detail. In these cases. This architectural change means that the text of logged IPsec events in Windows 2000 has changed in Windows XP and Windows Server 2003. This architectural change also means that significant changes had to be made in the RPC interfaces that are used for remote management. Remote management for IPsec should be performed using Remote Desktop (Terminal Server) connections.Part 2: Network Protection Technologies. Consequently. For more information. which causes the IPsec service to report a number of errors. the IPsec driver will provide stateful outbound communication by default if there is an IPsec policy assigned. the IPsec driver is not included in inbound and outbound IP traffic processing. refer to the "Troubleshooting Application Related Issues" section later in this chapter. In addition. • • The Windows 2000 IPsec implementation uses a module called the IPsec Policy Store (polstore. attention should be paid to the design of persistent policy or local policy as a "safe" policy to be applied in case of errors that occur when domain-based policy is applied. This design is changed and improved in Windows XP and Windows Server 2003 with the addition of new IPsec policy types (startup policy and persistent policy) and the Security Policy Database (SPD) component. The assigned IPsec policy cannot be read entirely or applied entirely.188 Server and Domain Isolation Using IPsec and Group Policy The following common problems can cause the IPsec service to fail during startup: • The computer was started in Safe Mode or Active Directory Recovery Mode .dll) so that the IPsec Policy Agent and the IPsec Policy Management MMC snap-in can use one module to access all three supported policy storage locations: local. If there is no active IPsec policy. For more information about LSPs and IPsec. In Windows 2000. see Changes to Functionality in Microsoft Windows XP Service Pack 2 . In Windows XP and Windows Server 2003. Both persistent policy and computer startup policy (bootmode exemptions) should be part of the troubleshooting investigation. Inbound connectivity will be blocked unless there is a bootexemption configured. but may cause communications to fail in many ways. For Windows Server 2003. such as by blocking Group Policy and the IPsec service from retrieving corrected policies. Use netstat –bov to show the processes and code modules for each port. The driver does not process packets until it has filters loaded by the IPsec service. These errors do not cause the service itself to fail.

There may be third-party software inhibiting the connection. Therefore. errors may be logged by the IPsec Policy Agent for problems with service startup. Failed to determine SSPI principal name for ISAKMP/Oakley service . Kerberos authentication will not function. • • • • • . Policy Agent failed to load IPSEC policies. Windows 2000 is designed to do this when the IPsec service starts. This failure may indicate that the computer is not able to successfully log in to the domain. On the internal network. may hang. This error may have been caused by insufficient kernel memory or improper initialization of the IPsec driver. or that cannot reach the domain controllers using the Kerberos protocol during IPsec service initialization. These errors include the following: • IP Security Policy Agent could not be started. investigate problems with the service control manager. Otherwise. This error is probably caused by problems the IPsec Policy Agent encountered when registering itself with the RPC subsystem. and may shut down. the IPsec Policy Agent will not function correctly. No IP Security policy will be enforced. or the operating system may be missing code modules that are required for this functionality. This Windows 2000 error message commonly occurs when the IPsec service starts (perhaps at computer startup time) on a remote network where an IPsec policy is assigned (perhaps from registry cache of domain policy) that requires Kerberos authentication and a domain controller is not available. this event would be logged on a computer that is not a member of the domain.Chapter 7: Troubleshooting IPsec 189 In Windows 2000. Failed to connect to SCM Database Error: <number>. contact Microsoft Product Support Services. An error occurred while the IPsec Policy Agent was loading all the filters into the IPsec driver. It may also be caused by IKE failing to initialize because of third-party Winsock LSPs. • Policy Agent failed to start. The IPsec driver could not be successfully loaded and interfaced with the TCP/IP stack. Policy Agent failed to connect to the IPSEC Driver. Policy Agent failed to start ISAKMP service. or the IPsec service not running in the local system context. It may also be caused by third-party security software preventing the network port allocation. Policy Agent RPC Server failed to… • • • • • • register protocol sequence register interface register interface bindings register interface endpoint register authentication mechanisms listen • Any of these errors can be caused by changes to advanced security settings or problems within the RPC service that cause the IPsec Policy Agent service to not properly initialize during service startup. This error usually occurs because IKE cannot gain exclusive control over UDP port 500 or port 4500 because another service is already using them. Therefore. It must run as local system. Failed to obtain Kerberos server credentials for ISAKMP/Oakley service . If the problem persists. Windows 2000 logs this message when the security support provider interface (SSPI) function call QueryCredentialsAttributes fails. The IPsec service cannot open the service control manager database. which may occur because the IPsec service was configured to run as a nonprivileged service account.

then permit and block filter actions still function as expected. which prevent IKE from using certain socket options. the IPsec service running as Local System must be able to obtain a Kerberos service ticket for the LDAP service on the Active Directory server. IPSec Services failed to initialize RPC server with error code: < number>. In Windows XP and Windows Server 2003. the SPD. File corruption or permissions may be the cause. If the driver is in normal mode. This error is caused by a problem with the IPsec driver loading. it will fail to load and the IPsec driver will also fail to load. IPSec Services put IPSec driver in block mode due to previous failures error code <number>. . IPSec Services has experienced a critical failure and has shut down with error code: <number>. A separate event would indicate if the IPsec driver was put into block mode. IPSec Services could not be started. Please contact your machine administrator to re-start the service. Problems with computer account logon to a domain controller. IPSec Services could not be started. If FIPS. Restart the computer. The IPsec service encountered the error indicated by the <number> in the event text and is no longer running. then the following issues may cause the IPsec service to be unable to retrieve the policy from Active Directory: • • • Problems with communication to domain controllers. Common sources of this problem are third-party Winsock LSPs. binding to the TCP/IP stack. Look for security settings or third-party security software that may inhibit driver loading. The IPsec driver was unable to load for some reason. There are options for LDAP signing and sealing using the Kerberos session key. This error will also be reported when IKE cannot gain exclusive control of UDP ports 500 and 4500. or initializing before attempting to add policy to it. IPSec Services failed to initialize IKE module with error code: < number>. the following IPsec service error events indicate that the service cannot start: • IPSec Services failed to initialize IPSec driver with error code: < number>. and the Policy Agent. Problems with issuing Kerberos tickets. The IPsec service depends upon the RPC subsystem for interprocess communication between IKE. • • • • Troubleshooting the Retrieval of IPsec Policy The IPsec service uses an authenticated and encrypted TCP LDAP query to download the assigned IPsec policy for all platforms. After restarting the computer. IPSec Services could not be started. If problems persist. Stopped IPSec Services can be a potential security hazard to the machine.190 Server and Domain Isolation Using IPsec and Group Policy • A Secure communications policy cannot be enforced because the IP Security driver failed to start. Therefore. Filters with a negotiate action simply drop traffic because IKE is not available. Use RPC troubleshooting techniques to confirm that RPC is working properly. Contact your system administrator immediately. then contact Microsoft Product Support Services. if problems persist. contact Microsoft Product Support Services. contact Microsoft Product Support Services. This message is a notification that the IPsec driver was put into block mode as a failsafe behavior because of errors encountered processing IPsec policy. If problems persist. If the proper IPsec policy assigned is confirmed to be stored by the IPsec CSE under the GPTIPsecPolicy registry key and the service is running. FIPS.sys internal signatures cannot be verified during initialization. This behavior is available only in Windows Server 2003.sys signature failure requires a replacement of the original signed binary file or a new binary file from Microsoft. The IPsec driver is still loaded and may either be in normal mode (enforcing IPsec policy filters) or in block mode. Block mode still allows inbound exemptions that were configured by using the netsh ipsec command.

Test computers should apply local IPsec policy and examine the details in the IPsec Monitor MMC snap-in to confirm expected filter ordering. . any IPsec policy objects or changes that are not yet saved are lost. If the IPsec Policy Management MMC snap-in is run over a slow link. registry tools can be used to examine and change the registry key value. Startup IPsec Policy The Netsh utility allows the configuration of the bootmode and bootexemptions options that are supported by Windows Server 2003 only. If a proper policy does not appear to be retrieved or installed from scripts. Tier 2 support should be able to use either command-line or GUI tools to confirm that the correct IPsec policy is being retrieved and that the policy is being correctly interpreted. when started in Safe Mode). This functionality could cause an IPsec policy to become corrupted. To do so. create policy in the local store and view it with the IPsec Policy Management MMC snap-in to verify its integrity. Policy corruption because of problems when saving objects to the store or because of accidental or intentional deletion of objects in the store. see below) The default OperationMode value requires the IPsec driver to perform stateful filtering of outbound traffic. use a Remote Desktop session to execute the snap-in as a local process. but other types of IPsec policy may have been configured in ways that cause problems. use the Netsh ipsec dynamic show config command to display the startup configuration. If the MMC snapin window is closed. If the IPsec service is not running (for example. it may take some time to save all changes in a large policy. Steps to delete each type of policy are shown here with the expectation that a policy refresh will cleanly install a correct policy. if a Windows 2000 system retrieves an IPsec policy with additional features it will simply ignore them. Problems finding the particular IPsec policy or component objects requested in the LDAP query. Note An IPsec policy that was created in Windows XP or Windows Server 2003 and uses new features that were made available in those releases may have those features silently removed if the policy is later edited and saved by the Windows 2000 IPsec Policy Management MMC snap-in. Troubleshooting procedures for each type of policy are reviewed in the rest of this section. which may or may not change the behavior of the IPsec policy when enforced on the Windows 2000 system. However. Generally. This solution uses only domain-based IPsec policy. then the problem is escalated to Tier 3 support. any command-line tool script that creates IPsec policy should be tested. Problems with read permissions for any of the requested IPsec policy objects. If the IPsec service is running. If the MMC snap-in is run over a slow link. A known issue with the IPsec Policy Management MMC snap-in occurs when managing IPsec policy in Active Directory or on a remote computer. Generally. Procedures to troubleshoot and fix IPsec policy read errors and corruption depend on the storage location. The configuration is stored in the following registry keys with default values shown: • • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPSec\ OperationMode=3 (Stateful) HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPSec\ BootExemptList = (exemption for inbound DHCP.Chapter 7: Troubleshooting IPsec 191 • • • • Problems with availability of the LDAP service.

Use ipseccmd show all to show all active policy. If the IPsec service is not running. The persistent policy registry key exists by default and is empty: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\Policy\Persistent In Windows XP.192 Server and Domain Isolation Using IPsec and Group Policy To troubleshoot traffic problems that are potentially caused by IPsec stateful filtering during startup. filter lists. In Windows Server 2003. To delete all of the objects (rules. the best way to detect persistent policy is to see that this registry key is not empty. naming conventions cannot be used to indicate filters and filter actions that originated from persistent policy. local or domain policies that were created using scripts may also have these names.exe. Therefore. If the IPsec service is running.netsh file: Netsh –f show_persistent. and the new policy conflicts with other settings that are already assigned. because it may be used in some environments troubleshooting instructions are provided in this chapter. The IPsec service discards names for the persistent policy rules when interpreting the policy into the active settings. Persistent IPsec Policy Persistent policy is supported by Windows XP and Windows Server 2003.netsh . specify the persistent policy name in the following command: ipseccmd. Persistent policy is applied first and merges with local or domain IPsec policy. However. use a registry tool to change OperationsMode=1 for permit. The ipseccmd show command reports active policy that is contained in the IPsec service and does not report that a particular setting originated from persistent policy.exe or ipseccmd. The following Netsh command script will display the configured persistent policy by using the commands in the show_persistent. Because ipseccmd does not provide names for filters and filter actions. Alternately. if you delete the Persistent key itself. However. when the IPsec service is started. One common error situation occurs when an existing persistent policy is not deleted before a new persistent policy is defined. set the IPsec driver to permit traffic at startup instead of performing stateful filtering. and filter actions) that are associated with a particular persistent policy. The persistent policy is stored at the same registry location referenced earlier. future ipseccmd commands will fail when attempting to create persistent policy. both local and domain policy must be unapplied before only the persistent settings can be viewed. However. To resolve corruption in persistent policy and policy conflicts. the persistent policy has full management capabilities that are similar to local and domain IPsec policy. which will include entries from persistent policy.exe -w PERS -p "policy name" –o The easiest way to ensure that all persistent policy is removed is to delete all subkeys under the Persistent key. The solution described in this guide does not use persistent policy. the IPsec driver does not apply the startup security mode if the IPsec service is configured for manual start or if it is disabled. including any persistent settings. restart the computer for the IPsec driver to load in permit mode. Filters have "text2pol{GUID}" style names whenever they are created by ipsecpol. After the service is configured for manual start or is disabled. delete all objects in the persistent policy store and execute the ipseccmd script again to create it. then use netsh ipsec dynamic set config bootmode value=permit to set the startup mode to permit.

To easily remove all local policy. Use the IPsec Policy Management MMC snap-in. It is also possible to use the sc policyagent control 130 command to reload policy. use the netdiag /test:ipsec /debug command to view assignment and active policy details. use the netsh ipsec static show gpoassignedpolicy command. the netsh ipsec command can be used to create the policy on a Windows Server 2003-based computer and then exported to a file that can be imported on Windows 2000 and Windows XPbased computers or imported into a domain after the policy has been tested. To force a reload of the assigned IPsec policy. registry tools. or the IPsec Monitor Active Policy node to display the currently assigned local policy. Alternatively. or the ipseccmd.exe -w REG -p "<policy_name>" –o command to delete the named IPsec policy. The IPsec Policy Management MMC snap-in or registry tools should be used to delete IPsec policy objects in the local store. and Windows Server 2003 and is stored under the following registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\Policy\Local If a local policy is assigned. stop and restart the IPsec service or unassign and reassign the IPsec policy using the IPsec Policy Management MMC snapin. To force the IPsec service to reload the policy.Chapter 7: Troubleshooting IPsec 193 The show_persistent. use the ipseccmd show gpo or the netdiag /test:ipsec command to view the assignment and active policy details. In Windows XP.netsh file is a text file that contains the following lines: Pushd ipsec static Set store persistent show all exit The following Netsh command script can be used to delete all persistent policy: pushd ipsec static set store persistent delete all exit Local IPsec Policy Local IPsec policy is supported by Windows 2000. stop and restart the service. delete all subkeys to the previously referenced storage location. For significantly easier troubleshooting. In Windows 2000. then the assigned local policy will be added to any configured persistent policy to become the active policy. When policy is reloaded. In Windows Server 2003. the assigned policy will be stored in the key as follows: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\Policy\Local\ActivePolicy If domain policy is not assigned. the IPsec Policy Management MMC snap-in. all IPsec and IKE SAs are deleted. which may cause connectivity delays or interruptions with computers that were actively using IPsec SAs to transmit and receive traffic. Windows XP. . IPsec policies that are created by ipsecpol or ipseccmd scripts should be edited by the IPsec Policy Management MMC snap-in to define filter names for each filter before they are used in production environments.

and delete policy. In Windows 2000. The netsh ipsec command can be used to unassign. • • . Alternatively. Similarly. The IPsec policy may have become corrupt. An object was defined in the IPsec policy (such as a rule or filter list) that currently does not exist. A registry tool should be used to determine whether the cache exists and contains the proper contents. The assigned domain IPsec policy is stored in the local registry at the following location: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\GPTIPSECPolicy The local registry cache of the domain policy is stored at: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\ IPSec\Policy\Cache The directory store should be managed by the IPsec Policy Management MMC snap-in or the netsh ipsec command running as local processes against Active Directory. Failed to locate Domain Controller. It may also be caused by a failure of the Kerberos protocol to issue the LDAP service ticket to the IPsec Policy Agent service. the Active Directory storage policy may have become corrupt. use the same commands that were listed for Windows XP. registry tools are used to delete the domain policy by deleting the GPTIPsecPolicy and Cache registry keys. the computer should be prevented from reading the appropriate GPO in Active Directory. The IPsec policy storage module could not find an LDAP directory from which to retrieve domain-based IPsec policy. The IPsec Security Monitor MMC snap-in and command-line tools will still show the domain policy as assigned because these tools read the GPTIPsecPolicy information that runs in the context of the local administrator user. Examine the IPsec policy by using the IPsec Policy Management MMC snap-in to see if all of the policy rules appear intact and if the Key Exchange properties (on the General tab) can be viewed properly. and Windows Server 2003. IPsec Monitor can be used to inspect different parts of the active policy. The IPsec Policy Agent service could not perform an authenticated LDAP bind to the Active Directory IP Security Policies container. Active Directory IPsec Policy This policy is supported by Windows 2000. To temporarily block domain policy from applying to the local computer. To refresh the domain-based IPsec policy assignment and reload the IPsec policy and update the cache contents. Then either the IPsec service must be restarted or the service control command used to force an IPsec policy reload without the domain policy. or the object was deleted (although existing IPsec policies still reference the object).194 Server and Domain Isolation Using IPsec and Group Policy Use the netsh ipsec dynamic show all command to see the current active policy configuration. the following errors may be seen when there are problems with the policy: • Failed to bind to Directory schema. This error is probably caused by the computer account failing to login successfully and receive Kerberos credentials. No available tool exists that will allow you to directly view the contents of the cache. reassign. Make sure the computer is a member of the domain and check network connectivity. For longer term blocking of domain-based IPsec policy. Windows XP. you can configure the permissions on both the GPTIPsecPolicy and Cache registry keys to deny read access to the local system account. To delete and reload the local policy on Windows Server 2003. Failed to bind to IPSEC Policy Storage object. use gpudpate /force.

either by using an LDAP call or a remote registry call. This event identifies that IPsec policy is locally defined on the computer and that domain-based policy could not be applied to override it. IPsec policy corruption may cause an invalid GUID in the query. Please contact your System Administrator. The policy may have become corrupt. • • In Windows XP and Windows Server 2003. Network connectivity may not be available to the destination IP address. the following events indicate that the IPsec service was unable to retrieve a particular policy type. This error is usually serious. the policy is ignored and the next assigned policy in the order of persistence is read. The LDAP or remote registry service may not be available. or local computer store cannot be opened. but that a failure occurred when reading at least one of the objects associated with the assigned policy. as (i) there's no Active Directory Storage or Active Local Registry policy or (ii) the Active Directory Storage policy couldn't be applied successfully and there's no Cached policy or no Active Local Registry policy. If a local policy cannot be successfully read in Windows Server 2003 and Windows XP SP2. Try deleting all persistent policy and then recreating it. The IPsec service detected that a local policy was assigned and attempted to read the policy. Policy Storage is not Open. but failed to read the contents of at least one of the persistent policy objects. The IPsec storage module was not successful in using LDAP signing and sealing to download the assigned IPsec policy. This event identifies the default state. Check permissions on all registry keys. or the IPsec policy storage was restored to an earlier time when the object did not exist. filter action. The IPsec Policy Storage module could not find the GUID for the object (rule. Using the Active Local Registry policy. The Active Directory. This error may be caused by any of the following: • • • Replication delays in which the referencing object arrives before the referenced object or if the queries become targeted at two different domain controllers. or ISAKMP settings) that is contained in the IPsec policy or one of the rules. This error is usually caused by a permission or authentication failure. Failed to locate Object in storage. The IPsec service detected that persistent policy was assigned and stored in the local registry. remote computer. Not using any IPsec policy. as (i) there's no Active Directory Storage policy or (ii) the Active Directory Storage policy couldn't be applied successfully and there's no Cached policy. The object may have been deleted while it was still being used in the IPsec policy. • PAStore Engine failed to load persistent storage IPSec policy on the machine for "<policyname>" with error code <number>.Chapter 7: Troubleshooting IPsec 195 • Failed to communicate with Directory Service on the Domain Controller. because the IPsec policy cannot be completely retrieved and therefore will not properly function. The object may have permissions that deny the ability to enumerate it or read it. • • • • • Cannot complete requested operation. PAStore Engine failed to load local storage IPSec policy on the machine for "<policyname>" with error code <number>. The policy may have become corrupt and should be deleted and recreated. Check permissions on all registry keys. in which no policy is assigned to the computer. • . filter list.

The IPsec service encountered at least one error when reading the assigned IPsec policy from Active Directory. or problems within the NIC driver or with other related Windows networking components may be the cause. This error almost always indicates a policy corruption or version problem. There may be no network interfaces or an internal error within the network interface manager. and the IPsec service will have retrieved and stored the latest version in the cache. Filters based on Interfaces will not be expanded and plumbed to the IPsec Driver. Therefore. such as those configured in a rule for a particular connection type (remote access. or it may have been caused by missing IPsec policy objects or objects to which read permission is not granted. • • • . Note that the error in the text of this message comes from the Windows source code directly and will appear as shown here. The data format of the IPsecData attribute in an IPsec policy object is not what it expected. The current network interface IP addresses are used to expand the policy's generic filters into specific filters. If the problem persists after a restart. It must be corrected before all IPsec policy settings are successfully applied as intended. which adjusts the IPsec filter configuration in the IPsec driver as quickly as possible if necessary (for example. Part of the IPsec policy contains a data format that the policy storage engine does not recognize. Any changes made to the Active Directory IPSec policy since the last poll could not be applied . Devices that are not fully PnP compliant. for example) rather than for all connections. IPsec policy features in Windows XP and Windows Server 2003 are designed to be transparent to the Windows 2000 IPsec Policy Agent. for My IP Address filters). Polling will continue to try to reach the directory. the IPsec service now considers the registry cache of the IPsec domain policy to be the primary source of policy. This error may have been caused by a lapse in network connectivity before all policy objects were retrieved. file corruption. Connectivity to Active Directory was originally available when the active policy was applied. because of loss of DNS service) and that no changes in the current active IPsec policy were made. IP Public Help API failed to get the Interface Table. The IPsec Policy Agent did not find any IP network interfaces to filter. In Windows 2000. Interface change events are signaled to the IPsec service. • Troubleshooting the Interpretation of IPsec Policy The interpretation of IPsec policy is performed after the complete policy is successfully retrieved from the appropriate storage location. IPsec filters based on interface type. Failed to read data from blob. This error is an indication of policy corruption or it may indicate a policy version problem at some point in the future. An error occurred when the IPsec Policy Agent tried to enumerate the list of interfaces on the computer. the following messages may indicate a problem with proper interpretation or configuration of the IPsec components to enforce the policy: • Data Type attribute specifies unrecognized data format. PAStore Engine polled for changes to the Active Directory IPSec policy. Policy Agent has not interface list. detected that Active Directory is not reachable and migrated to using the cached copy of the Active Directory IPSec policy.196 Server and Domain Isolation Using IPsec and Group Policy • PAStore Engine failed to load directory storage IPSec policy on the machine for "<policyname>" with error code <number>. contact Microsoft Product Support Services. Then the list of inbound and outbound specific filters is loaded into the IPsec driver for packet processing. This message simply indicates that at a regular polling interval the IPsec service did not successfully contact Active Directory (for example.

Discarding the IP address. Like the previous message about inserting a filter. Filters based on Interfaces will not be expanded and plumbed to the Ipsec Driver . and depend on the order in which the policy change is processed. This error means that any traffic that would match that filter will not be secured. An error occurred when the IPsec Policy Agent tried to enumerate all IP addresses on the computer by using a function call in the IP Helper API. Transient conditions when a new network interface is added or removed can cause this problem. which may in turn appear as temporary unexpected connectivity behavior when using this IP address. Otherwise. The IPsec policy contains no valid security methods in the filter action of a rule. this error indicates a problem in the internal state of the IP interface configuration. Policy contains no valid offers. Zero Phase 1 offers. the list of filters in the IPsec driver is not what the assigned IPsec policy requires. Results after an elapsed period of time may differ. The IPsec policy design should be analyzed to ensure the quick mode specific filters for inbound and outbound directions are not duplicated.Chapter 7: Troubleshooting IPsec 197 • IP Public Help API failed to get the IP Address Table. Could not add an entry to the IPSEC Policy table. which Microsoft Product Support Services would need to investigate further. There may be no IP addresses configured. Therefore no security actions (such as permit or block) and no negotiation of IPsec SAs will be performed for traffic using this IP address. or it does not contain settings for IKE main mode (Key Exchange settings configured from the General tab). This situation might be benign if the duplicate filter has the same action as the one already processed into the IPsec driver. Policy Agent failed to insert or update a filter in IPsec . With no filter action security methods (IKE quick mode Phase 2 offers). Policy Agent attempted to insert an existing filter into IPsec . Because the IP address is discarded by the IPsec Policy Agent (not by the TCP/IP stack). The IPsec policy is probably corrupt. The IPsec policy should be designed to avoid duplicate filters. No filters were added to the main filter list. The IPsec Policy Agent confirms that all IP addresses appear in the network interface list. security is not being provided as intended. However. The IPsec policy may only contain the default response rule. This error indicates a duplicate filter condition in the IPsec policy. The IPsec policy is probably corrupt if no IKE main mode security methods (ISAKMP policy objects) are found. Discarding the ISAKMP policy. • • • • • • • • • . A low kernel memory condition might cause this error. no IPsec filters will be created for that IP address. Discarding the Negotiation policy. or errors were encountered when the rules or filter lists were read. If the error persists. Zero Phase 2 offers. contact Microsoft Product Support Services. The IPsec driver detects that there is a duplicate filter and rejects the duplicate. this is an unsupported IPsec policy design. A new filter could not be added to the IPsec driver by the IPsec Policy Agent. The message indicates that the IPsec service will not create specific filters for this discarded IP address for generic My IP Address filters in policy. Matching filter mirror exists in filter list. or there may be problems similar to the above. IKE will fail in quick mode negotiations for traffic that matches the corresponding filters. Therefore. if the filter actions are different. IP address entry index not found in the Interface Table. The IPsec Policy Agent found no filters in the IPsec policy that was retrieved from storage.

This message replaces the several IP Helper API messages that were used in Windows 2000. It is also benign when it occurs during resumption from standby or hibernate modes and a different network interface configuration exists that is being detected during the resumption. this message should be followed by a successful application of cached policy (if it exists). Any persistent policy that had been already applied is removed. the IPsec Policy Agent attempts to read the latest policy from the Active Directory first. PAStore Engine failed to apply Active Directory storage IPsec policy on the machine for DN "<CN=ipsecPolicy{GUID}" with error code: <number>. This message could indicate that the policy is corrupt or that permissions were incorrect. In most cases. Please run IPSec monitor snap-in to further diagnose the problem. The IPsec service found persistent policy configured in the registry. This message is informational. but could not apply all of it successfully. it may indicate that a GPO is delivering an IPsec policy that does not exist. This error could indicate that the cache is corrupt or that permissions were incorrect. When a domain-based IPsec policy is assigned. This message reports that it was unable to retrieve the IPsec policy from the directory and is applying policy from the last domain policy. such as when a wireless network is no longer in range. • • • • • . and the IPsec driver will be set programmatically to block mode (with boottime exemptions) as indicated by a separate message. This can be a potential security hazard to the machine since some of the network interfaces may not get the protection as desired by the applied IPSec filters. This message indicates that the service encountered at least one error in applying local IPsec policy from the local registry. Local policy (if defined) would be next in the precedence order. as Active Directory Storage policy couldn't be applied successfully (network unreachable. policy integrity invalid.198 Server and Domain Isolation Using IPsec and Group Policy • Using Cached policy. PAStore Engine failed to apply local registry storage IPsec policy on the machine for "<policyname>" with error code <number>. and is often a notification that the domain controller is not reachable for mobile clients. which is cached in the registry. or that the policy is corrupt. However. cannot be read. Now it has encountered at least one error when applying the cached copy of assigned domain policy from the local registry. and that it had already failed to apply the policy that was retrieved from Active Directory. This message indicates that the IPsec service knows that domain policy should be assigned. These issues must be resolved for the IPsec policy to function properly. Please run IPSec monitor snap-in to further diagnose the problem. Windows XP uses the same event text. which will likely affect connectivity with other isolation domain members. etc. This is a benign error if it occurs when interfaces are added and removed or when connection states change. the following events indicate that a problem may have occurred interpreting IPsec policy. PAStore Engine failed to apply some rules of the active IPSec policy "<policyname>" on the machine with error code: <number>. In Windows Server 2003. PAStore Engine failed to apply locally cached copy of Active Directory storage IPSec policy on the machine for "<policyname>" with error code: <number>.) . such as if there are no quick mode security methods (offers). This condition is serious because none of the domain-based policy could be applied. • PAStore Engine failed to apply persistent storage IPsec policy on the machine for "<policyname>" with error code: <number>. IPSec Services failed to get the complete list of network interfaces on the machine. This problem usually occurs in combination with other problems discussed here. The IPsec service found the domain policy that was specified by the DN in the GPTIPsecPolicy registry key but was not able to apply it.

source Any. UDP. The quarantine client must pass a "quarantine key" to the internal LAN IP address of the VPN server to end the quarantine and gain full VPN access to the network. However. IKE main mode will fail because the server will respond to accept only certificate authentication. the RRAS service is a common source of IPsec policy conflict in some organizations. an internal IP address is assigned to the VPN client.Chapter 7: Troubleshooting IPsec 199 Policy Configuration Issues with RRAS VPN As noted in the Tier 1 support section earlier in this chapter. This section explains why the built-in IPsec policy for L2TP/IPsec VPN servers creates a conflict with the domain policy used in this solution. source 1701. the subnet filters in this solution will initiate an IKE negotiation from the internal IP address of the client VPN tunnel virtual interface to the internal LAN interface of the VPN server. This situation is one example of a duplicate filter problem. which is not compatible with the policy used for domain and sever isolation. the VPN client's domain IPsec policy is applied from cache when the laptop starts. However. is more specific than the server and domain isolation IKE main mode inbound filter. the IKE main mode-specific inbound filters are: • • From Any IP Address to <external interface IP address> -> Certificate authentication From Any IP Address to <internal LAN interface IP address> -> Certificate authentication The second filter. A second case of conflict can occur if the Internet VPN client uses the quarantine capabilities of the Connection Manager client. "Description of the IPSec policy created for L2TP/IPsec". the main mode-specific filter that controls IKE negotiation response to this server is the address part of the inbound filter: • From Any IP Address to My IP Address -> Certificate authentication Note that the use of "My IP Address" causes the inbound filter to be expanded for each IP address on the VPN server. The IKE inbound policy lookup will match the more specific main mode filter and reply with the IKE main mode settings for L2TP. destination 1701 (outbound) Note For more information on this policy. for the internal LAN interface. the IKE quick mode . The IPsec policy for L2TP/IPsec servers is automatically generated by the RAS Manager service (RASMAN) and includes the following filters: • • From Any IP Address to My IP Address. UDP. When the VPN quarantine connection is successfully established. Even if the policy did allow certificate authentication. Consequently. The client internal IP address may be covered by one of the subnet filters (Any <-> Internal Subnet) defined in the domain isolation IPsec policy. when an administrator uses a trusted client to manage the VPN server. which is: • From Any IP to Subnet -> Kerberos authentication Consequently. the IPsec policy design used in the Woodgrove Bank scenario is used to illustrate the issues. In the following discussion. the principles will apply to many enterprise-wide IPsec deployments. Certificate authentication will be used instead of the Kerberos authentication that was used for this solution. In this scenario. see Knowledge Base article 248750. Assuming the VPN server has an external interface IP address for Internet connectivity and an internal interface for LAN connectivity. it will initiate IKE to the VPN server's internal IP address. This filter is required so that VPN clients have IPsec-authenticated access end-to-end through the VPN tunnel to internal servers and any other workstations they may access. dest Any (inbound) From My IP Address to Any IP Address.

such as when IKE quick mode is initiated only from a computer that is typically the responder. Disable automatic IPsec policy for L2TP by using the ProhibitIPsec registry key for RASMAN. which will trigger the IKE negotiation from all domain members to the test server. and a reply that consists of an equal number of fragments is sent back. Note For more information about this configuration. then fragmented by the IP layer before being sent on the wire. The following options can be used to resolve the conflict between RRAS server L2TP/IPsec policy and the isolation policy: • • • Include RRAS server internal LAN IP addresses in the exemption list. and only Kerberos authentication for all traffic for domain isolation using the internal IP address. Troubleshooting IKE Negotiation IKE can be difficult to troubleshoot because errors may only occur under certain conditions. Use only PPTP. Similarly. This full IP packet will be encapsulated by the IPsec protocol. All fragments must be received at the destination. destination Any. or UDP source 1701. it is helpful to establish a static IP workstation or server for testing that will have a simple policy assigned. A more advanced test would be to confirm that IPsec encapsulation works with fragmentation to each area of the network by using ping –l 5000 <IP address> from the test server to computers located in each area. Auditing must be enabled to capture IKE audit events. See Knowledge Base article 314496. Experience has shown that devices that are congested or that serve as boundaries between different speed networks (for example. By analyzing and summarizing the IP addresses in the IKE main mode success audits. Fiber Distributed Data Interface (FDDI). Errors can also be caused by failures in the authentication . which exempts the internal NIC of the RRAS server from using IPsec. For this reason. The –l 5000 option causes Ping to create a 5000-byte ICMP packet payload. Disable the L2TP ports on the VPN servers. Then a domain logon script is deployed to perform ping <testserver> or nbtstat –n <testserver>. you can determine if all computers are receiving policy and verify that all areas of your network can reach the test computer by using IKE and IPsec protocols. destination 1701. and the VPN server would fail the negotiation because the proposed filter is too general. "How to Configure a L2TP/IPSec Connection Using Preshared Key Authentication". hosts that reside on different MTU links (such as Asynchronous Transfer Mode (ATM). The simplest solution in this list is the first. A default domain IPsec policy is deployed to all domain members that authenticates with the same preshared key and contains one rule with one filter for all traffic (including ICMP) to the test computer's IP address. "Default MTU Size for Different Network Topology". The VPN server will only accept quick mode filters that are specific to L2TP: UDP source 1701. between 1gigabit and 100 megabit Ethernet) may have problems passing fragments.200 Server and Domain Isolation Using IPsec and Group Policy from the client will propose the filter for all traffic. see Knowledge Base article 240262. Troubleshooting IKE IKE and IPsec most commonly fail when first deployed because network filtering does not permit UDP 500 or IPsec protocol packets. such as the predefined example Server (Request) policy that uses a preshared key. and Token Ring) require ICMP PMTU messages to properly discover the network path MTU for IPsec-protected TCP packets. Manually customize the IPsec policy configuration for L2TP to use only the external IP address for certificate authentication for UDP 1701 traffic.

although Windows 2000 events are similar in nature.log file. if the computer does not initiate a new IKE negotiation to its peer. When this happens. if you enable this log the IKE negotiation may slow down. To re-establish SAs. The Windows Security Log is the recommended starting point when trying to determine the reason for an IKE negotiation failure. such as any Kerberos protocol errors.log file. After the IKE negotiation process is well understood. which will change timing conditions and cause all of the existing state to be lost if the service is restarted to re-read the registry key (required in Windows 2000 and Windows XP SP1). Standard auditing can be enabled dynamically without changing the IPsec configuration or the running state of the service. the IPsec service must be restarted to enable detailed IKE logging to the Oakley. IKE failures cause the computer that experiences the failure condition to stop participating in the IKE negotiation. However in Windows 2000. a network trace from at least one side of the communication should be obtained (if possible) to identify problems within IKE before attempting to gather an Oakley. However. IKE logging can be enabled and disabled via command line when needed. Currently. there is no published . the IKE negotiation process. it may be necessary to stop and restart the IPsec service to study the network traffic and capture Oakley. Deployed servers in server and domain isolation scenarios should have the ability to capture a trace with Network Monitor. If IKE fails. The events documented in this section apply to Windows XP and Windows Server 2003. IKE audits may be disabled in cases where IKE auditing generates so many success or failure events that other Logon/Logoff category events cannot be effectively monitored. In Windows XP SP2 and Windows Server 2003. It does not address all IKE audit events or failure conditions. To see IKE events. In some situations. The Oakley.Chapter 7: Troubleshooting IPsec 201 system." After auditing is enabled. Be sure to check this value on computers that are being investigated. and then an IKE main mode initiation. the peer will have to wait five minutes for the IPsec SAs to time out before attempting to re-establish them.log file provides the most detailed logging available and may be required from both sides of the negotiation (with synchronized times). After the restart. and the other computer in the negotiation will usually exhaust its retry limit and time out.log file results from a clean state. IKE logging has been improved in each release since Windows 2000. and there may be several possible causes for the same error message. When the IPsec service is stopped manually or as part of a computer restart or shutdown. a quick mode negotiation is first attempted for one minute. the IKE service will make audit entries and provide an explanation for why negotiation failed. which causes IPsec SA state to remain on the peer. Descriptions of these error code values are available from Microsoft MSDN® on the System Code Errors (12000-15999) page. This section attempts to explain only the most common events related to the Woodgrove Bank domain isolation scenario. Sometimes the operating system disables the ability to send packets before IKE is done sending delete messages to all active peers. or when incompatible policy designs or pre-existing policy is merged with domain policy. Troubleshooting IKE negotiation requires in-depth knowledge of the expected behavior of IPsec policy design. try to study the failure and capture logs without making any changes. the peer can believe it is securely connected to the computer that is being restarted. all IKE audits can be disabled with a DisableIKEAudits registry key. IKE attempts to send delete messages to clean up the IPsec SA state on all actively connected peers. Error code values are often reported in IKE audit events and log details. The list of Event 547 failures and potential causes are described in detail in the following sections. and IKE failure events. the explanation for IKE negotiation failures is almost always reported as an Event 547 failure. in Windows XP SP2 and Windows Server 2003. However. auditing must be enabled for success or failure in the group security policy setting "Audit Logon Events.

The following example shows how these parameters will look in a 541 event: ESP Algorithm None HMAC Algorithm None AH Algorithm None Encapsulation None InboundSpi 31311481 (0x1ddc679) OutBoundSpi 0 (0x0) Lifetime (sec) <whatever is configured for QM lifetime> Lifetime (kb) 0 Note Soft SA events to the same destination IP address will have a different timestamps and different inbound SPI values. this file is created and a new empty Oakley. An IKE main mode success event 541 is created when a soft SA is established. Oakley. a server) has deleted these SAs and is not initiating a quick mode rekey. contact Microsoft Product Support. If the initiator is not configured to Fall back to clear.log file for as long as the IPsec service is running. limited to 50. Five minute delays may be experienced if one computer believes there is an active IPsec SA pair between them and the other computer (for example. Current log for IKE. Oakley. The following four detailed logging files are maintained for IKE: • • Oakley.sav.log.000 lines. Because of space constraints. Windows 2000 IKE supported single delete . although not impossible. For purposes of this guide. The indication that a 541 event shows a soft SA is that the outbound SPI is zero and all algorithms are shown as None.bak file is saved under this name when the IPsec service starts.202 Server and Domain Isolation Using IPsec and Group Policy guidance for interpreting the Oakley.log.log. IPsec-secured communication cannot be negotiated. For help with interpreting the Oakley. Oakley.bak. then failure to contact the destination will appear as an audit event 547 in the Security Log with one of the following text entries: • • • No response from peer Negotiation timed out IKE SA deleted before establishment completed However.log file is saved under this name when the IPsec service starts.log file.log. One minute connectivity delays are experienced whenever two computers become unsynchronized with regard to whether an active IKE main mode exists between them. such interpretation is considered part of the Tier 3 skill set.sav.log. The previous Oakley.log.log file. brief excerpts from log details are provided here for only a few errors. Correlation of IKE messages to packets in a network trace should be cross-checked using the ISAKMP header cookies and IKE quick mode message ID fields. Expected IKE Behaviors If the IKE main mode initiation is blocked by the network from reaching the intended destination IP address.log file is started. for domain isolation clients this condition may not appear as a failure if their policy allows Fall back to clear. • • A common mistake that is often made during troubleshooting is not synchronizing the time on the two computers from which logging and network traces are captured.000 lines are accumulated in Oakley.bak. This makes log correlation difficult. After 50. The previous Oakley. This file continues to be overwritten as necessary by the newer Oakley.

However." then these filters would have been deleted when the IP address disappeared. If the laptop still has an IKE main mode. However. The IKE main mode SAs continue to live for a default time of 8 hours on these remote computers. so it does not recognize these messages and does not reply. but may be deleted any time before that for internal reasons known to IKE. They are often common. and IKE quick mode deletes are sent to the former address of the laptop. and expected during regular operations for mobile clients. These two events are essentially the same. These delete messages will have the same source IP that was used for the IKE SA. none of the filters with a negotiate action use My IP Address in the domain isolation solution—they are all Any <-> subnet filters. the laptop user may notice a one minute delay when reconnecting to remote resources. On the laptop. benign. However. it will attempt an IKE quick mode negotiation. which means the IKE and IPsec SAs are not immediately deleted. The most common scenario for this situation is one in which a domain isolation laptop user ejects the laptop from a docking station to go to a meeting. the IKE SA delete messages are not received by the laptop at its former IP address. Accordingly. The docking station has a wired Ethernet connection. which were sometimes lost. IKE expires the retry limit after one minute and attempts a new IKE main mode negotiation. the IPsec SAs time out on remote computers that were formerly connected. IKE will attempt to send delete messages for these SAs and will delete them internally. If filters had been expanded from "My IP Address. there may now be a difference in IKE state between the laptop and these remote destinations. e-mail clients and other applications typically reconnect to the same destinations.Chapter 7: Troubleshooting IPsec 203 messages. and the act of removing that network interface requires that all filters associated with that interface be deleted (if they were filters expanded from My IP address). Whenever a filter of any type is deleted in the IPsec driver. Windows XP and Windows Server 2003 added support for "reliable" delete functionality in the form of multiple delete messages as a safeguard against dropped packets. The IKE negotiation may report a timeout for a variety of reasons. the driver informs IKE that all IKE SAs and IPsec SAs using that IP address must also be deleted. Consequently. Meanwhile. except when IKE terminates the negotiation with an "IKE SA deleted before establishment completed" event. which now succeeds. delete messages often do not get sent because there is no network connectivity in the seconds after the disconnection occurs (the laptop is ejected). the IPsec SA and IKE SA states remain active in the laptop because these filters are not deleted on each address change. In this particular case of Any <-> subnet filters the filters are never deleted. which frequently and quickly change network connectivity states when the following events occur: • • • • Users insert and remove laptops computers from docking stations Users unplug a wired connection Laptop computers hibernate or go into standby mode Computers go beyond the range of a wireless connection . The source IP address does not matter to the remote computer if the ISAKMP header cookie pair is recognized and cryptographic checks on the packet are valid. When the laptop finally reconnects to the docking station. Microsoft expects to be able to improve this behavior in future updates for all versions of Windows that support IPsec. File shares. it receives the same IP address again. The "Negotiation timed out" event occurs when any step of an IKE negotiation (except Fall back to clear) fails because the retry limit is exhausted. although a different source IP may be present on the connected interface at the time the delete is sent. The quick mode uses a cryptographic state that the remote peer has deleted. Of course.

However. any of these events make it appear as if the peer computer just dropped off the network. in all cases of negotiation timeouts and "IKE SA deleted before establishment completed" events. The remote computer will attempt to rekey or renegotiate until the IKE negotiation step times out. . Negotiation timeout failures were enhanced in Windows Server 2003 to identify where in the IKE negotiation the timeout occurred by identifying the last successful step in the negotiation. IKE main mode SAs will display in the IPSec Monitor MMC snap-in and through command-line query tools.5 IKE Main Mode Informational Log Messages Log text New policy invalidated SAs formed with old policy Description A Windows 2000 message that indicates an IPsec policy change caused deletion of current IKE or IPsec SAs. Tier 2 support should identify if the remote computer actually failed the negotiation by checking for 547 failure audit events on that computer for up to one minute before IKE logs the timeout. New IPsec SAs will be formed based on current traffic flow that use the new IPsec policy. These events can also be caused by the presence of network address translation (NAT) when IKE is negotiating without NAT-T capabilities. IKE Negotiation Success Events If IKE negotiation is successful. not IKE main mode SAs • • For Windows XP: IPsec monitor snapin. IKE Quick Mode was deleted 543. look for the following logged events in the computer’s events logs: Table 7.exe. ipseccmd show sas For Windows Server 2003: IPsec monitor snapin. IKE negotiation will also time out if the remote peer encounters a reason to fail the IKE negotiation. This error is benign. Therefore. IKE Main Mode was deleted IKE Main Mode Informational Log Entries To determine whether there is a problem with main mode exchange.204 Server and Domain Isolation Using IPsec and Group Policy • • A VPN connection is disconnected A PCMCIA network card is ejected while it is connected To a remote computer. IKE Main Mode or Quick Mode established 542. To show a list of the current IKE main mode SAs • For Windows 2000: ipsecmon. netsh ipsec dynamic show [mmsas | qmsas] Successful main mode and quick mode SAs will generate the following events in the Security Log if auditing is enabled: • • • 541. netdiag /test:ipsec /v Note This command shows only IPsec SAs.

it may be expected that computers do not have a rule to negotiate with certain address ranges or subnets. This event should not happen on properly configured systems. This event would be expected for outbound connections to computers that are not using IPsec and that are not members of the exemption list. IKE has recovered from denial of service prevention mode and has resumed normal operation IKE has recovered from what it believed was a denial of service attack condition and resumed normal operation. IKE security attributes are unacceptable. caused by high machine loads and/or a large number of client connection attempts. the computer is just under an extreme load. It also could be the result of a denial of service attack against IKE. but will not get replies because no policy is configured. Complete the steps in the "Verifying the Correct IPsec Policy" section to investigate. which are so common for disconnected computers. Computers on those subnets may attempt IKE initiation. These entries are informational and can be used to provide additional information to help determine the real cause of a problem. Tier 2 support should be alert to connectivity problems that block IKE negotiation. However. Tier 2 support focuses on investigating the IKE failure on the remote computer rather than negotiation timeouts. Delta Time 63 Negotiation timed out-Processed second (KE) payload Responder . Otherwise. No policy configured. and policy design incompatibility in which the responder to an IKE main mode or IKE quick mode negotiation does not find a specific filter to match the incoming request. If so. Delta Time 63 Negotiation timed out-Processed second (KE) payload Initiator. IKE Main Mode Negotiation Failure Events #547 The following IKE failure events can occur when an IKE main mode negotiation fails: • No response from peer. and is beginning denial of service prevention mode Description This condition could be normal. They can also indicate that the remote computer failed the IKE negotiation. Indicates that IKE believes it has been flooded with IKE main mode SA establishment requests so it is going to drop many of them as part of a denial of service attack response strategy. Otherwise. there will usually be many audits for failed IKE negotiations to spoofed IP addresses.Chapter 7: Troubleshooting IPsec 205 Log text IKE has a large number of pending Security Association establishment requests. • • Any of the following "Negotiation timed out" messages could be expected for reasons discussed previously. Existence of these entries does not indicate a failure to communicate. which will also produce this event. • • • • Negotiation timed out Negotiation timed out-Processed first (SA) payload Responder. This event indicates a problem if the source IP address is an address within the internal subnets or may indicate a mismatched filter set. Typically.

or because access to domain controllers is not available. then the IKE negotiation on the side that enforces the access rights will report a 547 event "Failure to Authenticate using Kerberos" as described earlier. If an inbound IKE negotiation fails because of "Access This Computer From Network" settings. Tier 2 support should attempt to correlate these events with unusual performance conditions. Then.206 Server and Domain Isolation Using IPsec and Group Policy IKE Quick Mode Audit Failures (547) The following IKE failure events can occur when an IKE quick mode negotiation fails: • • • • • • • No policy configured-Processed third (ID) payload Initiator Security association negotiation failure because of attribute mismatch General processing error Failed to obtain new SPI for the inbound SA from IPsec driver IPsec driver failed the Oakley negotiation. That event does not specifically indicate that the problem is the failure when checking "Access This Computer From Network rights. This situation could exist because there is no domain trust. the computer may not be able to obtain proper Kerberos tickets. Tier 2 support should first follow the steps in the "Verifying the Correct IPsec Policy" section. such as 100% CPU utilization or a very low kernel memory conditions." and therefore an Oakley. . If the computer is a member of a group that has authorized access. Complete the procedures discussed in the "Verifying Connectivity and Authentication with Domain Controllers" section earlier in this chapter. Note that IKE quick mode failures with negotiation timed out would be expected if the computer was no longer using its former IP address for any of the reasons explained earlier. Alternatively. Troubleshooting IKE Failures Caused by Authentication The following messages are related to IKE authentication failures: • • • • • • Specified Target is Unknown / No authority could be contacted for authentication The specified target is unknown or unreachable-Processed first (SA) payload Initiator. then the computer may not have Kerberos tickets that reflect the new membership. If any of these errors are received. The group membership of the IKE initiator should be investigated to see if it is in fact a member of an authorized group. Delta Time 0 IKE Authentication Credentials Are Unacceptable The Logon Attempt Failed Failed to Authenticate Using Kerberos IKE failed to find valid machine certificate The first two messages indicate that the Kerberos identity of the remote computer can not be used to obtain a service ticket for the remote computer.log file must be obtained from the server to see the specific error generated. no filter exists Failed to add Security Association to IPsec driver Negotiation timed out IKE quick mode should not fail for properly designed IPsec policies and under typical operational loads.

The application may hang until it is terminated or encounter other unusual errors if the connection does not succeed because of an IKE negotiation failure or an IPsec block filter. But TCP and UDP Require IPsec Some applications use an ICMP echo request (Ping) message to determine whether a destination address is reachable. For example. When this application traffic is protected by IPsec. The IPsec policy designer should understand these impacts. The following sections highlight these in the context of the Woodgrove Bank domain isolation solution. so an application may receive an ICMP response from a target destination. the application may not be able to connect to the target destination by using IPsec-protected TCP and UDP traffic when the IKE negotiation fails. These applications may make modifications to the packet. which will cause IPsec to drop the packet. such as report an error or try to connect to an alternate destination. . particularly the CPU utilization of both computers and network packet loss. Initial Connection Delays The IKE negotiation involves substantially more processing and time to complete than a TCP three-way handshake or an unauthenticated Nbtstat single packet query and response. Applying IPsec policy should be largely transparent to applications. which can result in a system error (blue screen). The network socket layer is not aware of IPsec filters or that IKE is negotiating security for traffic. IKE negotiations using Kerberos authentication typically complete successfully within 1-2 seconds. Application Hangs Waiting on Network Response Some applications are written to assume that the time to connect or to receive an error message is very quick. However. Alternatively. Applications that expect a quick TCP or UDP connection response from a target destination may determine that the destination is not responding and take some other action.Chapter 7: Troubleshooting IPsec 207 Troubleshooting Application-Related Issues This section discusses how application design may interact with the use of IPsec in Microsoft Windows. or that the connection was refused. applications may also interpret failures to reach domain controllers as being caused by the destination computer being unavailable. ICMP Permitted. the application may be confused by IPsec modifications. These applications will wait for the connection to complete (for the socket bind operation to complete) before they display changes in the user interface. However. this timing is dependent on many factors. Kernel Mode Network Packet Processing Affected Applications that involve network drivers or other kernel-level packet processing may not work properly when IPsec secures traffic. However. the application may appear to hang briefly during successful connections. The application usually interprets these conditions as being caused by the remote host being down or a network failure. IPsec does not protect ICMP traffic in this solution. Fall back to clear always requires three seconds before it allows the first packet of the TCP handshake to be sent unprotected. and support personnel should be aware of these impacts so that they can assist with rapid problem classification and identification. However. there are circumstances in which having IPsec policy assigned or protecting traffic may cause the application to behave differently.

Firewall functionality provided by the VPN client. which are called Winsock Layered Service Providers (LSPs). Therefore. If this function call is not allowed to pass through any installed LSP. If the IPsec service does not respond to the stop request. the outbound SNMP trap packets may be discarded by the IPsec driver before the soft SA is established. Even if IPsec is allowed to Fall back to clear. Layering that prevents native IKE and IPsec encapsulated packets from being passed through the third-party IPsec tunnel. IKE cannot monitor the required UDP ports. In Windows Server 2003. Winsock LSP troubleshooting consists of identifying that LSPs exist. Some SNMP-based tools depend upon SNMP trap events being sent to untrusted hosts that serve as event collectors. Netdiag /debug.208 Server and Domain Isolation Using IPsec and Group Policy Network Scanning from Hosts in Isolation Domain Affected Host-based tools that seek to rapidly probe remote IP addresses or open ports on the network may run much slower if IPsec attempts to secure their probe traffic. However. IPsec interprets this as a failure of the component to enforce required security policy and reacts defensively. For Tier 2 support. then the IPsec service must be configured as disabled and the computer restarted. Typically. for members of the trusted domain in this solution. The probe traffic may cause a denial of service on the local host by triggering IKE to initiate to hundreds of IP addresses within a few seconds or minutes. the native IPsec service is required. Similarly. which results in application failure or false reports that a destination cannot be reached.exe. "Fail to a Secure Mode" is invoked.0 part of the Windows Platform Software Development Kit (SDK). Even if IKE does appear to have proper initialization. An administrator must log in by using a desktop login to stop the IPsec service and restore connectivity. these clients disable the IPsec service and are not in conflict with native Windows IPsec.The IKE component of the Microsoft implementation of IPsec uses an extended Winsock API function whose function pointer is determined by calling WSAIoctl(). the ability for the computer to send and receive IKE and IPsec protocols may be blocked by the LSP or another installed third-party program. IPsec kernel packet processing for both may require the ESP protocol. and the IPsec driver is put in block mode. third-party IPsec implementations may conflict for the following reasons: • • • • • Both IKE implementations may need UDP port 500. Tier 3 support would be engaged to conduct further investigation to identify the application that installed the LSPs and to reorder them or remove them to see if the IPsec service or IKE no longer has problems. some UDP-based applications (such as NTP and the Windows domain controller locator) only wait three seconds for a reply to be received. An applet that is available for viewing LSPs in the Winsock 2. Third-Party IPsec VPN Clients A number of issues may occur when a third-party IPsec implementation is installed as part of a remote access VPN client. . Tools for detecting Winsock LSPs include: • • Sporder. Winsock Layered Service Provider Issues Some legitimate applications (such as personal firewalls) and some malicious applications (such as spyware) can cause problems by inserting their own network traffic inspection functions. Winsock LSP functions that are installed as part of the client.

this support will be provided by contracted IPsec specialists or support organizations. white papers. Tier 3 support may also be responsible for developing and documenting disaster recovery plans. or if your organization does not have sufficient expertise to use advanced troubleshooting techniques. Network monitor traces of communications taken from each side while IPsec policy is active. a network trace of traffic while IPsec is not active should also be made available. Tier 3 IPsec support requires in-depth understanding of IPsec operation and the Microsoft TCP/IP stack. • • . DNS lookups. additional expertise will be required to analyze the problem and find a resolution. WINS servers. and other related traffic can be identified. and domain controllers. Engaging Microsoft Product Support Services If the troubleshooting procedures described in this chapter do not help you solve your problem. It is important to collect as much diagnostic information as possible. before engaging Product Support Services. In some cases. such as Microsoft's own Product Support Services. and information about IPsec architecture and taxonomy. In many cases. Also indicate the IP addresses of the DNS servers. Also. showing the names of the source and destination computers. if communications fail while IPsec policy is active and they succeed with IPsec policy disabled or the service stopped. The network traces should include all inbound and outbound traffic on each computer (if possible) so that authentication requests. their IP addresses at the time that the log files are collected. you may need to escalate the problem to Microsoft Product Support Services. such as from logs and network monitoring. preferably simultaneously so that the packets can easily be correlated in the two trace files. It is very important that system time be synchronized between these systems so that logs and trace files can be correlated. Tier 3 Troubleshooting If Tier 2 troubleshooting fails to resolve the issue. Use this list to gather information that Tier 3 support or Microsoft Product Support Services will need to analyze the problem: • Security requirements for inbound and outbound authentication and authorization for each computer. The skills that Tier 3 staff members acquire may allow them to become responsible for training staff members in lower-level tiers and for developing supporting documentation such as technical briefs.Chapter 7: Troubleshooting IPsec 209 VPN vendors may be the best source of information about whether they support the native Windows IPsec service being enabled. and operating system versions (including Service Pack information). The native Windows VPN clients are compatible with IPsec transport mode end-to-end through the VPN tunnel. FAQs. A representative diagram of the scenario. This expertise is considered Tier 3 support. A brief description should be available to describe how the Group Policy and IPsec configuration on the computer fulfill these requirements. the VPN vendor gateway supports Windows 2000 and Windows XP PPTP and L2TP/IPsec VPN clients. Tier 3 support staff members require significant training on IPsec and the use of IPsec in server and domain isolation scenarios. support guides. and whether IPsec-protected transport mode traffic is supported end-to-end through their remote access connection.

It should be obvious to the reader that IPsec troubleshooting is technically complex and requires skills in many areas of technology besides IPsec. If the IPsec policies that apply to each computer can be easily saved. However. and Group Policy. including networking. also run portqry -v -local >portqry-v-computername. 3.txt. If a Windows RAS or VPN client is involved. Where applicable. • • • • To create a formatted text file of these filters 1. netdiag /debug >netdiag-debug-computername. The example scripts provided with this guide were tested in the Woodgrove Bank scenario test lab implementation to prove their effectiveness. A tab-delimited text file with all of the filter details can be imported into a spreadsheet or word processing document. (Netdiag generates a lot of network traffic.log and network traces are captured. not just domain or local policy. and Windows Server 2003.) For Windows XP and Windows Server 2003. Right-click the IKE Quick Mode Specific Filters node in the left pane of the tree. It is not possible to provide information about all potential errors. because IPsec and network security are so complex that all variations could not be listed. Active Directory. then they should be included. Select Export List. Save the tab-delimited text file as IKE-qm-<specific-computername>. The entire System Log. However.log files and network traces are collected.log files from each side that are collected at the time that the problem occurs and the time that the network monitor traces are recorded. the information provided in this chapter should make it possible for the reader to troubleshoot all but the most obscure issues that can affect a server and domain isolation solution. IKE Oakley.txt log output run just before or just after capturing the network trace. the RASDIAG tool should be used to collect information. 2. The details of the IPsec policy for each computer. and Application Log of each computer at the time that the Oakley.210 Server and Domain Isolation Using IPsec and Group Policy • For Windows 2000. . which does not need to be part of the network trace. The best format for analyzing the active policy on a computer is a listing of the IKE main mode-specific filters and IKE quick mode-specific filters from the IPsec Monitor MMC snap-in.txt or with a similar naming convention that includes the computer name. Summary This chapter provided detailed information about processes that will help both Tier 1 support (help desk staff) and Tier 2 support (IT professional network support staff) understand how to troubleshoot common IPsec communication problems. However the active IPsec policy of the computer is often the combination of several types of IPsec policy configuration. Security Log. Windows XP. The file names for these files should indicate the computer name. Any Group Policy-specific log files that created at the same time that the Oakley. these scripts are designed to be customized to meet an organization's needs and are therefore unsupported by Microsoft. the developers of this chapter have streamlined the possible options to focus the guidance on those areas that are most likely to experience problems in the type of server and domain isolation environment that was detailed in this guide.

and Windows Server 2003. see the following references: • • • • • • • • "Basic IPsec troubleshooting in Microsoft Windows 2000 Server" Microsoft Windows 2000 Advanced Documentation "Basic L2TP/IPSec Troubleshooting in Windows XP" IPsec Troubleshooting Tools The Architecture Overview diagram in the "Windows 2000 TCP/IP Implementation Details" white paper Figure B1 in the Overview of Windows 2000 Network Architecture How TCP/IP Works How IPSec Works . Windows XP. there is enough additional reading material referenced in the following section to keep you occupied for several years! More Information • • For detailed background information on IPsec. For detailed information on troubleshooting TCP/IP issues.Chapter 7: Troubleshooting IPsec 211 For those readers who wish advance their knowledge into the realms of Tier 3 support. see the following technical references: • • TCP/IP in Windows 2000 Professional The “Troubleshooting Name Resolution and Addressing” section of the “Configuring IP Addressing and Name Resolution” chapter within the Windows XP Professional Resource Kit "How to troubleshoot TCP/IP connectivity with Windows XP" Windows Server 2003 TCP/IP Troubleshooting • • • For online help and resource kit documentation that specifically discusses IPsec troubleshooting in Windows 2000. see How IPsec Works.

you configure IPsec rules (which determine IPsec behavior) and settings (which apply regardless of the rules that are configured). whether traffic is permitted. if applicable. whether to allow unsecured communication with computers that do not support IPsec. you configure a filter list that includes one or more filters. how to authenticate an IPsec peer. it is useful background information. and whether to use perfect forward secrecy (PFS). Additional information in the white paper describes the first model for using IPsec to secure network access to internal Microsoft® Windows® servers that process or store sensitive data. It is designed to provide the prerequisite level of understanding for IPsec as described in the Server and Domain Isolation Using IPsec and Group Policy guide . Although this extra information is not required to understand the Server and Domain Isolation Using IPsec and Group Policy guide . The IKE main mode SA protects the IKE negotiation itself.The SA created during the first IKE negotiation phase is known as the IKE main mode SA (also known as the ISAKMP main mode SA). a filter action. whether to accept initial incoming unsecured traffic. similar to a firewall rule. You can configure a filter action to permit. protocols. blocked. encryption and hashing algorithms. and other settings required to configure both the Internet Key Association (IKE) main mode and IPsec security associations (SA). If you configure a filter action to negotiate security. and port numbers. Key exchange settings and key exchange security methods determine the IPsec protocol wire formats (authentication header (AH) or encapsulated security payload (ESP)). and other settings. key lifetimes. Filters define the traffic that you want to inspect. a connection type. block. When you configure an IPsec rule. Although multiple IPsec policies can exist on a computer. you must assign it to a computer for the policy to be enforced. with source and destination IP addresses. or negotiate security (negotiate IPsec). only one IPsec policy can be assigned to a computer at a time. An IPsec rule is typically configured for a specific purpose (for example. processes. The content of this appendix was originally published as part of the "Using Microsoft Windows IPsec to Help Secure an Internal Corporate Network Server" white paper which was jointly written by Microsoft and Foundstone Strategic Security. “Block all inbound traffic from the Internet to TCP port 135”).Appendix A: Over view of IPsec Policy Concepts This appendix provides a detailed overview of the terms. An IPsec rule determines which types of traffic IPsec must examine. After you configure an IPsec policy. and concepts of IPsec. Introduction When you create an IPsec policy. The SAs created during the second IKE negotiation phase are known . A filter action defines the security requirements for the network traffic. An SA is the agreement of security settings associated with keying material. and an IPsec encapsulation mode (transport mode or tunnel mode). authentication methods. or security is negotiated. you must also configure key exchange security methods (and their preference order).

as the IPsec SAs (also known as IKE quick mode SAs because each IKE quick mode negotiation negotiates the IPsec SA for each direction). IPsec Policy Filters Filters are the most important part of an IPsec policy. which is required to negotiate a security relationship between two computers. IPsec filters are inserted into the IP layer of the TCP/IP networking protocol stack on the computer so that they can examine (filter) all inbound or outbound IP packets. IPsec tunnel mode rule configuration is very different from IPsec transport mode rule configuration. The filtering rules associated with an IPsec policy are similar to firewall rules. security might not be provided. see the Help and Support Center for Microsoft Windows Server™ 2003. and it does not support dynamic or stateful filtering features. By using the graphical user interface (GUI) provided by the IP Security Policy Management Microsoft Management Console (MMC) snap-in. The IPsec SAs protect application traffic. Filters are associated with a corresponding filter action by the security rule in an IPsec policy. you can configure IPsec to permit or block specific types of traffic based on source and destination address combinations and specific protocols and ports. Windows IPsec supports both IPsec tunnel mode and IPsec transport mode as an option in the rule. host-based firewall. Except for a brief delay. Note Windows IPsec is not a full-featured. such as tracking the established bit during the TCP handshake to control the direction in which communication can flow. This section provides the most important details to understand about how IPsec filters affect packet processing. or if the IP addresses change before the policy's filters are updated. If you do not specify the proper filters in either client or server policies. . What is important to understand is how all of the filters contained in all of the rules will combine to provide the required inbound and outbound access controls. Understanding IPsec Filtering Filter lists are simply listings of known subnets and infrastructure IP addresses. This section provides information about the following important IPsec policy concepts: • • • • • • • IKE negotiation process IPsec policy filters Security methods IPsec protocol wire formats IKE authentication IKE authentication method and security method preference order Security negotiation options For more information about IPsec policy concepts. IPsec is transparent to enduser applications and operating system services.

10/24.1. The most generic filter that can be defined would be one that matches any IP address. which is also referred to a how specific the filter is when selecting traffic. protocols. The IPsec service interprets all generic filters into specific filters at the time that the IPsec policy (or change) is being applied on the computer. or order. TCP source port Any. any protocol Any <-> 192. have no effect on the filtering behavior enforced by the IPsec driver during packet processing.168. These filters contain the full filter information about addresses. These filters use only the source and destination address of the filters defined in IPsec policy to control IKE main mode. then on protocols.1. IKE quick mode filters. IKE main mode lifetimes and limits on the number of session keys generated from the same master key. any protocol Any <-> 192. This approach ensures that the order of rules in a policy. Perfect forward secrecy security settings. any protocol Any <-> 192. The filter action that corresponds to the most specific filter matching a packet is the only action taken for that packet.Appendix A: Overview of IPsec Policy Concepts 215 The Windows Server 2003 IPsec Monitor MMC snap-in provides the most detailed view of the ordering of IPsec filters.10/24. IKE main mode filters. Each specific filter has a corresponding weight and a set of security methods that define: • • • • Options for AH or ESP encapsulation in transport or tunnel mode. Specific filters have a built-in algorithm for calculating the weight. A list of encryption and integrity algorithms. consider the following four filter definitions: • • • • Any <-> Any. Filters defined in IPsec policy are considered "generic" filters because they may have to be interpreted by the IPsec service when the policy is applied.168. and ports. The packets are matched against the most specific filters first to minimize the time required to process each packet against the total set of filters.0/24. IKE quick mode negotiates this filter definition to determine what traffic can be secured inside an IPsec security association pair. and finally on ports that may be defined within the filter. all protocols. All of the specific filters are sorted according to their weight. When the IPsec service processes a set of IPsec policy rules. Authentication method(s). destination port 25 . The weight of the filter is evaluated first on IP address. A higher weight value corresponds to a more specific filter. such as Diffie Hellman master key strength and the encryption and integrity algorithms used to protect the IKE negotiation itself. The IPsec driver matches all inbound and outbound IP traffic against these filters. in the order specified by the highest weight. The following section on the IKE negotiation process describes how IKE negotiates and manages IPsec security associations using these policy controls. the filters are copied into two types in order to provide control of the two phases of the IKE negotiation: 1. and ports. The IKE main modespecific filters each have an IKE main mode negotiation policy associated that defines: • IKE main mode security methods defined for the IPsec policy under key exchange settings. For example.168. IPsec security association lifetimes in kilobytes and seconds. • • 2. The IKE quick mode-specific filters are the list of filters that are given to the IPsec driver to enforce.1. and the ordering of filters in each different filter list.

In Windows XP and Windows Server 2003. protocol.168. performance testing should confirm acceptable impacts of a particular policy design. performance testing of IPsec policy designs should be factored into the planning effort. most general filter is Any <-> subnet.10 using TCP port 25 and the corresponding outbound responses from port 25. Potential Filter Design Issues When defining filters. Filters of the form From <IP address> To <IP address> regardless of protocol or ports were optimized by using the Generic Packet Classifier (GPC) driver for extremely fast lookup. As noted above. The precise impact on performance is very difficult to estimate because it depends on the overall traffic volume. My IP Address filters are used to create a more specific filter that permits Internet Control Message Protocol (ICMP) traffic to be sent and received in clear text among all computers. filters that specify Any IP Address to Any IP Address should be avoided for hosts running Windows 2000. Therefore. the rest of the more specific filters could be considered exceptions to the first filter.0 subnet with the exception of 192. any protocol. the amount of IPsec-secured traffic being processed. which means that Any IP <-> specific IP (or subnet) filters will require sequential searching. in the IPsec policy design for Woodgrove Bank. the second filter will be the most specific for that traffic. The use of My IP Address may be appropriate in many cases but may also cause problems for hosts with many IP addresses.10. the more filters in a policy. Filters that do not have a specific IP address for both source and destination cannot be optimized by GPC. an inbound packet destined to TCP port 25 will only match the fourth and most specific filter. The impact of a few hundred filters is not likely to be noticed except on very high throughput computers. This performance impact appears as degraded throughput.168. Finally. The fourth filter is the most specific because it specifies a destination IP address.10." If this filter is being used to block all traffic. Again.1. if traffic is sent to any IP address in the 192. My IP Address might be most appropriately used when permitting or denying traffic to a specific port or protocol. But the implementation is improved over Windows 2000. When all four are being enforced by the IPsec driver. such as a Web server hosting many virtual Web sites.1. certain combinations of source and destination address options should not be used. large exemption lists using My IP address to <specific exemption IP address> are easily supported provided there is enough non-paged kernel memory available to hold the entire filter list. It is only supported by Windows Server 2003 and Windows XP Service Pack 2 (SP2). It may also cause a delay in the availability of IPsec driver packet filtering if there are a large number of filters using My IP Address. the supported. The IPsec driver must scan the whole filter list sequentially to find a match. .168. and port number.168. Therefore.1. All four filters would match inbound traffic from any IP address to 192. increased non-paged pool kernel memory utilization.1. For example. this filter is used with a block action to achieve a default behavior of "Deny All. Typically. GPC can handle almost any number of these filters without throughput performance degradation. The delay may cause a window of vulnerability or delays in connecting securely with IPsec. If a remote system sends TCP traffic to any port other than 25 to 192. and increased CPU utilization. Generally. the more performance impact to packet processing.216 Server and Domain Isolation Using IPsec and Group Policy Any to Any filter is the most general filter possible to define. For Windows 2000. The IPsec service processes them during service startup and when an address change event happens. Windows 2000 does not have optimizations for handling large numbers of filters. or Any <-> My IP address if subnets are not used. many optimizations were made to speed up filter processing so that larger numbers of filters can be used in the IPsec policy. and the CPU loads on the computer. the third filter is matched.

if RFC 1918 private addresses are used as internal network subnets. the following IPsec policy parameters are negotiated: the encryption algorithm (DES or 3DES). The TCP/IP integration with IPsec in Windows XP and Windows Server 2003 was enhanced to filter all types of IP packets. symmetric. to authenticate identities. Windows XP SP2 supports the same filtering capabilities as Windows Server 2003. After IPsec policy parameters are negotiated. a domain logon script may need to be used to check if the IPsec service is running when they connect to the internal network. The responder sends either a reply accepting the offer or a reply with alternatives. public key certificate. mobile clients will be affected when they connect to hotels. . After the DiffieHellman exchange is complete. "IPSec default exemptions are removed in Windows Server 2003" for details on the removal of default exemptions and the degree of filtering support for multicast and broadcast traffic. Windows 2000 was not originally designed to provide filtering for packets using multicast and broadcast addresses because this traffic could not be secured using IKE negotiation. the Diffie-Hellman group to be used for the base keying material (Group 1. they may require Local Administrator rights to stop the IPsec service when connected to other networks. the Diffie-Hellman exchange of public values is completed. Clearly. "IPSec Default Exemptions Can Be Used to Bypass IPsec Protection in Some Scenarios" for an in-depth explanation of the security implications of default exemptions and the changes implemented by Service Pack 3 to remove some of them by default. IKE performs a two-phase operation: Phase 1 (main mode) negotiation and Phase 2 (quick mode) negotiation. Group 2048). then the client will experience three-second and greater delays connecting to every destination. the mobile client may fail to communicate in that environment. If the client is allowed to Fall back to clear. with the negotiation algorithms and methods. in Windows Server 2003. the IKE negotiation will fail because IKE will not be able to authenticate using domain trust (Kerberos). and potentially other internal networks. only limited support for filtering is provided. the two computers establish a secure. In order to ensure successful and secure communication. If mobile clients experience connectivity problems. or preshared key). secret keys between computers. The master key is used. multicast and broadcast packet types were part of the original default exemptions that bypassed IPsec filters. However. secret cryptographic keying material. The Diffie-Hellman algorithm is used to generate shared. The initiator of the communication then presents an offer for a potential SA to the responder. or. the integrity algorithm (MD5 or SHA1). See Microsoft Knowledge Base article 811832. See Knowledge Base article 810207. If the destination replies with an IKE response. Consequently. the IKE service on each computer generates the master key that is used to help protect authentication. Therefore. IKE Negotiation Process The IKE protocol is designed to help securely establish a trust relationship between each computer. negotiate security options. authenticated channel. and the authentication method (Kerberos version 5 protocol. Main Mode Negotiation During main mode negotiation. and dynamically generate shared. First. Confidentiality and authentication may be ensured during each phase by the use of encryption and authentication algorithms that are agreed upon by the two computers during security negotiations. home networks.Appendix A: Overview of IPsec Policy Concepts 217 If a mobile client in the organization is assigned a My IP Address <-> Any IP Address rule and is then placed on an external network. because IKE cannot negotiate security for multicast and broadcast. The result of a successful IKE main mode negotiation is a main mode SA. Group 2.

the authentication method that is configured in the IPsec policy on both computers for the IKE main mode SA should allow authentication to succeed in the direction from which the IKE main mode negotiation is initiated. As this summary indicates.218 Server and Domain Isolation Using IPsec and Group Policy Quick Mode Negotiation During quick mode negotiation. to help prevent a connection from being disrupted. IKE must be able to rekey the main mode SA and negotiate IKE quick mode in either direction.[IP2] CORPSRV Where: • • • • IP1 is the IP address of CORPCLI. the IKE main mode SA between CORPCLI and CORPSRV is bidirectional. a pair of IPsec SAs is established to help protect application traffic. session key material (cryptographic keys and key lifetimes. for each algorithm) is refreshed or exchanged. Likewise. the following SAs are established: CORPCLI [IP1] <-------. and other protocols. In the example scenario.IPsec SA [SPI=x] --------------------> [IP2] CORPSRV CORPCLI [IP1] <-------. and IPsec SAs can expire before a TCP connection ends. two new IPsec SAs are actually established. For example. which is inserted into the IPsec header of each packet sent. TCP connections can be established and ended while IPsec SAs continue. by using the quick mode negotiation to establish two new IPsec SA pairs before the lifetime of the existing IPsec SA pair expires. and the algorithm for encryption (DES or 3DES). IP2] -----> [IP2] CORPSRV CORPCLI [IP1] ---------. one IKE main mode SA and two IPsec SAs are established. the other SPI identifies the outbound IPsec SA.IPsec SA [SPI=y] ---------------------. which can include packets sent over TCP. IKE Main Mode SAs and IPsec SAs Each time IPsec is used to help secure traffic. if encryption is requested. an IKE main mode SA is automatically renegotiated as required (when a main mode SA has expired). y is the SPI that identifies the outbound IPsec SA for CORPSRV to CORPCLI. By Internet Engineering Task Force (IETF) design. One SPI identifies the inbound IPsec SA. the following policy parameters are negotiated: the IPsec protocol wire format (AH or ESP). for IPsec-secured communications to occur between CORPCLI and CORPSRV. The IKE main mode SA expires independently of the IPsec SA pair. IPsec SAs are not dependent on the state of upper-layer protocols. a common agreement is reached regarding the type of IP packets to be carried in the IPsec SA pair that is established. in seconds and kilobits. Either computer can initiate a quick mode negotiation by using the protection provided by the IKE main mode SA. x is the SPI that identifies the inbound IPsec SA for CORPSRV from CORPCLI. IP2 is the IP address of CORPSRV. the IPsec policy settings in the filter action for quick mode should allow successful bidirectional quick mode negotiation. Although this process is commonly referred to as rekeying the IPsec SA. . the hash algorithm for integrity and authentication (MD5 or SHA1). Each IPsec SA is identified by a security parameter index (SPI). User Datagram Protocol (UDP). If a new IPsec SA pair is needed. Therefore. During this time.IKE main mode SA [IP1. The life of the IKE main mode SA is measured only by time and the number of IPsec SAs that have been attempted (not by the number of bytes of data that is transferred in the IKE protocol). IKE attempts to renegotiate. First. After IPsec policy parameters are negotiated.

see the “Deploying IPsec” chapter in the Windows Server 2003 Deployment Kit. IPsec encrypts only the IP payload. and it is the default mode for Windows IPsec. The data is readable but protected from modification and spoofing. original IP packet by creating a new IP packet that is then protected by one of the IPsec protocol wire formats (AH or ESP). such as site-to-site networking through the Internet. and key lifetimes that are used to create the quick mode inbound and outbound SAs.Appendix A: Overview of IPsec Policy Concepts 219 Security Methods Security methods are used during the IKE main mode negotiation to define the encryption and hashing algorithms and the Diffie-Hellman group that is used to create the main mode SA and to help secure the IKE negotiation channel. Windows IPsec is used in transport mode primarily to help protect end-to-end communication (such as communications between clients and servers). . and anti-replay protection for the entire packet (both the IP header and the data payload carried in the packet). For information about IPsec in tunnel mode. You can use IPsec in transport mode or tunnel mode. Security methods are also used during the quick mode negotiation to define the encapsulation mode (transport or tunnel). The protection that is provided depends on the mode in which IPsec is used and the protocol wire format. AH AH provides data origin authentication. AH does not provide data confidentiality. IPsec Encapsulation Modes and Protocol Wire Formats IPsec helps protect data in an IP packet by providing cryptographic protection of an IP payload. When IPsec transport mode is used. the sending gateway encapsulates the entire. IPsec Protocol Wire Formats IPsec supports two protocol wire formats: AH or ESP. IPsec Encapsulation Modes IPsec tunnel mode is most commonly used to help protect site-to-site (also known as gateway-to-gateway or router-to-router) traffic between networks. When IPsec tunnel mode is used. IPsec protocol wire format (AH or ESP). except for the fields in the IP header that are allowed to change in transit. the IP header is not encrypted. which means that it does not encrypt the data. IPsec transport mode is used to help protect host-to-host communications. encryption and hashing algorithms. IPsec transport mode encapsulates the original IP payload with an IPsec header (AH or ESP). data integrity.

ESP ESP provides data origin authentication.509 version 3 public key infrastructure (PKI) certificate. the ESP header is placed before the TCP data. a computer X. the IKE initiator proposes a list of authentication methods to the IKE responder. and then specify the integrity and encryption algorithms to use. in the Custom Security Method Settings dialog box. As shown in the following figure. and an ESP trailer and ESP authentication trailer are placed after the TCP data. in the properties for the appropriate rule. The IP header is not protected. in the properties for the appropriate rule. The responder uses the source IP address of the initiator to identify which filter controls the IKE negotiation.1 Authentication header in a packet To use AH. or communication fails. The responder then replies to inform the initiator of the agreed-upon authentication method. If the selected authentication method fails. select the Data and address integrity without encryption (AH) check box. IKE does not provide a . and the option of confidentiality for the IP payload only. select the Data integrity and encryption (ESP) check box. The two communication endpoints must have at least one common authentication method. Figure A. The authentication method list that corresponds to the filter in the responder’s IPsec policy is used to select one authentication method from the initiator’s list. and then specify the integrity algorithm to use. or a preshared key. IKE Authentication IKE uses mutual authentication between computers to establish trusted communications and requires the use of one of the following authentication methods: Kerberos version 5 protocol. integrity and authentication are provided by the placement of the AH header between the IP header and the TCP data. data integrity. IKE Authentication Process During IKE negotiation. in the Custom Security Method Settings dialog box. Figure A. anti-replay protection. ESP in transport mode does not protect the entire packet with a cryptographic checksum.2 ESP data in a packet To use ESP.220 Server and Domain Isolation Using IPsec and Group Policy As shown in the following figure.

IKE Authentication Methods It is important to choose the authentication method that is appropriate for your IPsec policy. the computer certificate must have a public key length that is greater than 512 bits and use a Digital Signature key usage. then the main mode SA is renegotiated automatically. during main mode negotiation each IPsec peer sends its computer identity in unencrypted format to the other peer. not which specific certificate to use. For this reason. To remove the exemption for Kerberos protocol traffic. and it simplifies certificate deployment by enabling certificate auto-enrollment and renewal and by providing several default certificate templates that are compatible with IPsec. you can use Certificate Services to automatically manage computer certificates for IPsec throughout the certificate lifecycle. For information about the default exemptions in Windows 2000 and Windows Server 2003. the main mode SA lasts for eight hours. If data is still being transmitted at the end of eight hours. During the certificate selection process. and clients must have an associated computer certificate. you define an ordered list of acceptable root certificate authorities (CAs) to use. Note Certificates obtained from Certificate Services with the advanced option set for Enable strong private key protection do not work for IKE authentication because you cannot enter the required personal identification number (PIN) to access the private key for a computer certificate during IKE negotiation. Kerberos protocol traffic is exempt from IPsec filtering. When Kerberos authentication is used. Any computer in the domain or in a trusted domain can use this method of authentication. If authentication is successful and the main mode negotiation is successfully completed. An IPsec policy rule associates each IP address in a filter with an authentication method list so that IKE can determine which authentication method list to use with each IP address. to help secure computers that are connected to the Internet. IKE performs a series of checks to help ensure that specific requirements are met for the computer certificate.Appendix A: Overview of IPsec Policy Concepts 221 method for trying a different authentication method. By default. consult Special IPsec considerations. To use certificates for IKE authentication. An attacker can send an IKE packet that causes the responding IPsec peer to expose its computer identity and domain membership. Both computers must have a common root CA in their IPsec policy configuration. certificate authentication is recommended. For example. Kerberos Version 5 Protocol Authentication The Kerberos version 5 protocol is the default authentication standard in Windows 2000 and Windows Server 2003 Active Directory domains. Certificate Services is integrated with Active Directory and Group Policy. Public Key Certificate Authentication In Windows 2000 Server. in Windows 2000 through Service Pack 3 and in Windows XP. . The computer identity is unencrypted until encryption of the entire identity payload takes place during the authentication phase of the main mode negotiation. you must modify the registry and then add an appropriate IPsec filter to help secure this traffic.

222 Server and Domain Isolation Using IPsec and Group Policy Preshared Keys If you are not using Kerberos authentication and do not have access to a CA. For example. a stand-alone computer on a network might need to use a preshared key because neither Kerberos authentication (through the computer’s domain account) nor certificates from a CA can enable successful IKE authentication in some scenarios. and it is therefore difficult to keep secret. By default. if attackers can capture IKE negotiation packets. To enable IPsec CRL checking Caution: Incorrectly editing the registry may severely damage your system. with a DWORD entry named StrongCrlCheck. Other failures that are encountered during CRL checking (such as the revocation URL being unreachable) do not cause certificate validation to fail. Before making changes to the registry. where: a. If you must use preshared key authentication. Microsoft does not recommend the use of preshared key authentication in Active Directory because the key value is not securely stored. Important: Preshared keys are easily implemented but can be compromised if they are not used correctly. For more information. Under HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PolicyAgent\. 1. you should back up any valued data on the computer. a preshared key can be used. any authenticated user in the domain can view a preshared key if it is stored in an Active Directory-based IPsec policy. IPsec CRL Checking If you use certificate-based authentication. use a 25-character or longer random key value and a different preshared key for each IP address pair. and a local IPsec policy can be read by any system service with Local System user rights. Additionally. b. A value of 1 causes CRL checking to be attempted and certificate validation to fail only if the certificate is revoked (default for Windows XP and Windows Server 2003). Preshared key authentication is provided for interoperability purposes and compliance with RFC standards. By default in Windows 2000. Assign this entry any value from 0 through 2. . These practices result in different security rules for each destination and help ensure that a compromised preshared key compromises only those computers that share the key. IPsec CRLs are not automatically checked during IKE certificate authentication. 2. which means that CRL checking is required and that certificate validation fails if any error is encountered during CRL processing. Set this registry value for enhanced security. Any member of the local Administrators group can view a local IPsec policy. add a new Oakley key. A value of 0 disables CRL checking (default for Windows 2000). see "Authentication Vulnerabilities in IKE and Xauth with Weak Pre-Shared Secrets" authored by John Pliam of the Institute for Mathematics and its Applications. c. you can also enable IPsec certificate revocation list (CRL) checking. A value of 2 enables strong CRL checking. published methods can enable the attackers to discover preshared key values. The preshared key value is stored in plaintext in an IPsec policy.

IKE Authentication Methods and Security Method Preference Order You can configure an IPsec rule to specify only one authentication method or one security method. Because CRLs persist across computer restarts. Figure A. you can specify that both the Kerberos version 5 protocol and public key certificate authentication are offered as authentication methods but give the Kerberos protocol the higher preference. There is a delay between the time that the revoked certificate is placed on an updated and published CRL and the time when the computer that performs the IPsec CRL checking retrieves this CRL. Stop. IKE must succeed using the selected authentication method. and then restart the IPsec service by running the net stop policyagent and net start policyagent commands at a command prompt. Alternatively. Note IPsec CRL checking does not guarantee that certificate validation fails immediately when a certificate is revoked. then CORPSRV uses this authentication method and continues establishing communication. b. Restart the computer. The computer does not retrieve a new CRL until the current CRL has expired or until the next time the CRL is published. as shown in the following figure. restarting the computer does not resolve the problem. Do one of the following: a. IKE does not attempt to use a different .Appendix A: Overview of IPsec Policy Concepts 223 3.3 Authentication method preference order If a client attempts to connect to CORPSRV but only accepts public key certificates for authentication. you can specify a preferred list of authentication and security methods. Preference order applies to authentication methods and security methods so that you can specify each method from most preferred to least preferred. For example. or the communication is blocked. if a CRL cache problem occurs. CRLs are cached in memory and in \Documents and Settings\UserName\Local Settings\Temporary Internet Files by CryptoAPI.

it is recommended that you select this check box so that the client can Fall back to clear and initial connectivity can be established when IPsec is disabled on the server. You can configure Master key PFS in the Key Exchange Settings dialog box.224 Server and Domain Isolation Using IPsec and Group Policy authentication method if the negotiation fails. communication is blocked. you can allow the client to Fall back to clear. If the peer does not respond to the request for security negotiation within three seconds. For client policy. If the peer does respond within three seconds and the security negotiation fails. Fall Back to Clear When Fall back to clear is allowed. for example a TCP SYN packet) is accepted if it matches the inbound filter associated with the filter action. in the properties for a filter action. on the Security Methods tab. the communication that matches the corresponding filter is blocked. where. and Session key PFS on the Security Methods tab. If you enable this option and the server does not respond to the client’s request to negotiate security. The upper-layer protocol response packet (for . Windows IPsec does not allow unsecured communication if the IKE negotiation fails or if an error is experienced during an IKE negotiation (after the reply). for example. traffic is secured by IPsec when possible (if the computer at the other end of the connection supports IPsec with a complementary filter action and filter in its policy). such as failure to authenticate or to reach agreement on security parameters. Security Negotiation Options You can configure whether an IPsec policy allows Fall back to clear (fall back to unsecured communication). in the general properties for a rule. In some cases. The same principle applies to security methods. traffic might be secured by Lightweight Directory Access Protocol (LDAP) encryption or remote procedure call (RPC) authentication mechanisms). normal inbound TCP/IP traffic (traffic that is not secured by IPsec. Keep in mind that although IPsec might not secure such traffic. Inbound passthrough. Inbound Passthrough When Inbound passthrough is allowed. Fall back to clear is a setting that allows interoperability with the following: • • • Computers running operating systems earlier than Windows 2000 Computers running Windows 2000 or later systems that do not have IPsec policy configured Computers running non-Microsoft operating systems that do not support IPsec To enable or disable Fall back to clear. in the properties for a filter action. you can either enable or disable this option. ESP might be preferred over AH. but traffic can be sent unsecured if the peer does not have an IPsec policy to respond to the request for security negotiation. Soft SAs allow normal TCP/IP communication with no IPsec encapsulation to occur. it is useful to allow Fall back to clear. For initial deployments. IKE allows Fall back to clear only if there is no reply. an SA for plaintext traffic (a soft SA) is created. If you clear this check box. and if the server does not respond to the client’s request to negotiate security. However. another application might help secure the traffic (for example. For security reasons. select or clear the Allow unsecured communication with non-IPsecaware computers check box.

Two IPsec SAs are then negotiated. When you enable the default response rule in the client IPsec policy.Appendix A: Overview of IPsec Policy Concepts 225 example. you must also enable it in the client policy. When session key PFS is enabled. in the properties for a filter action. then you do not need to enable the Inbound passthrough option in the server IPsec policy. you should never enable this option on computers connected to the Internet. If you enable master key PFS in a server policy. a TCP SYN ACK packet) matches the corresponding outbound filter and triggers a security negotiation. Session key PFS can be used without a reauthentication and is less resource-intensive than master key PFS. The Inbound passthrough option allows a server to use the default response rule to initiate the security negotiation to clients. Session Key and Master Key PFS PFS is a mechanism that determines whether the existing keying material for a master key can be used to derive a new session key. clients do not need to maintain a filter that contains the IP address of the server. it is recommended that you enable session key PFS or master key PFS only in hostile environments where IPsec traffic might be exposed to sophisticated attackers who will try to compromise the strong cryptographic protection provided by IPsec. Additionally. Because there is significant overhead in enabling this option. select or clear the Allow unsecured communication. You can enable session key PFS by selecting the Use session key perfect forward secrecy (PFS) check box. Master key PFS requires a reauthentication and is resourceintensive. To achieve this. on the Security Methods tab. a new Diffie-Hellman key exchange is performed to generate new master key keying information. not necessarily to the entire communication. If you enable session key PFS in a server policy. . To enable or disable Inbound passthrough. and the traffic is IPsec-secured in both directions. you do not need to enable it in the client policy. PFS helps ensure that the compromise of a single key permits access only to data that is protected by PFS. in the Key Exchange Settings dialog box. It requires a new main mode negotiation for every quick mode negotiation that occurs. but always respond using IPsec check box. PFS helps ensure that a key used to protect a transmission cannot be used to generate additional keys. You can configure master key PFS by selecting the Master key perfect forward secrecy (PFS) check box. If you do not enable the default response rule in the client IPsec policy. in the general properties for a rule.

This value should be increased to 60 minutes in production. the host refreshes its policy from the Active Directory® directory service. The same functionality can be accomplished by setting the Sessions per MM to 1. IKE Main Mode Lifetime: 3 hours.Appendix B: IPsec Policy Summar y This appendix provides a concise listing of information about all policy settings for the isolation groups used in this solution. 3DES/SHA1/Medium (2). Sessions Per MM: 0. Mirrored – Currently Empty Description: "IP addresses for all Cluster VIPs in the organization" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL . General Policy Configuration The following information is contained in all of the policies that are defined in this solution. Policy General Settings • Policy refresh: 5 minutes for test environment rollout.Cluster VIP Exemption List" Filter: My <-> Specific IP Address. 3DES/MD5/Medium (2). IKE MM Key Exchange security methods: 3DES/SHA1/High (2048). infinite. After 60 minutes. • Default Response Rule = disabled Rule 1 Filter List: "IPSEC . this has been deprecated as a feature in Microsoft® Windows® Internet Key Association (IKE) because of lack of support in other products and to eliminate duplicate functionality. Windows 2000 and Windows XP SP1 and earlier IKE compatibility is ensured by using Medium (2). making it possible to quickly respond to any compromises of the network. This functionality allows changes to an already assigned IPsec policy to be deployed to the entire organization's network in (at most) an hour. and will be ignored by Windows 2000 and earlier versions of Windows XP. Master PFS: Not used. • • • • Note High (2048) is supported only by Microsoft Windows Server™ 2003 and Windows XP SP2.

UDP.21.168. Mirrored Any <-> 192.168. Mirrored Any <-> 192.DNS Exemption List" Filter: Any <-> 192. Mirrored Description: "IP Addresses for all WINS servers in the organization" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL .DHCP.1.WINS Exemption List" Filter: Any <-> 192. SRC Port 68 to DST Port 67.1. Mirrored Description: "IP addresses for all DCs in organization" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 5 Filter List: "IPSEC .Rule 2 Filter List: "IPSEC .1. Negotiation Traffic" Filter: My <-> Any.22.22.Domain Controller Exemption List" Filter: Any <-> 192.22. Mirrored Description: "Allows DHCP Negotiation traffic" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 3 Filter List: "IPSEC .168.1.21.168. Mirrored Description: "IP Addresses for all DNS servers in the organization" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 4 Filter List: "IPSEC .168.1.

1.Exempt Subnets" Filter: My <-> Specific IP Subnet.10.LOB Application Servers Exemption List" Filter: Any <-> 192.ICMP. Mirrored Description: "Allows ICMP traffic" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 8 Filter List: "IPSEC . All Traffic" Filter: My <-> Any. Mirrored – Currently Empty Description: "Subnets to be exempted from IPsec communication" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL . Mirrored – Currently Empty Description: "Specific IP addresses to be exempted from IPsec communication" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 9 Filter List: "IPSEC . ICMP.Exempt Addresses" Filter: Any <-> Specific IP Address.168. Mirrored Description: "IP Addresses for all LOB servers in the organization" Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule 7 Filter List: "IPSEC .Appendix B: IPsec Policy Summary 229 Rule 6 Filter List: "IPSEC .

Rule 6. Rule 8. Rule 7. This rule is not to be included in the policies if the filter list is empty.1. This rule permits non-IPsec communication to hosts in the exemption list. Used to identify IPsec policy version. Rule 5. Woodgrove Bank created this filter list for their line of business application servers.1. Rule 10. ICMP.Policy Version: (1.1." Filter Action: IPSEC-Permit Authentication: Kerberos Tunnel: No Connection Type: ALL Rule Behavior Explained Rule 1 This rule is required to exempt outbound communication to the cluster VIPs.1.0. This rule permits non-IPsec Dynamic Host Configuration Protocol (DHCP) negotiation to be used.2. The filter used to implement the filter list is a dummy filter that consists of two specific IP addresses that permit ICMP traffic. Rule 4. Rule 3. This rule should not be included if there is no need for this server to communicate to the cluster VIPs. This rule is only used to track versioning information for the policy. This rule permits non-IPsec communication to Windows Internet Naming Service (WINS) systems in the exemption list. This rule permits non-IPsec communication to subnets in the exemption list. This dummy filter is required because one cannot add an empty filter list to a policy. This rule permits the use of non-IPsec Internet Control Message Protocol (ICMP) traffic. Mirrored Description: "Not a real filter list.230 Server and Domain Isolation Using Ipsec and Group Policy Rule 10 Filter List: "IPSEC .1 <-> 1. This rule permits non-IPsec communication to hosts in the exemption list. Rule 9. . This rule permits non-IPsec communication to Domain Name System (DNS) systems in the exemption list. This rule permits non-IPsec communication to domain controller systems in the exemption list. Rule 2.041001.1600)" Filter: 1. This rule is not included in the policies if the filter list is empty.

policy. Rule 11 Filter List: IPSEC – Organizational Subnets Filter: Any <-> internal subnets. mirrored Filter Action: "IPSEC – Secure Request Mode (Ignore Inbound. It matches traffic destined to secure subnets and requests that IPsec be negotiated. and GPOs used to create the No Fallback isolation group in the solution for Woodgrove Bank. mirrored Filter Action: "IPSEC – Full Require Mode (Ignore Inbound. No Fallback Isolation Group Policy This section provides the details of the filters. Rule Behavior Explained Rule 11. policy. Allow Outbound)" Security Method Preference Order: ESP-null/SHA1. Disallow Outbound)" . filter actions.Appendix B: IPsec Policy Summary 231 Isolation Domain Policy This section provides the details of the filters. ESP-3DES/SHA1 then ESP-3DES/MD5 DO NOT Accept unsecured communications Allow unsecured communication with non-IPsec-aware hosts Authentication: Kerberos Tunnel: No Connection Type: ALL All other policy settings are the same as listed in the "General Policy Configuration" section earlier in this appendix. but it can communicate with non-IPsec-aware clients if it initiates the communication. all traffic. filter actions. Rule 11 Filter List: IPSEC – Organizational Subnets Filter: Any <-> internal subnets. This rule is the most general rule defined in the policy. ESP-null/MD5. all traffic. and Group Policy objects (GPO) used to create the Isolation Domain in the solution for Woodgrove Bank. It will not accept unsecured communication from non-IPsec-aware clients.

ESP-3DES/SHA1 then ESP-3DES/MD5 DO NOT Accept unsecured communications DO NOT Allow unsecured communication with non-IPsec-aware hosts Authentication: Kerberos Tunnel: No Connection Type: ALL All other policy settings are the same as listed in the "General Policy Configuration" section earlier in this appendix. Consequently. filter actions. and GPOs used to create the Boundary isolation group in the solution for Woodgrove Bank. Rule Behavior Explained Rule 11. The boundary host is assumed not to be mobile and therefore can use subnets to define its network and should be secured almost as a Bastion Host in the Windows Server 2003 security guide.232 Server and Domain Isolation Using Ipsec and Group Policy Security Method Preference Order: ESP-null/SHA1. policy. Boundary Isolation Group Policy This section provides the details of the filters. the IPsec policy should be merged with filters that reduce the attack surface where possible. mirrored Filter Action: "IPSEC – Request Mode (Accept Inbound. all traffic. ESP-null/MD5. This rule is the most general rule defined in the policy. It matches traffic destined to secure subnets and requires that IPsec be negotiated. It must be highly protected against untrusted attack. ESP-null/MD5. Policy General Settings: IKE Main Mode Lifetime: 20 Minutes Rule 11 Filter List: IPSEC – Organizational Subnets Filter: Any <-> internal subnets. Allow Outbound)" Security Method Preference Order: ESP-null/SHA1. ESP-3DES/SHA1 then ESP-3DES/MD5 Accept unsecured communications Allow unsecured communication with non-IPsec-aware hosts Authentication: Kerberos Tunnel: No Connection Type: ALL . It does not allow any communication with non-IPsec-aware clients.

It matches traffic destined to secure subnets and requests that IPsec be negotiated.Appendix B: IPsec Policy Summary 233 All other policy settings are the same as listed in the "General Policy Configuration" section earlier in this appendix. It will accept traffic from non-IPsec-aware clients as well as initiate communication to said clients. This rule is the most general rule defined in the policy. Rule Behavior Explained Rule 11. mirrored Filter Action: "IPSEC – Require Encryption Mode (Ignore Inbound. then ESP-3DES/MD5 DO NOT Accept unsecured communications DO NOT Allow unsecured communication with non-IPsec-aware hosts Authentication: Kerberos Tunnel: No Connection Type: ALL All other policy settings are the same as listed in the "General Policy Configuration" section earlier in this appendix. and GPOs used to create the Encryption isolation group in the solution for Woodgrove Bank. filter actions. It matches traffic destined to secure subnets and requires that Encrypted IPsec be negotiated. This rule is the most general rule defined in the policy. Encryption Isolation Group Policy This section provides the details of the filters. It does not allow any communication with non-IPsec-aware clients. policy. Disallow Outbound)" Security Method Preference Order: ESP-3DES/SHA1. Rule 11 Filter List: IPSEC – Organizational Subnets Filter: Any <-> internal subnets. all traffic. Rule Behavior Explained Rule 11. .

manipulating users. including Active Directory structure and tools. which is described earlier in this guide. and configuration of IPsec policy. . Windows system security. If you have not already done so. Such people might include the following: • • Business sponsors. This appendix is intended to be used in conjunction with the other chapters in this guide. and the application of security templates using Group Policy or command line tools. You should also read and understand the implications of the support requirements detailed in Chapter 6. it is strongly recommended that you read the previous chapters before continuing with this appendix. Security and audit personnel. the use of security templates. and network architectures. This guidance discusses installation and configuration of Microsoft® Windows Server™ 2003. groups. Active Directory concepts. groups. and other Active Directory objects.Appendix C: Lab Build Guide This appendix provides complete guidance for building the required infrastructure to support isolation groups that use IPsec. networking. including security concepts such as users. "Managing a Server and Domain Isolation Environment. and use of Group Policy. Before proceeding with this appendix. and access control lists (ACL). auditing. you should also have read the planning guidance provided in this guide and have a thorough understanding of the architecture and design of the solution. This appendix also provides the implementation instructions that were used to roll out the baseline IPsec policy for the Woodgrove Bank scenario. which explain the design process and rationale behind the implementation decisions that are used in this appendix. preparation of the Active Directory® directory service. Familiarity with Windows Server 2003 is also required in the following areas: • • • Installation of the operating system. This appendix also explains the tasks and processes that are needed to successfully create and implement a baseline IPsec policy infrastructure." before implementing the guidance in this appendix. Organizational Prerequisites You should consult with other people in your organization that may need to be involved in the implementation of this solution. Knowledge Prerequisites You should be familiar with concepts of IPsec. Prerequisites This section contains information that will help you determine your organization's readiness to implement the solution.

or modify IPsec policies. view. To use it. installation media. administration. it was not used in this solution. see New features for IPsec. This tool is an add-on Group Policy management tool that simplifies the administration of Group Policy across the enterprise. Baseline Implementation Prerequisites Before the tasks in this appendix are performed. These tools are: • Netsh. Domain Name System (DNS). There is nothing specific to Windows Server 2003 that would prevent this solution from working properly with Windows 2000. Although it is a Microsoft Management Console (MMC) snap-in. this solution was only tested using Windows Server 2003 Active Directory. • • Server hardware that is adequate to run Windows Server 2003. it does not appear in the default listing of Administrative Tools on the computer. Web server." of this guide. and operations personnel. ensure that the current infrastructure is physically capable of supporting the overhead of the IPsec implementation. However. Note Windows Server 2003 introduced a number of improvements that affect IPsec policies. IT Infrastructure Prerequisites The appendix also assumes that the following IT infrastructure exists: • A Windows Server 2003 Active Directory domain running in mixed or native mode. there are a number of items that should be in place to ensure a successful deployment. • • . filter actions and filter lists. For more information about the enhancements made to IPsec in Windows Server 2003. "Determining the Current State of Your IT Infrastructure. because this option is more complex to manage. However. The process that will help you verify this capability is discussed in Chapter 3. Windows Server 2003 Standard Edition and Enterprise Edition licenses. Download the Group Policy Management Console with Service Pack 1 tool. This solution uses universal groups for Group Policy object (GPO) application. run mmc.• • Active Directory engineering. Group Policy Management Console (GPMC). and product keys. This solution uses Netsh scripts to configure the domain policies. It is used to configure both local policy on a Windows Server 2003 system and domain policy. Hardware Requirements Before the baseline IPsec infrastructure is rolled out. and network engineering administration and operations personnel. it is still possible to apply the GPO through the use of standard global and local group configurations. This tool allows the administrator to create. IP Security Policy Management Console. Tools Four primary tools can be used to configure the IPsec policies and enable them through Active Directory GPOs. If the organization does not run in mixed or native mode. This command-line tool is provided with Windows Server 2003. Note The structure of your IT organization will determine whether these roles may be filled by a number of people or whether fewer people span several roles.exe at a command prompt and add the snap-in manually.

By first deploying a policy without any secure subnets. 3. the team moved on to implement the Standard.exe program. the following steps were performed: To copy configuration scripts 1. Copying Configuration Scripts To set up the IPsec policies. . Copy the script files from this solution's Tools and Templates folder to the C:\IPsec Scripts folder. this tool does not appear by default in the Administrative Tools menu but must be loaded manually through the mmc. and Encryption isolation groups. The policy mechanism that is available in Active Directory can greatly simplify this process. In the Woodgrove Bank scenario. Like the IP Security Management Console. "Designing and Planning Isolation Groups. 2.Appendix C: Lab Build Guide 237 • IP Security Monitor Management Console. After the computers were operating under the Boundary Isolation Group Policy. It is recommended that these tools be obtained and installed on the implementation team workstations so that team members can spend some time to familiarize themselves with the functionality of each tool before implementation begins. Create the folder C:\IPsec Scripts. Implementing the IPsec Policies The process of getting the correct IPsec policy to each intended computer in a large organization can quickly become complex. The configuration scripts provided with the solution were used to configure the Woodgrove Bank lab. the administration team was able to identify any computers that had a local IPsec policy assigned and consider that information. A set of computers were selected for a pilot and added to the appropriate groups that controlled the new policies. the first task is to copy the required configuration scripts to the domain controller that will be used to store them. any additional conflicts that were found were resolved. As subnets were added to the policy. The following sections in this appendix provide the information required to implement the IPsec policies. Any issues were resolved and additional computers were added to the groups until the isolation groups were fully populated. This approach allowed administrators to move forward slowly and resolve any outstanding issues without significant impact on the communication between computers. This tool allows an administrator to view the various rules applied to a computer in addition to main mode and quick mode security associations (SAs) that have been associated with it." of this guide. Outbound Clear Allowed. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. These isolation groups were deployed by using the "deployment by group" method that is explained in Chapter 4. Deployment of the Baseline Policy Woodgrove Bank chose to implement their deployment by first moving all computers into the Boundary isolation group by using the build-up method.

If you change the folder name. Netsh scripts were used to implement the IPsec filter lists and filter actions. Note Test any scripts against the local policy stores on a computer that runs Windows Server 2003 by setting the store focus on local. double-click the Gpmc. In this solution. its installation on subsequent servers is optional.msi installation file. In addition. it does not matter which drive the folder is on. ensure that the MMC snap-in is focused on the local computer rather than the domain. Launch the IP Security Policy Management MMC snap-in and confirm that the filter lists and filter actions have been created in Active Directory. To create the IPsec filter lists and filter actions 1.msi installation file from the Microsoft Download Center. After the scripts are debugged. Follow the setup wizard prompts to install the GPMC. the scripts can be easily ported across domains or forests. accept all defaults. . Note Installation of the GPMC slightly changes the user interface of the Active Directory Users and Computers MMC for the computer on which it is installed. modify the store configuration to focus on the domain for final import. many administrators find it easier to maintain and update scripts that use the Netsh command-line tool. you must update its name in the Constants. ensure that the script being run in step 2 is configured with set store location=local . Although the IP Security Policy Management MMC snap-in provides a graphical interface for IPsec.238 Server and Domain Isolation Using IPsec and Group Policy Installing the Group Policy Management Console The GPMC is used to install and configure the GPOs that are used by the solution.txt" Note If any empty filter lists are created through the script. 4. The GPMC only needs to be installed on IPS-CH-DC-01. and then press ENTER: netsh –f "c:\IPsec Scripts\PacketFilters. Log on to the IPS-CH-DC-01 domain as a domain administrator of the Americas domain. 2. 2. Ensure that you are logged on as a member of the domain Administrators group on IPS-CH-DC-01. To install the Group Policy Management Console 1. 3. From Windows Explorer. Important: You should install GPMC in the Program Files folder. Implementing IPsec Filter Lists and Filter Actions Creation of the IPsec filter lists and filter actions is accomplished by using either the Netsh tool or the IP Security Policy Management MMC snap-in. Later procedures use some of the tools installed by GPMC. type the following. the following error message will display at the command line: ERR IPsec [05022]: No filters in FilterList with name "<Filter List Name>.txt file. Open a command prompt. In step 3. For more information about using the GPMC. You should also use the default installation folder (GPMC) within the Program Files folder. Note To test against local policy. This message can be ignored safely. Download the Gpmc. and if you install it elsewhere they will be unable to locate the GPMC tools unless this file is updated. and to download the installation file. 3. see the Group Policy Management Console with Service Pack 1 page on the Microsoft Download Center.

1 IPsec Policy to Script File Mapping IPsec policy name IPSEC – Boundary Isolation Group IPsec Policy (1. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain.txt 3. Until the policies are linked within Active Directory. The following table lists each GPO name and the IPsec policy name being delivered by that GPO. Script file name BoundaryIGPolicy. these GPOs will not deliver any IPsec policies to the environment.0.0.1600) To create the IPsec policies 1. the scripts that create the IPsec policies can be run.txt NoFallbackIGPolicy. This script file name will be used in step 2 of the following procedure. ensure that the script being run in step 2 is configured with set store location=local .1600) IPSEC – No Fallback Isolation Group IPsec Policy (1. In step 3.0.1600) IPSEC – Isolation Domain IPsec Policy (1. The following table lists the policy name and the script file that creates the policy.2 Woodgrove Bank GPO to IPsec Mapping GPO name IPSEC – Boundary Isolation Group Policy IPSEC – No Fallback Isolation Group Policy IPsec policy name IPSEC – Boundary Isolation Group IPsec Policy (1. Launch the IP Security Policy Management MMC snap-in and confirm that the IPsec policies have been created in Active Directory. Netsh will display an error beginning with "ERR IPsec [05022] …" This message can be ignored safely.0. type the following. 2.Appendix C: Lab Build Guide 239 Implementing IPsec Policies After the filter lists and filter actions have been created.041001.0.1600) IPSEC – No Fallback Isolation Group IPsec Policy (1.041001.041001. Note To test against local policy. Table C.0. and then press ENTER: netsh –f "c:\IPsec Scripts\<Script Filename>" Note If a filter list is empty. ensure that the MMC snap-in is focused on the local computer rather than the domain.1600) IPSEC – Encryption Isolation Group IPsec Policy (1. Each of these GPOs was named after the IPsec policy to which it is assigned within the GPO. For each policy.041001.041001. Table C. Note The policies created by the scripts are configured with a polling interval of five minutes for testing purposes.txt EncryptionIGPolicy.041001.1600) .txt IsolationDomainPolicy. Creating GPOs for IPsec Policies Woodgrove Bank created four GPOs to deliver IPsec policies. Open a command prompt.

240 Server and Domain Isolation Using IPsec and Group Policy GPO name IPSEC – Isolation Domain Policy IPSEC – Encryption Isolation Group Policy IPsec policy name IPSEC – Isolation Domain IPsec Policy (1. Expand Computer Configuration. A universal group that consists of computer accounts that are allowed to communicate with untrusted computers.0. 7. Launch the GPMC. a staged roll out was implemented without moving any computer accounts to special OUs. Repeat steps 4-9 for each <GPO name> and <IPsec policy name> combination from the previous table. Table C. right-click <IPsec policy name>.0.041001. expand Windows Settings. The drawback is that the organization must have good group management tools. Furthermore.com).1600) IPSEC – Encryption Isolation Group IPsec Policy (1. 5. type <GPO name> and then click OK. The primary benefit was that the policies could be linked at the domain level rather than through multiple organizational units (OUs).1600) To create the GPOs for IPsec policies 1.com. 10. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. which simplified the management of policy application. 9.corp. 6. the computer accounts that participated in the pilot were added to the appropriate groups. and then expand americas. expand Security Settings. Setting the Security on the IPsec Group Policies Woodgrove Bank used security ACLs on the GPO that contains the IPsec policies to control the application of the policies. and then close the GPO editor. Expand Forest: corp. Right-click Group Policy Objects. and then click New.3 Woodgrove Bank Universal Groups GPO name CG_NoIPsec_computers IPsec policy name A universal group that consists of computer accounts that do not participate in the IPsec environment— typically infrastructure computer accounts. expand the domain. 8.woodgrovebank. universal groups were used to control policy across all domains. Right-click <GPO name>. and then click IP Security Policies on Active Directory (corp. CG_BoundaryIG_computers .041001.woodgrovebank. and then click Edit. and then click Assign. 2. 4.woodgrovebank. Because the Woodgrove Bank forest was in Native mode.com. Instead. In the right pane. Ensure that <IPsec policy name> is assigned. In the Name text box. Creating Groups A set of groups was created to control how policy was applied throughout the Woodgrove Bank organization. 3.

the delegated administrative group will need to be granted Full Control on the IP Security container in Active Directory. Note If an organization is going to delegate administrative rights to someone other than the Domain Admins group to manage IPsec policies. 2. and then click Properties. To create the Woodgrove Bank universal groups 1. A universal group that consists of computer accounts that are part of the No Fallback isolation group.4 Woodgrove Bank Policy Group Permissions GPO name IPSEC . type the first <Description> from the previous table. click New. The following table shows the ACLs to be added to each GPO. 6.Boundary Isolation Group Policy Group or account name CG_NoIPsec_computers CG_BoundaryIG_computers IPSEC – No Fallback Isolation Group Policy CG_NoIPsec_computers CG_NoFallbackIG_computers IPSEC – Isolation Domain Policy CG_NoIPsec_computers CG_ IsolationDomain_computers Rights assigned Deny Apply Group Policy Allow Read and Apply Group Policy Deny Apply Group Policy Allow Read and Apply Group Policy Deny Apply Group Policy Allow Read and Apply Group Policy . Launch Active Directory Users and Computers on IPS-CH-DC-01. 3. Click OK. type the first <Group name> from the previous table. Configuring GPO Security Groups are used to control which computers get what policies for IPsec participation. In the Group Name text box. Repeat steps 2-4 for each group. In the Description text box. Repeat steps 6-8 for each of the groups listed in the previous table. Table C. Select Universal security group. The security ACLs need to be configured on each of the newly-created IPsec policies so that the appropriate groups are configured. and then click OK. 9. A universal group that consists of computer accounts that are part of the Isolation Domain. Right-click the Users container. and then click Group. 8. 5.Appendix C: Lab Build Guide 241 GPO name CG_ EncryptionIG_computers CG_ IsolationDomain_computers CG_NoFallbackIG_computers IPsec policy name A universal group that consists of computer accounts that are in the Encryption isolation group. 4. Right-click the first <Group name>. 7.

Launch the GPMC on IPS-CH-DC-01 as a domain administrator of the Americas domain. 4. and then click the Delegation tab. 2. select <Group or account name>.corp. domain computers will be removed from the CG_BoundaryIG_computers group. expand the domain. In the Enter the object names to select text box. 10. and then expand Group Policy Objects. 11. If Apply permissions are also granted. expand americas. To implement this restriction.242 Server and Domain Isolation Using IPsec and Group Policy GPO name IPSEC – Encryption Isolation Group Policy Group or account name CG_NoIPsec_computers CG_ EncryptionIG_computers Rights assigned Deny Apply Group Policy Allow Read and Apply Group Policy Note The Boundary Isolation Group Policy is configured to allow the Domain Computers group to apply the policy for the initial build-up process by placing the Domain Computers group in the CG_BoundaryIG_computers group.woodgrovebank. and then click OK. and the CG_BoundaryIG_computers group was placed in the DNAG_EncryptionIG_computers . Note Ensure that the entry for Authenticated Users was granted only Read permissions in the security ACL for each policy. and then set the <Rights assigned> in the Permissions check boxes. If the right being assigned is a Deny right. Click OK. To set the group permissions on the GPO 1. 3. click Authenticated Users. 7. Expand Forest: corp. Click the first <GPO name> from the previous table. The Encryption Isolation Group Policy was configured so that DNAG_EncryptionIG_computers was granted the "Deny access to this computer from the network" right. otherwise. a group called DNAG_EncryptionIG_computers is created to deny its members access to computers in the Encryption isolation group. Blocking Boundary Isolation Group Computers from Initiating Connections to Encryption Isolation Group Computers Woodgrove Bank required that computers in the Boundary isolation group be prevented from initiating communications with computers in the Encryption isolation group. 8. Repeat step 3-11 for each <Policy name>. enter each <Group or account name> from the previous table. 9.woodgrovebank. the policy will be deployed to all computers. In the Group or user name scroll box. click Yes when the message box is shown. After all computers are moved to their respective groups. 5. 12. In the Group or user names text box. Click the Advanced button. separated by semicolons. 6.com. Repeat step 8 for each <Group or account name> associated with the <Policy name>. proceed to step 12. and clear the Allow right Apply Group Policy check box.com. Click the Add button.

Click OK again. 9. . expand the domain. To populate the DNAG_EncryptionIG_computers group with the CG_BoundaryIG_computers group 1. and then expand User Rights Assignment. and then click Group. 2. Select the Domain local security group. Close the GPMC. expand americas. In the right pane. Right-click IPSEC – Encryption Isolation Group Policy and then click Edit.com. and then click Add. In the text field. Launch the GPMC on IPS-CH-DC-01 as a domain administrator of the Americas domain. Launch Active Directory Users and Computers on IPS-CH-DC-01. 2. Expand Forest: corp. 13. In the Description text box. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. and then expand Group Policy Objects. 6. 5. 6. To create the DNAG_EncryptionIG_computers group 1. expand Security Settings. 3. right-click the DNAG_EncryptionIG_computers security group. type DNAG_EncryptionIG_computers 4. click New. and then click Users. 5. Click the Add User or Group button. 3. This configuration was accomplished by modifying the IPSEC – Encryption Isolation Group Policy GPO. Click OK to close the Properties page.Appendix C: Lab Build Guide 243 group. expand Local Policies. Right-click the Users container. 4. 12.com. and then click Properties. Right-click DNAG_EncryptionIG_computers. 7. and then click Properties. 2. Click OK. type Used to Deny Access to Encryption Isolation Group 7. type CG_BoundaryIG_computers and then click OK. In the Enter the object names to select text box. 8. In the Group name text box. 3. 11. Expand Computer Configuration. Close the Group Policy Editor. Click the Browse button. Click OK. Expand the domain. 5. 6.woodgrovebank. 4. and then click OK.corp. Select the Define these policy settings check box. type DNAG_EncryptionIG_computers and then click OK. Right-click Deny access to this computer from the network. To configure IPSEC – Encryption Isolation Group Policy to block members of DNAG_EncryptionIG_computers 1. Click the Members tab. expand Windows Settings. and then click Properties. 10.woodgrovebank.

5. In the Enter the object names to select text box. In the right pane. and then click Properties. there will be a delay between the time the Domain Computers group is added to the CG_BoundaryIG_computers group and when the Boundary Isolation Group Policy is applied. and then click Properties. Expand the domain. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. Click OK again. 3. • • IPS-RT-DC-01 IPS-CH-DC-01 To add infrastructure servers to the CG_NoIPsec_computers group 1. Click OK again. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. right-click the CG_NoIPsec_computers security group. Note Because of replication delays and polling frequency of the IPsec policies. and then click OK. 2. 2. The Domain Computers group is added to the CG_BoundaryIG_computers group to implement this plan. 5. the following infrastructure server computer accounts were added to the CG_NoIPsec_computers security group. 6. 7. To add domain computers to the CG_BoundaryIG_computers Group 1. and then click Add. and then click Users. 4.244 Server and Domain Isolation Using IPsec and Group Policy Adding Domain Computers to the Boundary Group For the initial deployment. In the right pane. separated with semicolons. In the Enter the object names to select text box. Expand the domain. if a server's IP address changes). right-click the CG_BoundaryIG_computers security group. Adding Infrastructure Servers to the CG_NoIPSec_Computers Group To ensure that the infrastructure servers do not receive a policy that could interrupt communication (for example. Otherwise. Click the Members tab. . The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately. and then click OK. 6. 3. Click the Members tab. and then click Users. select the Computers check box. and then click Add. the Boundary isolation group is used as the default isolation group for the IPsec-aware clients in the enterprise. type Domain Computers and then click OK. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. Click the Object Types button. type the name of each computer in the preceding list. 4.

In the right pane.5 Link Order of Group Policy Objects at the Domain Level Link order 1 2 3 4 5 Group Policy object name IPSEC – Encryption Isolation Group Policy IPSEC – No Fallback Isolation Group Policy IPSEC – Isolation Domain Policy IPSEC – Boundary isolation Group Policy Default Domain Policy Using the Policy Build-up Method to Enable the Baseline IPsec Policy The first task in the rollout of the IPsec infrastructure is the deployment of the Boundary Isolation Group Policy by using the policy build-up deployment method. if there are OUs that block policy application. it is configured to apply to all computers for the first stage of the deployment. they need to be linked to locations within the domain environment. The policy was initially deployed with no secure subnets defined. This allowed the Woodgrove Bank administrators to fix any existing local IPsec policies. the IPsec GPOs would have to be linked directly to the OUs for the policy application to work. select all the IPSEC-named policies. it was deemed the safest policy to deploy gradually into the environment. To link the IPsec policies to the existing GPOs 1. 4. use the arrow keys to order the policies as shown in the following table. Expand the domain. .Appendix C: Lab Build Guide 245 Linking IPsec Policies and GPOs in a Domain Environment Before IPsec policies can be distributed. Although the Boundary isolation group is not intended to be the Isolation Domain for all computers in the Woodgrove Bank environment. the OU structure is not overly important to policy distribution. Another alternative might be to enable policy enforcement on the domain IPsec policy GPOs. 2. 3. Launch the GPMC as a domain administrator. However. and then click OK. Table C. Because Woodgrove Bank chose to administer the GPOs through the use of security groups. and then click Link an Existing GPO. subnets are added one by one and tested to ensure that IPsec negotiation occurred correctly. Next. 5. In the Group Policy objects list. Because the Boundary Isolation Group Policy allows and accepts non-IPsec communication. Right-click the domain name.

13. 4.255.0/24 255. Ensure that the Mirrored option is selected. Launch the IP Security Policy Management MMC snap-in. 7. Launch the IP Security Policy Management MMC snap-in. 3.246 Server and Domain Isolation Using IPsec and Group Policy Adding Subnets to the Secure Subnets Filter List After the empty Boundary Isolation Group Policy was applied to the computers in the organization and any conflicts with existing local IPsec policies were resolved.255. 9. 2.0 Organizational LAN subnet 172. Right-click IP Security Policies on Active Directory. type the corresponding description from the previous table.0 Organizational LAN subnet 192.168. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. 5.168. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. 6. On the Description tab.0 Netmask Description 192. and then click Edit. the policy was given time to apply to the computers in the organization and any conflicts were resolved. 12. Click Add.0/24 To create the first entry in the secure subnets filter list 1. Ensure that the Use Add Wizard check box is cleared. On the Addresses tab. The build-up of the policy consisted of identifying the organizational subnets to be secured.1600) . Click OK to close the IP Filters Properties dialog box.0. 10. 11. After adding the first entry to the filter list.255. Woodgrove Bank administrators began the build-up of the policy. In the Destination Address drop-down list. The identified subnets were added to the policy one by one.255.0 255. 3.10. This process was repeated until the entire secure subnets filter list was deployed. On the Manage IP Filter Lists tab.1. To add the secure subnet filter list to the Boundary Isolation Group Policy 1. Click OK to close the IP Filter List dialog box.041001. The following table lists the identified secure subnets used in the lab at Woodgrove Bank to closely mirror their production network: Table C. .1. the filter list is added to the policy. Click Close to close the Manage IP filter lists and filter actions dialog box. click Any IP Address. and then click Manage IP filter lists and filter actions. click IPSEC – Organization Secure Subnets. and then click Properties.1. 8. 2. and then fill out the IP address and subnet mask boxes using the information in the previous table. click A Specific IP Subnet. Right-click IPSEC – Boundary Isolation Group IPsec Policy (1.6 Secure Subnets List for Woodgrove Bank Test Lab Subnet 172. After each subnet was added.10. in the Source Address drop-down list.1.

14. 5. Verifying the Baseline Deployment After the policy objects are created and deployed into Active Directory in an inactive state. Allow Outbound). On the Tunnel Settings tab. Launch the IP Security Policy Management MMC snap-in. On the Connection Type tab. 12. and then fill out the IP address and subnet mask using the information in the previous table. Verification can help minimize any potential disruption to the participating hosts if there is an error in the baseline configuration. Ensure that the Use Add Wizard check box is cleared. 8. Ensure that the Mirrored option is selected. a process of verification should be undertaken before configuring the baseline policy to enforce the Baseline isolation group for all computers in the organization. 10. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. and then click Edit. Click Add. ensure that the Kerberos method is the only method that is listed. Click OK to close the IPSEC – Boundary isolation group IPsec Policy (1. 7. 15. 8. Click OK to close the Edit Rule Properties dialog box. 6. On the Description tab. On the Authentication Methods tab.0. On the Addresses tab. 9. On the Filter Action tab. click IPSEC – Request Mode (Accept Inbound. 12. click IPSEC – Organization Secure Networks. Click OK to close the IP Filter List dialog box.1600) Properties dialog box. 9. On the IP Filter List tab. In the Destination Address drop-down list.Appendix C: Lab Build Guide 247 4. 10. click Any IP Address. ensure that the This rule does not specify an IPsec tunnel check box is selected. 7. Allow the policy to apply and then run the verification steps listed in the "Verifying the Baseline Deployment" section later in this appendix. Click OK to close the IP Filters Properties dialog box.041001. type the corresponding description from the previous table. 3. and then click Manage IP filter lists and filter actions. ensure that the Use Add Wizard check box is not selected. 11. . in the Source Address drop-down list. 2. On the Manage IP Filter Lists tab. 4. On the Rules tab. ensure that the All network connections check box is selected. Repeat steps 2-14 for each subnet. 5. 13. Click Close to close the Manage IP filter lists and filter actions dialog box. click A Specific IP Subnet. and click Add. click IPSEC – Organization Secure Subnets. 11. 6. Allow the policy to apply and then run verification steps listed in the "Verifying the Baseline Deployment" section later in this appendix. Right-click IP Security Policies on Active Directory. To add the remaining subnets to the secure subnets filter list 1.

For each computer that a net view command was initiated against. Non-participants negotiated a soft SA. To test functionality of the IPsec policies that are applied 1. open a command prompt.248 Server and Domain Isolation Using IPsec and Group Policy Functional Implementation Tests The simplest test that can be performed to confirm IPsec functionality is to attempt to execute net view commands against computers that are in the secure organization network and against computers that are not in subnets listed in the secure organization network. whereas other situations require a command-line tool that may or may not be installed with the operating system. The ESPIntegrity column should not be set to <None>. use the names of both other secured subnet computers and computers that are not in secure subnets. Verifying IPsec Policy Application Determining which IPsec policy is active on a computer is a challenge because there is no consistent method that works across platforms. and press ENTER. expand Quick Mode. type net view \\<computer name>. A soft SA should be created between an IPsec participant and a computer that is not in a subnet listed in the secure organization network. Launch the IP Security Monitor MMC snap-in on the computer that initiated the net view commands. The ESP-Integrity column should be set to <None>. For <computer name>. These tasks involve identifying the IPsec policy that is currently active on the computer and determining what type of SA was negotiated. Although most of these settings can be monitored using standard tools. 2. 4. Computers that are in a secure subnet should negotiate a hard SA that will be visible within the IP Security Monitor MMC snap-in. expand <computer name>. Test Tools and Scripts for the Functionality Tests A number of configuration settings must be monitored during the functionality tests. From a secured subnet computer. In some cases you can identify the IPsec policy through the graphical user interface (GUI). . and then click Security Associations. Expand IP Security Monitor. confirm the following: • • Secure organization network participants negotiated a hard SA. 3. two tasks require tools with which a standard administrator might not be familiar.

the administrator logs on to the computer.DC=com Policy Path: LDAP://CN=IPsecPolicy{efa2185d-1a1d-40f6b977-314f152643ca}.exe command-line tool. the administrator logs on to the computer. the administrator can identify the currently applied IPsec policy by using the IPseccmd. To retrieve the policy name and information.0. To retrieve the policy name and information. . the administrator can identify the currently applied IPsec policy by using the Netdiag command. the administrator can identify the currently applied IPsec policy by using the Netsh command-line tool. .CN=IP Security.0. . To retrieve the policy name and information.041001. . the administrator logs on to the computer.DC=com Windows Server 2003 For computers that run Windows Server 2003. . .Appendix C: Lab Build Guide 249 Windows 2000 For computers running Windows 2000 Server. .1600)' Windows XP For computers running Windows XP.041001. launches a command prompt and types the following: Netdiag /test:IPsec The following is example output from this command: IP Security test .1600) Description: Isolation Domain Policy (Allow Outbound) Last Change: Fri Sep 03 15:20:29 2004 Group Policy Object: IPSEC – Isolation Domain Policy Organizational Unit: LDAP://DC=americas.CN=System. : Passed Directory IPsec Policy Active: ' IPSEC – Isolation Domain IPsec Policy (1. launches a command prompt and types the following: netsh IPsec static show gpoassignedpolicy .DC=americas. launches a command prompt and types the following: IPseccmd show gpo The following is example output from this command: Active Directory Policy ----------------------Directory Policy Name: IPSEC – Isolation Domain IPsec Policy (1.DC=woodgrovebank.DC=woodgrovebank. .

but AD Policy is Overriding Using IP Security Monitor to Determine SA Type The IP Security Monitor MMC snap-in is used to examine the main mode and quick mode SAs. Enabling Organization Secure Subnets Filter List on Remaining Policies Before you enable the IPsec policies that remain. Allow Outbound) . ESP Confidential.DC=woodgrovebank.0. Disallow Outbound) IPSEC – Secure Request Mode (Ignore Inbound. ESP with SHA1 and no authentication would have HMAC-SHA1 under the ESP Integrity field. A soft SA will have <None> under all three fields. and <None> for the other two fields. During troubleshooting. or ESP Integrity fields.0. a system administrator can identify IPsec peers to the computer on which the tool is running. Earlier in this appendix. For example. the Secure Organization Network filter list was empty and could not be added to the policy. the associated filters. the Secure Organization Network filter list needs to be added to each policy. This task is required because at the time of the policy creation.1600) AD Policy DN : LDAP://CN=IPsecPolicy{efa2185d-1a1d-40f6b977-314f152643ca}.7 Policy and Filter Actions Mapping Policy name IPSEC – No Fallback Isolation Group IPsec Policy (1. indicating that the responder fell back to clear. the Secure Organization Network filter list was implemented and can now be added to the remaining policies. The following table shows the policy names and the associated filter action assigned to the Secure Organization Network filter list.1600) IPSEC – Isolation Domain IPsec Policy (1.250 Server and Domain Isolation Using IPsec and Group Policy The following is example output from this command: Source Machine : Local Computer GPO for <IPS-TZ-W2K-02> GPO Name : IPSEC – Isolation Domain Policy Local IPsec Policy Name : NONE AD IPsec Policy Name : IPSEC – Isolation Domain IPsec Policy (1.1600) Filter action IPSEC – Full Require Mode (Ignore Inbound. By examining the SAs under the Quick Mode tree. the IP Security Monitor MMC snap-in can be used to determine what type of SA has been negotiated between peers.DC=americas. the ESP Confidential field would contain either DES or 3DES.041001. a hard SA is created. Internet Key Exchange (IKE) policies. If the hard SA also has negotiated encryption. This SA will have some value other than <None> in one or more of the Authentication. When a computer negotiates an IPsec connection. and negotiation policies.DC=com Local IPsec Policy Assigned: Yes.0. Table C.041001.041001.CN=IP Security.CN=System.

ensure that the This rule does not specify an IPsec tunnel check box is selected. If global or universal groups had been used. On the Connection Type tab.Appendix C: Lab Build Guide 251 Policy name IPSEC – Encryption Isolation Group IPsec Policy (1. Note Care must be taken when you define this option because computers that need to initiate communication with computers in the network access group (for example. On the Tunnel Settings tab. monitor systems that use polling) will fail if they are not included in the network access group. which has a lifetime of 8 hours. click IPSEC –Organization Secure Subnets. 11. On the Filter Action tab. . Click OK to close the <Policy Name> Properties dialog box. 9. 5. 7. 3. which refreshes every 60 minutes. and then click Properties. group changes take effect on a much timelier basis. ensure that the Kerberos method is the only method that is listed. 4. click Add. Right-click <Policy Name>. Repeat steps 3-11 for each policy listed in the previous table. preshared keys or certificates could be used to implement individual network access groups. ensure that the All network connections check box is selected. By using domain local groups. These groups were then used to define the initiators. administrators can configure the executive client computers so that they only accept incoming traffic initiated from executive computers but still maintain their ability to initiate traffic to other resources. click the corresponding <Filter Action> from table C. Note Although this solution uses domain local groups with the "Access this computer from the network" right to implement the network access group. Click OK to close the Edit Rule Properties dialog box. For example. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. Woodgrove Bank implemented the network access group by using domain local groups because these groups are stored in the session ticket.7.0. 2. by using network access groups. Launch the IP Security Policy Management MMC snap-in. 12. On the Rules tab. 10. They granted the initiators group the "Access this computer from the network" right on the responders. the network access group would have been stored in the ticket granting ticket (TGT). On the IP Filter List tab. Disallow Outbound) To add the Secure Organization Network filter list to IPsec policies 1. and removed the Authenticated Users group from the right. 6. 8. Implementing Network Access Groups The designers at Woodgrove Bank chose to implement network access groups through the use of domain local groups. On the Authentication Methods tab.1600) Filter action IPSEC – Require Encryption Mode (Ignore Inbound. Enabling Network Access Group Configuration Network access groups are used to further restrict the IPsec responder to only accept connections from a select group of initiator computers and identified users.041001.

Launch Active Directory Users and Computers on IPS-CH-DC-01. and then click Properties. 9. and then click Group. 6. Adding Accounts to Network Access Group Security Groups Woodgrove Bank added the identified computers that act as initiators of traffic within the network access group to the appropriate domain local groups that are used to implement the network access group. 2. 8. Repeat steps 2-4 for each group listed. Creating Security Groups to Control Access Table C. Expand the domain. type the <Group Name> from the previous table. Right-click the Users container. 3. which is used to control access in the Encryption isolation group. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain.9 Woodgrove Bank Isolation Group Membership Group name Members IPS-SQL-DFS-02 IPS-ST-XP-05 To populate the group listed in the previous table 1. In Group Scope select Domain local and then click OK. click New. 5. ANAG _EncryptedResourceAccess_computers IPS-SQL-DFS-01 . Repeat steps 6-8 for each group listed in the previous table. and then click Users. In the Group name text box. Right-click the <Group Name>.8 Woodgrove Bank Network Access Group Security Groups Group name ANAG _EncryptedResourceAccess_computers Description A domain local group that is used to limit which computers can access encrypted resources ANAG _EncryptedResourceAccess_users A domain local group that is used to limit which users can initiate communication with the restricted encrypted resource To create the groups listed in the previous table 1. 2. Click OK. 4. type the <Description> from the previous table. 7.252 Server and Domain Isolation Using IPsec and Group Policy The designers at Woodgrove Bank identified one network group. The following table lists the membership of the network access group that was identified by Woodgrove Bank. Table C. In the Description text box.

5. In the Enter the object names to select text box. and then click Users. 6. 4. 7. the GPO assigned the appropriate network access group security groups the "Access this computer from the network" right on the appropriate computers acting as responders. 6. Click the Object Types button. right-click the <Group Name> security group. . In the right pane. and then click Add. In the right pane. 3.Appendix C: Lab Build Guide 253 3. Adding User Accounts to Network Access Group Security Groups Woodgrove Bank identified the user accounts that are authorized to initiate traffic within the network access group and added them to the appropriate domain local groups used to implement the network access group.10 Woodgrove Bank Network Access Group Membership Group name Members ANAG _EncryptedResourceAccess_users User7 To populate the groups listed in the previous table 1. If there are multiple users. type the name of each user in the Members column of the previous table. select the Computers check box. 4. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. Click the Members tab. In the Enter the object names to select text box. and then click Properties. Click the Members tab. 5. Creating a Group Policy Object to Grant the "Access This Computer from the Network" Right Woodgrove Bank created a GPO to enforce the defined network access group. Click OK. type the name of each computer in the Members column of the previous table and separate each member with a semicolon. Click OK. separate each with a semicolon. and then click OK. Then click OK. and then click Properties. right-click the <Group Name> security group. Expand the domain. Table C. The following table lists the membership of the network access group that was identified by Woodgrove Bank. Click OK. 2. Specifically. and then click Add.

10. Table C. type <GPO name> and then click OK. Click the Add User or Group button. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. Right-click Group Policy Objects. and then click User Rights Assignment. 13. Select the Define these policy settings check box. Expand Computer Configuration. 6. 5. expand Windows Settings.254 Server and Domain Isolation Using IPsec and Group Policy The administrators created the following table. expand Local Policies. In the right pane. Click OK. and then click New. 11.com. Click the Browse button.com. right-click Access this computer from the network and then click Properties. Launch the GPMC. expand Security Settings. 4. In the Enter the object names to select text box. which lists the GPO name and the associated group names used to implement the network access group. 8. Click OK again. Table C. expand the domain. 2. 7. To assign "Access this computer from the network" right 1. 12. The administrator will need to determine if any additional groups should be granted this right. and separate each with a semicolon. 14.woodgrovebank. and then expand americas. the GPOs need to be linked to a location within the domain environment. 9. 3.woodgrovebank. Woodgrove Bank chose to distribute the GPO by linking it to the appropriate OU in Active Directory. In the Name text box. Close the GPMC. as shown in the following table. Linking Network Access Group Policy Objects Before you distribute network access group policies. type the <Group name> for each group listed in the previous table. Right-click <GPO name> and then click Edit.12 Network Access Group GPO Name and Target OU Network access group GPO name Target OU Encrypted Network Access Group Policy Database Servers .corp.11 Woodgrove Bank Isolation Group Policy Definition GPO name Encrypted Resource Access Isolation Group Policy Group name ANAG_EncryptedResourceAccess_computers ANAG_EncryptedResourceAccess_users Administrators Backup Operators Note The listed groups are the minimum that should be added. Expand Forest:corp.

7. 2. Expand Forest: corp. 2. 5. Confirm that the Authenticated Users group is not present. Repeat steps 1-6 for each <Computer Name> listed in the previous table. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. and then click Link an Existing GPO. 4. 6. Woodgrove Bank confirmed that the user rights assignments were being updated appropriately.Appendix C: Lab Build Guide 255 To link a GPO policy to target OU 1. expand the domain. and then click OK. Functional Implementation Tests After Woodgrove Bank confirmed that the security groups were granted the appropriate user right. 3.woodgrovebank.com. Prerequisite Implementation Tests Before it tested the functionality of the computers in the network access group. Confirm that <Group Listed in User Right> group is present. In the Group Policy objects list.corp. in the right pane. Close the Local Security Policy tool. administrators tested the functionality of the computers in the network access groups. expand User Rights Assignment. 3.woodgrovebank. expand americas. and then locate the <Target OU>. double-click Access this computer from the network.13 Network Access Group Membership Computer name Group listed in user right IPS-SQL-DFS-01 ANAG_EncryptedResourceAccess_computers ANAG_EncryptedResourceAccess_users IPS-SQL-DFS-02 ANAG_EncryptedResourceAccess_computers ANAG_EncryptedResourceAccess_users To confirm the correct group membership in the network access group 1. Woodgrove attempted to perform net view . Woodgrove Bank performed the following steps on the computers listed in the following table. click <Network Access Group GPO Name>. 4. Woodgrove Bank used this information to confirm that the access right restrictions were in place and functioning. 5. After sufficient time had passed for replication and policy update to occur. Launch the Local Security Policy tool. Right-click <Target OU>. and then. the computers that belonged to the network access groups were tested against each other.com. Table C. Verifying Deployment of Network Access Groups After creating and deploying the network access groups and policy objects. Log on to <Computer Name> as a domain administrator of the Americas domain. Launch the GPMC. Expand Local Policies.

the Isolation Domain policy should apply to the pilot computers and take effect. Launch a command prompt and then run the following command: net view \\<Responder> 5. 3. 4. expand Quick Mode. Repeat steps 1-5 for each unique <Initiator> listed in the previous table. the administrator must identify a group of computers that will be used for the pilot test. 2. and lists the type of SA negotiated. Implementing the Isolation Domain Woodgrove Bank identified the following computers to be used in the pilot: • • • • IPS-TZ-XP-01 IPS-TZ-W2K-02 IPS-TZ-XP-06 IPS-WEB-DFS-01 . In addition to this test.14 Network Access Group Functional Test Expected Results Initiator IPS-TZ-XP-06 IPS-TZ-XP-06 IPS-TZ-XP-06 Responder Result SA negotiated None None IPS-SQL-DFS-01 Fail IPS-SQL-DFS-02 Fail IPS-ST-XP-05 Succeed Hard SA Succeed Hard SA IPS-SQL-DFS-01 IPS-SQL-DFS-02 Succeed Hard SA IPS-SQL-DFS-01 IPS-ST-XP-05 IPS-ST-XP-05 IPS-ST-XP-05 IPS-SQL-DFS-02 IPS-SQL-DFS-01 Succeed Hard SA IPS-SQL-DFS-01 Succeed Hard SA IPS-SQL-DFS-02 Succeed Hard SA To complete the functional test 1.256 Server and Domain Isolation Using IPsec and Group Policy commands against various initiator and responder combinations. The following table lists the initiator and responder for each execution of net view. expand <Initiator>. and then click Security Associations. this group of computers should represent a cross-section of the organization's IT infrastructure and include both clients and servers. Ideally. they used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. Table C. Launch the IP Security Monitor MMC snap-in. Log on to <Initiator> as a domain administrator in the Americas domain. Enabling the Isolation Domain Before the Isolation Domain policies are rolled out. Expand IP Security Monitor. Use the IP Security Monitor MMC snap-in to confirm that the appropriate SA was negotiated for each successful connection 6. indicates whether it should succeed or fail. After sufficient time has elapsed for replication. The identified computer accounts are added to the CG_IsolationDomain_computers group.

Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. and lists the type of SA negotiated. 2. and then click Users. you must pass credentials for the local administrator of the target computer. In the Enter the object names to select text box.1600). the next step was to perform the some basic functional tests to ensure that the policy was operating as expected. The following table lists the target computers for each execution of net view. and then click OK. indicates whether it should succeed or fail. 6.041001. Note When you attempt a net view command against an untrusted computer. and then click Properties. Confirm that the output shows that the directory policy name is IPSEC – Isolation Domain IPsec Policy (1. To confirm that the correct IPsec policy was applied on IPS-TZ-XP-06 1. Click the Members tab. they used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. 5. Woodgrove Bank confirmed sufficient time had passed for replication and policy update to occur and then that the correct IPsec policy was applied to it. Launch a command prompt and then run the following command: IPseccmd show gpo 3. separate them with semicolons. 3. Note After the computers are added to the CG_IsolationDomain_computers universal group. sufficient time should be allowed for replication of the group membership changes throughout the forest and for the policy to apply to the hosts. Click OK again. 7. 2. select the Computers check box. Expand the domain. In the right pane. right-click the CG_IsolationDomain_computers security group.Appendix C: Lab Build Guide 257 To add pilot computers to the CG_IsolationDomain_computers group 1. a process of verification should be undertaken to confirm that the computer functions properly within the isolation group. Prerequisite Implementation Tests Before it ran any functional tests on the computer in the Isolation Domain. 4. Functional Implementation Tests After Woodgrove Bank confirmed that the policy was applied to IPS-TZ-XP-06. . In addition.0. Verifying Deployment of the Isolation Domain After the policy objects have been created and deployed into Active Directory in the active state. and then click Add. Log on to IPS-TZ-XP-06 as a domain administrator of the Americas domain. Woodgrove Bank attempted to perform net view commands from IPS TZ-XP-06 to various computers in other isolation groups. type the name of each computer in the preceding list. and then click OK. Click the Object Types button.

15 Isolation Domain Expected Functional Test Results Target computer Result IPS-TZ-W2K-02 IPS-UT-XP-03 IPS-PRINTS-01 SA negotiated Succeed Hard SA Succeed Soft SA Succeed Hard SA IPS-WEB-DFS-01 Succeed Hard SA To perform the functional test on each target computer 1. 4. 5. 3. In the right pane. Launch the IP Security Monitor MMC snap-in. 4. and then click Security Associations. and then click Properties. expand IPS TX XP 06. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Implementing the No Fallback Isolation Group Woodgrove Bank placed those computers that cannot initiate unauthenticated communication to untrusted computers in the CG_NoFallbackIG_computers universal group. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. and then click OK. 2. be sure to pass local administrator credentials with the net view command. Repeat steps 3-4 for each <Target Computer> listed in the previous table. and then launch Active Directory Users and Computers. select the Computers check box. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. Launch a command prompt. . 5. and then click Add. Enabling the No Fallback Isolation Group Computers placed in the No Fallback isolation group cannot initiate unauthenticated traffic to untrusted computers.258 Server and Domain Isolation Using IPsec and Group Policy Table C. To populate the CG_NoFallbackIG_computers group 1. 3. right-click the CG_NoFallbackIG_computers security group. expand IP Security Monitor. Click the Members tab. 2. and then click Users. expand Quick Mode. Log on to IPS-TZ-XP-06 as a domain administrator in the Americas domain. Expand the domain. Click the Object Types button.

Note Because of replication delays and polling frequency of the IPsec policies. and after sufficient time had passed for replication and policy update to occur. and then once more.Appendix C: Lab Build Guide 259 6. Woodgrove Bank attempted to perform net view commands from IPS LT-XP-01 to various computers in other isolation groups. Note When you attempt a net view command against an untrusted computer. a process of verification should be undertaken to confirm that the computer functions properly within the isolation group. indicates whether it should succeed or fail. type IPS-LT-XP-01 and then click OK. they used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. In the Enter the object names to select text box. Confirm that the output shows that the directory policy name is Outbound Clear Allowed. In addition to this. Click OK again. Otherwise. Prerequisite Implementation Tests Before it ran any functional tests on the computers in the No Fallback isolation group. you must pass credentials for the local administrator of the target computer. To confirm that the correct IPsec policy was applied on IPS-LT-XP-01 1. Log on to IPS-LT-XP-01 as a domain administrator of the Americas domain. Woodgrove Bank confirmed that the correct IPsec policy was applied. Functional Implementation Tests After Woodgrove Bank confirmed that the policy was applying to IPS-LT-XP-01. 2. there will be a delay between the time the computer is added to the CG_NoFallbackIG_computers group and when the No Fallback Isolation Group Policy is applied.16 Outbound Clear Allowed Expected Functional Test Results Target computer Result IPS-PRINTS-01 IPS-TZ-XP-01 IPS-UT-XP-03 SA negotiated Succeed Hard SA Succeed Hard SA Fail None . the next step was to perform the some basic functional tests to ensure that the policy was operating as expected. Launch a command prompt and then run the following command: IPseccmd show gpo 3. The following table lists the target computers for each execution of net view. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. Verifying Deployment of the No Fallback Isolation Group After the policy objects have been created and deployed into Active Directory in the active state. Table C. and lists the type of SA negotiated. 7. The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately.

In addition. and then click Users. 4. Launch a command prompt. 6. expand Quick Mode. there will be delay between the time the computer is added to the CG_EncryptionIG_computers group and when the Encryption Isolation Group Policy is applied. and then click Add. and then click Properties. 5. In the right pane. In the Enter the object names to select text box. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. Click the Members tab. 2. To populate the Require Encryption group 1. Implementing the Encryption Isolation Group The implementation team at Woodgrove Bank identified those computers that required IPsec encryption and placed them in the Require Encryption universal group. servers that host data are configured to restrict who can access them through the network by implementation of an isolation group for the selected servers. Repeat steps 3-4 for each <Target Computer> listed in the previous table. access to the server can be controlled by modifying the "Access this computer from the network" right. Launch the IP Security Monitor MMC snap-in. The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately. type IPS-SQL-DFS-01. and then click Security Associations. 3. Care should be taken when changing rights on a server to ensure that legitimate users are not blocked from accessing it. By using an additional group policy and a security group. expand IP Security Monitor. Note The isolation group used in this section was implemented earlier in the "Enabling Isolation Group Configuration" section of this document. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. be sure to pass local administrator credentials with the net view command. 5. Log on to IPS-LT-XP-01 as a domain administrator in the Americas domain. and then click OK. Click the Object Types button. Expand the domain. 4. Otherwise. . Note Because of replication delays and polling frequency of the IPsec policies. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. expand IPS LT XP 01. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. right-click the CG_EncryptionIG_computers security group. 2. IPS-SQLDFS-02 and then click OK. 3.260 Server and Domain Isolation Using IPsec and Group Policy To perform the functional test on each target computer 1. select the Computers check box. Enabling the Encryption Isolation Group Computers that are placed in the Encryption isolation group require their traffic to be encrypted. 7. and then launch Active Directory Users and Computers. Click OK.

Appendix C: Lab Build Guide 261 Verifying the Encryption Isolation Group Deployment After the policy objects have been created and deployed into Active Directory in the active state. and lists the type of SA negotiated. 6. The following tables list the target computers for execution of net view. Table C. a process of verification should be undertaken to confirm that the computer functions properly within the isolation group. 9.Encryption Isolation Group IPsec Policy (1. and then run the following command: netsh IPsec static show gpoassignedpolicy 3. To confirm that the correct IPsec policy was applied 1. in the right pane. Expand Local Policies. Repeat steps 1-8 on IPS-SQL-DFS-02. Woodgrove attempted to perform net view commands against IPS-SQL-DFS-01 and IPS-SQL-DFS-02. 2.041001. Launch a command prompt. Log on to IPS-SQL-DFS-01 as a domain administrator of the Americas domain. Exit the Local Security Policy tool. whether it should succeed or fail." 4. Note When you attempt a net view command against an untrusted computer. Functional Implementation Tests After Woodgrove Bank confirmed that the policy was applied to IPS-SQL-DFS-01 and IPS-SQL-DFS-02. In addition. Prerequisite Implementation Tests Before it ran any functional tests on the computer in the Encryption isolation group.17 IPS-SQL-DFS-01 Expected Functional Test Results Target computer Result IPS-TZ-XP-01 IPS-PRINTS-01 IPS-UT-XP-03 SA negotiated IPS-SQL-DFS-02 Succeed Hard SA Succeed Hard SA Succeed Hard SA Fail None .1600). the next step was to perform some basic functional tests to ensure that the policy was operating as expected. and after sufficient time had passed for replication and policy update to occur. 8. double-click Access this computer from the network. and then. Woodgrove Bank confirmed that the correct IPsec policy was applied to the IPS SQL-DFS-01 and IPS-SQL-DFS-02 computers. 7. Confirm that the output shows that the Directory Policy name is "IPSEC . Confirm that the ANAG_EncryptedResourceAccess_computers and ANAG_EncryptedResourceAccess_users groups are present. Launch the Local Security Policy tool. they used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. 5. expand User Rights Assignment. Confirm that the Authenticated Users group is not present.0. you must pass credentials for the local administrator to the computer.

3. expand Quick Mode. Expand the domain. and then launch Active Directory Users and Computers. Note Because of replication delays and polling frequency of the IPsec policies. a process of verification should be undertaken to confirm that the computer functions properly within the isolation group. Enabling the Boundary Isolation Group Woodgrove Bank placed the computers that must initiate or receive unauthenticated communication from untrusted computers in the CG_BoundaryIG_computers universal group. Log on to IPS-CH-DC-01 as a domain administrator of the Americas domain. 4. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. In the right pane. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. 4. In the Enter the object names to select text box. Click the Members tab. Implementing the Boundary Isolation Group The implementation team at Woodgrove Bank identified those computers that belonged to the Boundary isolation group and placed them in the CG_BoundaryIG_computers universal group. and then click Users. Click OK. Repeat steps 3-4 for each <Target Computer> listed in the previous table.262 Server and Domain Isolation Using IPsec and Group Policy To test the functionality of the implementation on target computers 1. select the Computers check box. To populate the CG_BoundaryIG_computers group 1. and then click Add. 5. 5. right-click the CG_BoundaryIG_computers security group. type IPS-PRINTS-01 and then click OK. and then click Properties. The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately. 6. Click the Object Types button. expand IP Security Monitor. 3. be sure to pass local administrator credentials with the net view command. 2. 7. Log on to IPS-SQL-DFS-01 as a domain administrator in the Americas domain. and then click Security Associations. Otherwise. Launch the IP Security Monitor MMC snap-in. Launch a command prompt. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Verifying the Boundary Isolation Group Deployment After the policy objects are created and deployed into Active Directory in the active state. and then click OK. 2. . expand IPS SQL-DFS-01. there will be delay between the time the group is added to the CG_BoundaryIG_computers group and when the Boundary Isolation Group Policy is applied.

18 IPS-PRINTS-01 Expected Functional Test Results Target computer Result IPS-UT-XP-03 IPS-TZ-XP-01 SA negotiated Succeed Soft SA Succeed Hard SA None IPS-SQL-DFS-01 Fail To test the functionality of the implementation on target computers 1. Launch the IP Security Monitor MMC snap-in. In addition. . be sure to pass local administrator credentials with the net view command.1600). and then click Security Associations. Confirm that the output shows that the Directory Policy name is "IPSEC – Boundary Isolation Group IPsec Policy (1. Log on to IPS-PRINTS-01 as a domain administrator in the Americas domain. 4. To confirm that the correct IPsec policy was applied to IPS PRINTS-01 1. Note When you attempt a net view command against an untrusted computer you must pass credentials for the local administrator to the computer. expand IP Security Monitor. The following table lists the target computers for each execution of net view. Woodgrove Bank confirmed that the correct IPsec policy was applied to the computer.0. Table C. expand IPS PRINTS-01. indicates whether it should succeed or fail.Appendix C: Lab Build Guide 263 Prerequisite Implementation Tests Before it ran any functional tests on the computer in the Boundary isolation group. expand Quick Mode. Launch a command prompt and then run the following command: netsh IPsec static show gpoassignedpolicy 3.041001. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Repeat steps 3-4 for each <Target Computer> listed in the previous table. Log on to IPS-PRINTS-01 as a domain administrator of the Americas domain. and lists the type of SA negotiated for each computer participating in the Encrypted Resource Access group." Functional Implementation Tests After Woodgrove Bank confirmed that the policy was applied to IPS-PRINTS-01. 5. 2. the next step was to perform some basic functional tests to ensure that the policy was operating as expected. and after sufficient time had passed for replication and policy update to occur. they used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. 3. Open a command prompt and run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. 2. Woodgrove attempted to perform net view commands against the computers listed in the following table.

In the right pane. 5. Click the Members tab. Note Because of replication delays and polling frequency of the IPsec policies. Click OK again. Note Because of replication delays and polling frequency of the IPsec policies. To add Domain Computers to the CG_IsolationDomain_computers Group 1. This approach ensures that any new computers added to the domain are automatically added to the Isolation Domain unless they have requirements that place them into another isolation group. and then click Add. the link order of the IPsec policies needs to be updated. Expand the domain. and then click Remove.264 Server and Domain Isolation Using IPsec and Group Policy Configuring the Isolation Domain as the Default Isolation Group Before the final functional tests were performed. the Woodgrove Bank administrators configured the security on the Isolation Domain so that it applies to all domain computers. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. 4. The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. and then click Users. 2. and then click Properties. To remove Domain Computers from the CG_BoundaryIG_computers group 1. the policy will apply after the session ticket times out and is refreshed with the new local group membership information. In the right pane. Reordering the IPsec Policy Link Order To ensure that the correct policies are applied to the hosts. Otherwise. Click the Members tab. 5. type Domain Computers and then click OK. 6. In the Enter the object names to select text box. This task has to do with the fact that the Standard Isolation . 2. and then click Users. right-click the CG_IsolationDomain_computers security group. 3. there will be delay between the time the Domain Computers group is added to the CG_IsolationDomain_computers group and when the Isolation Domain Group Policy is applied. 3. Click OK. the Domain Computers group was removed from the CG_BoundaryIG_computers group. click the Domain Computers group. right-click the CG_BoundaryIG_computers security group. In addition. Expand the domain. 4. Otherwise. Click Yes to remove the group. there will be delay between the time the group is removed from the CG_BoundaryIG_computers group and when the Boundary Isolation Group Policy is removed. Launch Active Directory Users and Computers on IPS-CH-DC-01 as a domain administrator of the Americas domain. 6. The computer can be restarted at this point if there is a need to have the IPsec policy applied to it immediately. and then click Properties.

you must pass credentials for the local administrator to the computer. Note When you attempt a net view command against untrusted computers. indicate whether it should succeed or fail. which was used as the default policy during initial deployment. use the arrow keys to order the policies as shown in the following table. In the Linked Group Policy objects list. The following tables list the target computers for each execution of net view. 4. the next step was to perform some basic functional tests to ensure that the policies were operating as expected. Expand the domain. Launch the GPMC as a domain administrator.19 Link Order of Group Policy Objects at the Domain Level Link order 1 2 3 4 5 Group Policy Object name IPSEC – Encryption Isolation Group Policy IPSEC – No Fallback Isolation Group Policy IPSEC – Boundary Isolation Group Policy IPSEC – Isolation Domain Policy Default Domain Policy Final Functional Tests—All Isolation Groups Enabled After Woodgrove Bank enabled all of the isolation groups. Table C. 2. The following procedure tests connectivity from IPS-SQL-DFS-01 (acting as an initiator) to various computers in the other isolation and network access groups. and list the type of SA negotiated for each computer that was selected for test purposes. Although some basic functional tests were done as each policy was implemented. The administrators attempted to perform net view commands with one or more computers in each isolation group against computers in other isolation groups to verify that the appropriate connectivity was established. 3. . Woodgrove Bank administrators were unable to perform a complete functional test because the isolation groups were enabled one at a time.Appendix C: Lab Build Guide 265 Group Policy was denoted as the default policy rather than the Boundary Isolation Group Policy. Additionally. Multiple computers were selected in some isolation groups because they have different traffic patterns. depending on whether they are the responder or initiator. Click the domain name. the administrators used the IP Security Monitor MMC snap-in to confirm that the appropriate SAs were created. To link the IPsec policies to the existing GPOs 1.

expand IPS TZ XP-06. Encryption IPS-UT-XP-03 Fail Initiator does not support Fall back to clear. Log on to IPS-TZ-XP-06 as a domain administrator in the Americas domain. Launch the IP Security Monitor MMC snap-in. IPS-PRINTS-01 Succeed Computer can successfully negotiate Hard SA with IPsec. Log on to IPS-SQL-DFS-01 as a domain administrator in the Americas domain. IPS-PRINTS-01 Succeed Computers can successfully negotiate IPsec. expand IPS SQL-DFS-01. 2. . Repeat steps 3-4 for each <Target Computer> listed in the previous table. None To test connectivity from target computers 1.20 IPS-SQL-DFS-01 Expected Functional Test Results Target computer IPS-ST-XP-05 IPS-TZ-XP-01 Result Reason SA negotiated Hard SA with Encryption Hard SA with Encryption Succeed Computers can successfully negotiate IPsec. IPS-UT-XP-03 Succeed Computers can successfully negotiate IPsec.21 IPS-TZ-XP-06 Expected Functional Test Results Target computer IPS-SQL-DFS01 IPS-ST-XP-05 IPS-TZ-XP-01 Result Fail Reason Responder is part of the Encrypted Resource Access. 4. 2. and then click Security Associations. Launch the IP Security Monitor MMC snap-in.266 Server and Domain Isolation Using IPsec and Group Policy Table C. be sure to pass local administrator credentials with the net view command. expand IP Security Monitor. To test connectivity from target computers 1. expand Quick Mode. 5. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. expand IP Security Monitor. SA negotiated None Hard SA Hard SA Hard SA Soft SA Succeed Computers can successfully negotiate IPsec. Table C. Succeed Computers can successfully negotiate IPsec. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Launch a command prompt. Succeed Computers can successfully negotiate IPsec. and then click Security Associations. The following procedure tests connectivity from IPS-TX-XP-06 (which acts as an initiator) to various computers in the other isolation and network access groups. 3. expand Quick Mode.

Succeed Initiator is part of the Encrypted 01 Resource Access group. Launch a command prompt. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. . The following procedure tests connectivity from IPS-ST-XP-06 (which acts as an initiator) to various computers in the other isolation groups. The following procedure tests connectivity from IPS-TZ-XP-01 (which acts as an initiator) to various computers in the other isolation and network access groups. Launch a command prompt.22 IPS-ST-XP-05 Expected Functional Test Results Target computer Result Reason SA negotiated Hard SA with Encryption Hard SA Hard SA Soft SA IPS-SQL-DFS. expand Quick Mode. expand IPS ST XP-05.Appendix C: Lab Build Guide 267 3. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. and then click Security Associations. 4. 5. Succeed Computers can successfully negotiate IPsec. Table C. Repeat steps 3-4 for each <Target Computer> listed in the previous table. To test the connectivity from target computers 1. 2. Succeed Computers can successfully negotiate IPsec. be sure to pass local administrator credentials with the net view command. be sure to pass local administrator credentials with the net view command. Launch the IP Security Monitor MMC snap-in. 3. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. Repeat steps 3-4 for each <Target Computer> listed in the previous table. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. expand IP Security Monitor. IPS-TZ-XP-01 IPS-PRINTS01 IPS-UT-XP-03 Succeed Computers can successfully negotiate IPsec. 4. Log on to IPS-ST-XP-05 as a domain administrator in the Americas domain. 5.

23 IPS-TZ-XP-01 Expected Functional Test Results Target computer IPS-SQL-DFS01 IPS-ST-XP-05 IPS-PRINTS01 IPS-UT-XP-03 Result Fail Reason Responder part of the Encrypted Resource Access group. To test connectivity from target computers 1. expand IPS TZ XP-01. Succeed Computers can successfully negotiate IPsec. Table C. 5. Launch the IP Security Monitor MMC snap-in. The following procedure tests connectivity from IPS-LT_XP-01 (which acts as an initiator) to various computers in the other isolation and network access groups. expand IP Security Monitor. expand Quick Mode. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. 4. 2. expand IP Security Monitor. 2. 3. Launch a command prompt. Log on to IPS-TZ-XP-01 as a domain administrator in the Americas domain. be sure to pass local administrator credentials with the net view command. Succeed Initiator supports Fall back to clear. . 3. Repeat steps 3-4 for each <Target Computer> listed in the previous table. and then click Security Associations. Fail Initiator does not support Fall back to clear. To test connectivity from target computers 1. SA negotiated None Hard SA Hard SA Soft SA Succeed Computers can successfully negotiate IPsec. expand IPS LT-XP 01. be sure to pass local administrator credentials with the net view command. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Log on to IPS-LT-XP-01 as a domain administrator in the Americas domain.268 Server and Domain Isolation Using IPsec and Group Policy Table C. Launch a command prompt. Succeed Computers can successfully negotiate IPsec. SA negotiated None Hard SA Hard SA None Succeed Computers can successfully negotiate IPsec. and then click Security Associations. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. Launch the IP Security Monitor tool.24 IPS-LT-XP-01 Expected Functional Test Results Target computer IPS-SQL-DFS01 IPS-ST-XP-05 IPS-TZ-XP-01 IPS-UT-XP-03 Result Fail Reason Responder is part of the Encrypted Resource Access group. expand Quick Mode.

SA negotiated None Succeed Computers can successfully negotiate IPsec. The following procedure tests connectivity from IPS-UT-XP-03 (which acts as an initiator) to various computers in the other isolation and network access groups. Launch a command prompt. Responder does not support Fall back to clear and Inbound passthrough. Responder does not support Fall back to clear and Inbound passthrough. . 2. Launch the IP Security Monitor MMC snap-in. and then run the following command: net view \\<Target Computer> Note For IPS-UT-XP-03. Log on to IPS-PRINTS-01 as a domain administrator in the Americas domain.Appendix C: Lab Build Guide 269 4. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. 4. and then click Security Associations.25 IPS-PRINTS-01 Expected Functional Test Results Target computer IPS-SQLDFS-01 IPS-ST-XP05 IPS-TZ-XP01 IPS-UT-XP03 Result Fail Reason Responder explicitly denies access to Boundary hosts. Responder is part of the Encrypted Resource Access group. The following procedure tests connectivity from IPS-PRINTS-01 (which acts as an initiator) to various computers in the other isolation groups. Hard SA Hard SA Soft SA To test connectivity from target computers 1.26 IPS-UT-XP-03 Expected Functional Test Results Target computer IPS-SQLDFS-01 IPS-ST-XP05 IPS-TZ-XP01 IPSPRINTS-01 Result Fail Reason Responder does not support Fall back to clear and Inbound passthrough. 5. Repeat steps 3-4 for each <Target Computer> listed in the previous table. be sure to pass local administrator credentials with the net view command. expand Quick Mode. 5. expand IP Security Monitor. Table C. expand IPS PRINTS 01. Repeat steps 3-4 for each <Target Computer> listed in the previous table. 3. Table C. SA negotiated None Fail Fail None None Soft SA Succeed Responder supports Fall back to clear and Inbound passthrough. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Succeed Computers can successfully negotiate IPsec. Succeed Initiator supports Fall back to clear. Responder is part of the Encrypted Resource Access group.

Performed a staged rollout of the Boundary isolation group and Isolation Domain to the entire enterprise. filter actions. and IPsec policies in Active Directory. expand IPS UT XP-03. 4. be sure to pass domain administrator credentials with the net view command.270 Server and Domain Isolation Using IPsec and Group Policy To test connectivity from target computers 1. rules. 3. expand IP Security Monitor. Configured several isolation groups to control Responder access. Launch the IP Security Monitor MMC snap-in. Configured the GPOs in Active Directory to correctly apply the IPsec policies. Summary When you complete the tasks in this appendix you have accomplished the following: • • • • • • • • Created the filter lists. Enabled and tested the Encryption isolation Group. and then run the following command: net view \\<Target Computer> Note For all domain-based computers. 2. . Repeat steps 3-4 for each <Target Computer> listed in the previous table. expand Quick Mode. Enabled and tested the Isolation Domain. Use the IP Security Monitor MMC snap-in to check the Security Associations field for each successful connection to confirm that the appropriate SA was negotiated. Enabled and tested the Boundary isolation group. Log on to IPS-UT-XP-03 as a domain administrator in the Americas domain. 5. Enabled and tested the No Fallback isolation group. and then click Security Associations. Launch a command prompt.

Information Disclosure. If IPsec is widely used. both of these options could generate serious consumer backlash. port numbers. . Repudiation. ISP filters are typically implemented by using hardware functions of routers that operate on specific protocol types (User Datagram Protocol [UDP] or Transmission Control Protocol [TCP]).Appendix D: IT Threat Categories This appendix provides a list of potential threats and attacks that can affect an organization and explains how a server and domain isolation solution can help mitigate them. leaving the ISP with only two very extreme options: ban all IPsec traffic. which many advanced users resent. reasoning that VPN is a business service that requires a higher fee subscription. Some Internet service providers (ISPs) have developed filtering practices that attempt to combat both man-in-the-middle attacks and spoofing of e-mail. and not data or acknowledgement). Man-in-the-Middle Attacks One common technique used by hackers is the man-in-the-middle attack. Tampering. The use of IPsec effectively disables this kind of filtering. and they justify this restriction by the need to fight junk e-mail. Elevation of Privilege) model and how the security measures implemented as part of this solution can be used to help mitigate them. many ISPs only authorize users to send e mail through the ISP's servers. or TCP flags (initial connection packet. This technique places a computer between two communicating computers in a network connection. Spoofing Identify Threats Spoofing identity threats include anything done to illegally obtain or access and use another person's authentication information. and attempts to prevent users from running servers in their homes. Other examples include attempts to ban some forms of virtual private networking (VPN) traffic. This technique provides the "man in the middle" with a live connection to the original computers and the ability to read and/or modify messages as they pass between them while the two computers' users think they are communicating only with each other. or ban traffic with certain identified peers. For example. Threats Identified by STRIDE This section describes a number of network security threats identified by the STRIDE (Spoofing. the restriction also prevents authorized users from using a legitimate e-mail service provided by a third party. However. Some cable ISPs try to block audio or video traffic in an attempt to force users to use their own voice-over-IP or video-streaming services. This category of threat includes man-in-the-middle attacks and trusted host communications with untrusted hosts. such as a user name or password. and the in-between computer then impersonates one or both of the original computers. Denial of Service.

" depicts the communication requirements for the Woodgrove Bank scenario and also the methodology that was used to create IPsec policies that govern how communications occur. Not every host that will be isolated requires communication with untrusted hosts. and other parties have no way to prove otherwise. even without obtaining the regular user’s credentials. Chapter 5. Tampering with Data Tampering with data threats involve the malicious modification of data. For example. or data as it flows between two computers on an open network. and eavesdropping. Because the attacker is in the middle of the exchange. Threats in this category include unauthorized connections and network sniffing. The use of IPsec for either encryption or authentication protects endpoints from session hijacking. Examples include unauthorized changes made to persistent data (such as defacement of a Web site). Session Hijacking Properly designed authentication mechanisms and long random passwords will resist network sniffing and dictionary attacks. Information Disclosure Information disclosure threats involve the exposure of information to individuals who are not supposed to have access to it. the greatest threat is spoofing.Trusted Hosts Communicating with Untrusted Hosts This threat is actually a superset of several smaller threats and includes the issues of general spoofing of identity. The simplest way to perform session hijacking is to first attempt to place the attacker’s computer somewhere in the connection path by using a specialized hacking tool. they can terminate one side of the TCP connection and maintain the other side by using the correct TCP/IP parameters and sequence numbers. modification of data between endpoints in a transmission. Session hijacking could enable an attacker to use a regular user’s privileges to access or modify a database. The vendor can then use the signed receipt as evidence that the user received the package. Examples include the ability of users to read files to which they were not granted access and the ability of an intruder to read data that is in transit between two computers. Because IPsec uses a policy-based mechanism to determine the level of security required between two hosts when negotiations begin. An example of this type of threat would be a user performing a prohibited operation in a system that lacks the ability to trace the prohibited operation. because the intent is to deceive a trusted host into thinking it is communicating with a trusted host. a user who purchases an item from a Web-based vendor might have to sign for the item when they receive it. attackers may use session hijacking to capture a session after the regular user has been authenticated and authorized. . Repudiation Repudiation threats involve users who deny that they performed an action. most of these issues are addressed by careful consideration of the tradeoffs between security and communication and then thoughtful design and implementation of an IPsec policy that reflects the preferred outcome. Nonrepudiation refers to the ability of a system to counter repudiation threats. or possibly to install software for further penetration. information held in a database. However. The attacker will observe the exchange and at some point take over. However. "Creating IPsec Policies for Isolation Groups. respectively. One specific threat in this category is session hijacking.

and administrators would have . and such attacks are called distributed denial of service attacks. but a denial of service attack launched from within an organization's network would still succeed if the attacking host or hosts can authenticate and communicate using IPsec. This access is sometimes explicit (as in the case of intranet Web servers) and sometimes implicit because of the poor security protection of some applications. Denial of Service Denial of service attacks are directed attacks against a specific host or network. Internet-based distributed denial of service attacks will be rendered harmless. which avoids the problem of sending a plaintext password but is only slightly more challenging. IPsec requires authentication before establishing communications. it was observed that networks would not have been flooded with the worm's traffic if they would have had simple rules in place that limited UDP traffic to up to 50 percent of available bandwidth. Many applications use a challengeresponse mechanism. hackers use network sniffing tools to log TCP connections and obtain a copy of the communicated information. which redirect IP traffic through the attacker’s computer and make it easy to log all connections. Target computers are usually compromised somehow. and therefore helps mitigate most distributed denial of service attacks (except those that use a trusted attacker scenario). Discriminating Between Standard and Attack Traffic Shortly after the Slammer worm struck in January 2003. Although these tools do not work very well on switched networks. and an attacker who sniffs the network will find this information easy to obtain. Using IPsec to encrypt such exchanges effectively protects against network sniffing. It is possible to set up policy rules that require a set of applications to be accessible only after a successful IPsec negotiation. The compromised computers are referred to as zombies. Network Sniffing Attackers attempt to capture network traffic for two reasons: to obtain copies of important files during their transmission. but dictionary attacks can often deduce it from a copy of the challenge and response.Appendix D: IT Threat Categories 273 Unauthorized Connections Many network configurations have a very trusting security posture and grant access to vast amounts of information from computers inside the perimeter. The infected hosts would have quickly filled up 50 percent of the bandwidth with UDP traffic. Denial of service attacks can be distributed across many attackers to focus the effort on a particular target. but attackers can bypass these tests by forging addresses. and a malware script or program is installed on them that allows the attacker to use the computers to direct a coordinated flood of network traffic to another computer or group of computers. and to obtain passwords so that the attackers can extend their penetration. it is possible even on switched networks to attack the Address Resolution Protocol (ARP) by using other specialized tools. but the rest of the bandwidth would have remained available for operational traffic. A few protocols (Post Office Protocol 3 [POP3] and File Transfer Protocol [FTP]. In other words. Automatic teller machines (ATM) would have continued working. On a broadcast network. which results in an inability of the network to handle the traffic and thereby disrupts the legitimate flow of traffic. Some policies rely on simple address tests. These attacks usually send more traffic to a host or router than it can handle within a given time. for example) still send plaintext passwords over the network. IPsec can be used to implement an additional connection check. The attacker will not be able to read the password directly.

However. Elevation of privilege threats include situations in which an attacker has effectively penetrated all system defenses to exploit and damage the system. Nimda. Under stress. and so on. In typical conditions. the propagation mechanisms of the CodeRed. Elevation of Privilege This type of threat allows an unprivileged user to gain privileged access that enables them to compromise or possibly destroy an entire system environment. One form of denial of service attack operates in a distributed fashion that directs a number of computers to simultaneously attack traffic to a selected target. which were all supposed to send crippling traffic to whitehouse. physical security in the form of security guards. in most network-level attacks. USA). such simple policies that can be left in place can provide a reliable safety net. By using IPsec for important traffic. cameras in data centers. TCP traffic. and apply these collected statistics as fair queuing weights during periods of heavy congestion. it is usually possible to program such a policy by default in routers. and the resulting traffic crippled many local and regional networks. Each infected computer performed hundreds of thousands of infection attempts on indiscriminate targets. network administrators can monitor the mix of traffic on the network and determine how much of it is UDP traffic. DC. and keycards or keys on doors all help prevent a trusted device from becoming compromised. physical security is completely bypassed.gov (the domain of the White House in Washington. a weighted fair queuing algorithm can engage to ensure that the resource is shared according to a standard pattern. administrators can apply a slightly more sophisticated version of the UDP policy. which operate by sending excess traffic to saturate either a specific server or a specific portion of a network.274 Server and Domain Isolation Using IPsec and Group Policy been able to use TCP to apply patches and propagate policies. In fact. and it allows network operators to distinguish between different types of traffic. access controls to sensitive locations. Worms and Denial of Service Attacks The recent past has shown that networks are vulnerable to denial of service attacks. and provides an added level of protection to potential victims of the attack. In fact. collect long term trends and statistics during periods of standard network activity. and Slammer worms were denial of service attacks against the Internet. Although the policy of limiting UDP traffic is simplistic. For example. It slows down attackers by forcing expensive computations. Physical security is the lowest layer of defense for most IT security threats. Other Threats Not all threats fit cleanly into the STRIDE model. Physical Security Physical security involves providing physical access to a system or resource to only the minimum number of users who need it. Physical security still provides significant value as part of a defense-in-depth approach. The following items depict other threats and describe their potential impact on a server and domain isolation solution. The CodeRed worm first tried to penetrate a number of Web servers. Internet Control Message Protocol (ICMP) traffic. these can be especially difficult to defend against. Using multiple . IPsec protects against denial of service attacks in several ways.

Because this type of attack targets the user of the computer. the attacker has more information that they can use to focus their efforts. Application Security Most attacks that are directed at applications attempt to exploit vulnerabilities that exist in those applications or the operating system. Most of the protocols and services designed for networks were not created with the potential for malicious intent in mind. Because IPsec is implemented at the network layer of the Open System Interconnection (OSI) model.Appendix D: IT Threat Categories 275 methods of physical security is important and can help prevent some of the more serious data center security breaches. it determines whether a packet is permitted or discarded before that packet ever reaches the application. Without physical security. A number of network threats were described in some detail earlier in this appendix. All security discussed in this solution is based on the assumption that physical security has been addressed. no other security measures can be considered effective. Network Security A network is a system of interconnected computers. This project is one in which the company is developing a new product or service. Additional information about how IPsec protects against some of these network attacks can be found in the “Configuring TCP/IP Name Resolution” section of the “Configuring IP Addressing and Name Resolution” chapter within the Windows® XP Professional Resource Kit. For example. It should be very clear that compromised physical security always means that all security layers have been compromised. and the wide availability of the Internet caused many malicious users to focus their efforts on systems and services for exploitative purposes or to cause disruption. easy network access. Similarly. If the operator provides the attacker with the name of the supervisor and perhaps even the location or contact information for that person. IPsec cannot protect against it. a malicious user who has access to isolated systems and abuses that access (often referred to as a trusted attacker) will need to be prevented using other security technologies. using server and domain isolation will not resolve all of the threats that organizations face. This behavior means that IPsec cannot make application-level determinations but can be used provide security for application traffic at a lower level. a would-be attacker could use the telephone to call the target company and then ask for the name of the supervisor in charge of a particular project. . The advent of high-speed computing. Only a thorough understanding of available options and a detailed knowledge of the technical challenges will allow organizations adequately to protect their IT environments. Social Engineering Social engineering is the act of exploiting weaknesses in human behavior to gain access to or learn more about a system. Summary Clearly. which is what the attacker wants to know more about.

gov/omb/memoranda/fy04/m04-04. The Antivirus Defense-in-Depth Guide http://go.microsoft.microsoft.microsoft.gov/snac/support/defenseindepth.com/fwlink/?LinkId=69753 Microsoft Operations Framework http://go.nsa.com/fwlink/?LinkId=69766 NIST Computer Security Division Web site http://csrc.com/fwlink/?LinkId=69755 Healthcare Without Boundaries: Integration Technology for the New Healthcare Economy http://go.html NSA Security Recommendation Guides http://nsa2.microsoft.microsoft.dll http://go.microsoft.com/fwlink/?LinkId=28732 Network Access Protection http://go.microsoft.nsa.com/win2k/download.gov/ia/industry/niap.gov/publications/index.Links The following section summarizes the links to external resources that this document references.com/fwlink/?LinkId=69757 "E-Authentication Guidance for Federal Agencies" memorandum in PDF format http://www.whitehouse.pdf National Information Assurance Partnership http://www. The aim of this section is to make it easier for you to add links to your own documentation.nist.com/fwlink/?LinkId=69765 How to Restrict the Use of Certain Cryptographic Algorithms and Protocols in Schannel.microsoft.www.cfm Overview: Windows 2000 Common Criteria Certification http://go.com/fwlink/?LinkId=69764 "Defense in Depth" white paper in PDF format http://www.pdf Enterprise Design chapter of the Security Architecture Blueprint within the Windows Server System Reference Architecture http://go.com/fwlink/?LinkId=69761 Virtual Private Networks http://go.com/fwlink/?LinkId=69759 FIPS 140 Evaluation http://go.microsoft.com/fwlink/?LinkId=69752 Microsoft Solutions Framework http://go.com/fwlink/?LinkId=69763 TechNet Security Center http://go.htm .microsoft.com/fwlink/?LinkId=69762 Introduction to Network Access Protection http://go.microsoft.conxion.microsoft.

com/fwlink/?LinkId=69787 Windows Script 5.6 for Windows 2000 and XP http://go.com/fwlink/?LinkId=69779 SMS 2003 Asset Management http://go.com/fwlink/?LinkId=69783 Windows Management Instrumentation (WMI) CORE 1.com/fwlink/?LinkId=69781 IBM http://www.microsoft. "Defining the Security Landscape" http://go.com/fwlink/?LinkId=69839 Members of an Extremely Large Number of Groups Cannot Log On to the Domain http://go.microsoft.microsoft.com/fwlink/?LinkId=69789 New Resolution for Problems That Occur When Users Belong to Many Groups http://go.com/fwlink/?LinkId=69775 Securing Windows 2000 Server: Chapter 2.com/fwlink/?LinkId=69777 Microsoft Systems Management Server http://go.com/fwlink/?LinkId=69782 Windows Management Instrumentation http://go.6 for Windows 98.microsoft.microsoft.com/fwlink/?LinkId=69780 Microsoft Windows Script Downloads http://go.com/fwlink/?LinkId=69784 Microsoft Windows Script 5.0) installation package http://go. Windows Millennium Edition.microsoft.microsoft.microsoft.microsoft.microsoft.Wireless Networking http://go.com/fwlink/?LinkId=69840 Using Microsoft Windows IPSec to Help Secure an Internal Corporate Network Server http://go.microsoft. and Windows NT 4.microsoft.microsoft.com/fwlink/?LinkId=69842 .microsoft.com/fwlink/?LinkId=69776 New features for IPSec http://go.com/fwlink/?LinkId=69841 Internet Protocol Security for Windows 2000 Server http://go.com/fwlink/?LinkId=69774 Determining Your IPSec Needs http://go.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69788 Deploying Windows Firewall Settings for Microsoft Windows XP with Service Pack 2 http://go.5 (Windows 95/98/NT 4.com/fwlink/?LinkId=23277 Improving Security with Domain Isolation http://go.microsoft.com/fwlink/?LinkId=69778 L2TP/IPSec NAT-T update for Windows XP and Windows 2000 http://go.microsoft.microsoft.com Configuring Firewalls http://go.6 Documentation http://go.ibm.0 http://go.com/fwlink/? LinkId=69786 Microsoft Windows Script 5.

com/fwlink/?LinkId=41652 IPSec NAT-T is not recommended for Windows Server 2003 computers that are behind network address translators http://go.com/fwlink/?LinkId=69965 Administering Group Policy with the GPMC http://go.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69859 .microsoft.microsoft.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69888 Back up System State data http://go.microsoft.com/fwlink/?LinkId=69852 The default behavior of IPsec NAT traversal (NAT-T) is changed in Windows XP Service Pack 2 http://go.com/fwlink/?LinkId=69847 Windows XP Service Pack 2 Support Tools http://go.microsoft.microsoft.com/fwlink/?LinkId=69856 IPSec Troubleshooting Tools http://go.com/fwlink/?LinkId=69853 Deploying Windows Firewall Settings for Microsoft Windows XP with Service Pack 2 document download http://go.microsoft.com/fwlink/?LinkId=69850 Group Policy Management Console with Service Pack 1 http://go.com/fwlink/?LinkId=69855 The Cable Guy—October 2004: Problems with Using Network Address Translators http://go.com/fwlink/?LinkId=69849 Windows 2000 Server Resource Kit http://go.com/fwlink/?LinkId=69843 Information Security at Microsoft Overview http://go.com/fwlink/?LinkId=69845 IPSec Default Exemptions Can Be Used to Bypass IPsec Protection in Some Scenarios http://go.com/fwlink/?LinkId=69966 Deploying IPsec chapter from the Windows Server 2003 Deployment Kit http://go.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69851 Windows Server 2003 Service Pack 1 http://go.com/fwlink/?LinkId=69846 IPSec default exemptions are removed in Windows Server 2003 http://go.Links 279 IPsec http://go.com/fwlink/?LinkId=69857 IPsec troubleshooting in Microsoft Windows 2000 Server http://go.microsoft.com/fwlink/?LinkId=69844 Windows Server 2003 Active Directory http://go.microsoft.com/fwlink/?LinkId=69890 Understanding IPSec Protection During Computer Startup http://go.microsoft.com/fwlink/?LinkId=69858 Windows XP Service Pack 2 Support Tools download http://go.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69854 Windows Server 2003 Group Policy http://go.

microsoft.com/fwlink/?LinkId=69861 Troubleshooting Kerberos Errors document download http://go.microsoft.Part 2: Network Protection Technologies http://go.microsoft.microsoft.com/fwlink/?LinkId=69866 How to Enable IPSec Traffic Through a Firewall http://go.microsoft.280 Server and Domain Isolation Using IPsec and Group Policy Active Directory Operations Overview: Troubleshooting Active Directory-Related DNS Problems http://go.com/fwlink/?LinkId=69873 Default MTU Size for Different Network Topology http://go.com/fwlink/?LinkId=69876 The “Troubleshooting Name Resolution and Addressing” section in the “Configuring IP Addressing and Name Resolution” chapter in the Windows XP Professional Resource Kit http://go.microsoft.com/fwlink/?LinkId=69871 Description of the IPSec policy created for L2TP/IPSec http://go.com/fwlink/?LinkId=69875 TCP/IP in Windows 2000 Professional http://go.microsoft.microsoft.microsoft.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69860 HOW TO: Use Portqry to Troubleshoot Active Directory Connectivity Issues http://go.microsoft.com/fwlink/?LinkId=69862 Troubleshooting Kerberos Delegation document download http://go.microsoft.com/fwlink/?LinkId=69877 Windows Server 2003 TCP/IP Troubleshooting http://go.com/fwlink/?LinkId=69864 IPSec Policy Permissions in Windows 2000 and Windows Server 2003 http://go.com/fwlink/?LinkId=69869 Troubleshooting Group Policy in Microsoft Windows Server document download http://go.microsoft.com/fwlink?LinkID=69863 How IPsec Works http://go.com/fwlink/?LinkId=69874 System Code Errors (12000-15999) http://go.microsoft.com/fwlink/?LinkId=69878 .microsoft.microsoft.com/fwlink/?LinkId=69867 Connections time out when client computers that are running Windows Server 2003 or Windows XP try to connect to a server on a wireless network that uses IPsec NAT-T http://go.com/fwlink/?LinkId=69872 How to configure an L2TP/IPSec connection by using Preshared Key Authentication http://go.microsoft.com/fwlink/?LinkId=69870 Changes to Functionality in Microsoft Windows XP Service Pack 2 .com/fwlink/?LinkId=69886 How to troubleshoot TCP/IP connectivity with Windows XP http://go.com/fwlink/?LinkId=69868 White Paper: Troubleshooting Group Policy in Windows 2000 http://go.microsoft.com/fwlink/?LinkId=69865 Troubleshooting Translational Bridging http://go.

com/fwlink/?LinkId=69885 Authentication Vulnerabilities in IKE and Xauth with Weak Pre-Shared Secrets http://www.microsoft.com/fwlink/?LinkId=69883 How TCP/IP Works http://go.com/fwlink/?LinkId=69882 Overview of Windows 2000 Network Architecture http://go.com/fwlink/?LinkId=69880 Basic L2TP/IPSec Troubleshooting in Windows XP http://go.umn.com/fwlink/?LinkId=69887 .microsoft.microsoft.microsoft.microsoft.microsoft.com/fwlink/?LinkId=69881 Microsoft Windows 2000 TCP/IP Implementation Details http://go.microsoft.microsoft.Links 281 IPsec troubleshooting in Microsoft Windows 2000 Server http://go.com/fwlink/?LinkId=69884 Special IPsec considerations http://go.ima.com/fwlink/?LinkId=69879 Microsoft Windows 2000 Advanced Documentation http://go.edu/~pliam/xauth The “Configuring TCP/IP Name Resolution” section of the “Configuring IP Addressing and Name Resolution” chapter in the Windows XP Professional Resource Kit http://go.

Volt Information Sciences Chrissy Lewis. Richard Harrison. V6 Security. Infosys Technologies Jay Zhang .Acknowledgments Microsoft Solutions for Security and Compliance (MSSC) would like to acknowledge and thank the people who were directly responsible or made a substantial contribution to writing and reviewing Server and Domain Isolation using IPSec and Group Policy . Inc. Wadeware Wendy Prowell. Content Master Charles Denny William Dixon. Siemens Agency Services David Mowers Jeff Newfeld Rob Oikawa Tessa Porterfield Bill Reid Program Managers Jeff Coon. Volt Information Sciences Testers Mehul Mediwala. Content Master Jennifer Kerns. Content Master Editors John Cobb. Content Master Release Manager Karl Seng. Volt Information Sciences Lea Galanter. Infosys Technologies Balambikai P.. Authors and Experts Steve Clark David Coombes. Content Master Steve Ryan. Volt Information Sciences Bomani Siwatu Alison Woolford. Volt Information Sciences Karl Grunwald Dan Hitchcock Masoud Hoghooghi Joanne Kennedy Mohan Kotha Karina Larson. Siemens Agency Services Contributors Tony Bailey Kimmo Bergius Chase Carpenter Barbara Chung David Cross Michael Glass. Content Master Steve Wacker.

Avanade Jason Popp Steve Riley Henry Sanders Lee Walker Shain Wray Liqiang (Larry) Zhu . Avanade Sinead O’Donovan Greg Petersen.Reviewers Chris Black Geoff Brock Charisa Martin Cairn Mathieu Groleau Jeff Hamblin Patrick Hanrion Nate Harris Craig Nelson.