Microsoft SQL Server

Best Practices v1.0 – Nov 2013

 2

Copyright 2013 Nutanix, Inc.

All rights reserved. This product is protected by U.S. and international

copyright and intellectual property laws.

Nutanix is a trademark of Nutanix, Inc. in the United States and/or

other jurisdictions. All other marks and names mentioned herein may
be trademarks of their respective companies.

MSSQL on Nutanix – Best Practices | 2

|2
Table of Contents
1. Executive Summary ..................................................................................... 5

2. Introduction .................................................................................................. 6
3
3. Solution Overview ........................................................................................ 7

3.1. The Nutanix Architecture........................................................................................ 7

3.2. Microsoft SQL the Nutanix Way ............................................................................. 9

4. Best Practice Checklist .............................................................................. 13

5. Best Practice Detail .................................................................................... 17

5.1. MSSQL Server Sizing .......................................................................................... 17

5.2. General Sizing ...................................................................................................... 20

5.3. MSSQL Storage Sizing ........................................................................................ 21

5.4. Nutanix ................................................................................................................. 23

5.5. Network ................................................................................................................ 24

6. Validation & Benchmarking ........................................................................ 25

6.1. Environment Overview ......................................................................................... 25

6.2. Test Scripts & Configurations .............................................................................. 27

6.3. Benchmarks ......................................................................................................... 40

6.4. How to Interpret the Results ................................................................................ 41

7. Results ....................................................................................................... 44

7.1. SQLIO – Single VM .............................................................................................. 44

8. Further Research ....................................................................................... 51

9. Conclusion ................................................................................................. 52

10. Appendix: Configuration ............................................................................. 53

11. References ................................................................................................ 54

11.1. Table of Figures ................................................................................................... 54

11.2. Table of Tables .................................................................................................... 55

MSSQL on Nutanix – Best Practices | 3

|3
12. About the Author ........................................................................................ 56

MSSQL on Nutanix – Best Practices | 4

|4
1. Executive Summary
This document makes recommendations for the design, optimization and scaling of Microsoft
SQL Server (MSSQL) deployments on Nutanix. It shows the scalability of the Nutanix Virtual
Computing Platform and provides detailed performance and configuration information on the
scale-out capabilities of the cluster when leveraged for MSSQL deployments. 5

Extensive testing has been performed to simulate real-world workloads and conditions of a
MSSQL environment on Nutanix. The sizing data and recommendations made in this document
are based upon multiple testing iterations and thorough technical validation. The solution and
testing data was completed with MSSQL deployed on VMware vSphere, both running on the
Nutanix Virtual Computing Platform. Testing validation was done using SQLIO, SQLSIM and
HammerDB.

The Nutanix platform offers the ability to run both MSSQL and VM workloads simultaneously on
the same platform. Density for MSSQL deployments will be primarily driven by the database’s
CPU and storage requirements. Test validation has shown it is preferred to increase the
number of MSSQL VMs on the Nutanix platform, as compared to scaling the number of SQL
instances, to fully take advantage of its performance and capabilities. From an I/O standpoint
the Nutanix platform was able to handle the throughput and transaction requirements of a
demanding MSSQL server given NDFS localized I/O and server attached flash.

A single VM running SQLIO was able to get over 35,000 random IOPS and over 1.2 GBps of
sequential throughput. During the tests latencies peaked at 1 ms for read, however were
normally in the microseconds range. Write latencies were at 1 ms for 8 KB and 64 KB block
sizes and peaked at <5 ms for 512 KB.

Sizing for the pods was determined after careful consideration of performance as well as
accounting for additional resource for N+1 failover capabilities.

MSSQL on Nutanix – Best Practices | 5

|5
2. Introduction

Audience
6
This best practices document is part of the Nutanix Solutions Library and is intended for
architecting, designing, managing, and/or supporting Nutanix infrastructures. Consumers of this
document should be already familiar with VMware vSphere, Microsoft SQL Server (MSSQL),
and Nutanix.

This document has been broken down to address key items for each role focusing on the
enablement of a successful design, implementation, and transition to operation.

Purpose

This document will cover the following subject areas:

o Overview of the Nutanix solution
o The benefits of MSSQL on Nutanix
o Overview of the high-level MSSQL best practices for Nutanix
o Expanded detail on the MSSQL best practices for
o Design and configuration considerations when architecting a MSSQL solution on Nutanix
o Benchmarking MSSQL performance on Nutanix

If you are interested in the high-level best practices continue with the ‘Best Practice Checklist’
section below.

MSSQL on Nutanix – Best Practices | 6

|6
3. Solution Overview
3.1. The Nutanix Architecture

The Nutanix Virtual Computing Platform is a scale-out cluster of high-performance nodes, or 7

servers, each running a standard hypervisor and containing processors, memory and local
storage (consisting of SSD Flash and high capacity SATA disk drives). Each node runs virtual
machines just like a standard virtual machine host.

Controller
Controller User
User VM(s)
VM(s)
VM
VM

VM I/O
Passthrough Hypervisor
Hypervisor

SCSI
SCSI Controller
Controller
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
SSD
SSD
SSD
SSD

Figure 1 Nutanix Node Architecture

In addition, local storage from all nodes is virtualized into a unified pool by the Nutanix
Distributed File System (NDFS). In effect, NDFS acts like an advanced NAS that uses local
SSDs and disks from all nodes to store virtual machine data. Virtual machines running on the
cluster write data to NDFS as if they were writing to shared storage.

User
User VM(s)
VM(s) User
User VM(s)
VM(s) User
User VM(s)
VM(s)

Hypervisor
Hypervisor VM I/O Hypervisor
Hypervisor VM I/O ... Hypervisor
Hypervisor VM I/O

SCSI
SCSI Controller
Controller Controller
Controller SCSI
SCSI Controller
Controller Controller
Controller SCSI
SCSI Controller
Controller Controller
Controller
VM
VM VM
VM VM
VM
HDD
HDD
HDD
HDD

HDD
HDD
HDD
HDD

HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD

HDD
HDD
HDD
HDD

HDD
HDD
HDD
HDD
SSD
SSD

SSD
SSD

SSD
SSD
SSD
SSD

SSD
SSD

SSD
SSD

NDFS
SCALE
SCALE

Figure 2 Nutanix Architecture

MSSQL on Nutanix – Best Practices | 7

|7
NDFS is VM aware and provides advanced data management features. It brings data closer to
virtual machines by storing the data locally on the system, resulting in higher performance at a
lower cost. Nutanix Virtual Computing Platform can horizontally scale from as few as three
nodes to a large number of nodes, enabling organizations to scale their infrastructure as their
needs grow.

The Nutanix Elastic Deduplication Engine is a software-driven, massively scalable and 8

intelligent data reduction technology. It increases the effective capacity in the disk tier, as well
as the RAM and flash cache tiers of the system, by eliminating duplicate data. This substantially
increases storage efficiency, while also improving performance due to larger effective cache
capacity in RAM and flash. Deduplication is performed by each node individually in the cluster
allowing for efficient and uniform deduplication at scale. This technology is increasingly
effective with full/persistent clones or P2V migrations.

Sequential streams of data are

Only a single instance of the shared VM Each node participates in, and performs,
fingerprinted at 4K granularity for
data is pulled into the cache upon read its own fingerprinting and deduplication
efficient deduplication

VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N

Hypervisor
Hypervisor Hypervisor
Hypervisor Hypervisor
Hypervisor

Cache
Cache Cache
Cache Cache
Cache
Cache Cache Cache
CVM
CVM CVM
CVM CVM
CVM
Storage
Storage Storage
Storage Storage
Storage
...

NDFS
Figure 3 Elastic Deduplication Engine

Inspired by the Google File System, NDFS delivers a unified pool of storage from all nodes
across the cluster, leveraging techniques including striping, replication, auto-tiering, error
detection, failover’ and automatic recovery. This pool can then be presented as shared storage
resources to VMs for seamless support of features like vMotion, HA, and DRS, along with
industry-leading data management features. Additional nodes can be added in a plug-and-play
manner in this high-performance scale-out architecture to build a cluster that will easily grow as
your needs do.

MSSQL on Nutanix – Best Practices | 8

|8
3.2. Microsoft SQL the Nutanix Way

The Nutanix platform operates and scales Microsoft SQL Server (MSSQL) in conjunction with
the other hosted services providing a single scalable platform for all deployments. For existing
sources and platforms, interaction with the MSSQL platform on Nutanix will occur over the
network. The figure below shows a high-level view of the MSSQL on Nutanix solution: 9

Users
Users

Operational
Service Intelligence
Interaction

Data Integration

External
External Sources
Sources
Desktops
Desktops Applications/Data
Applications/Data Servers
Servers

MSSQL VMs
SCALE

SCALE

Figure 4 MSSQL on Nutanix Conceptual Arch

The Nutanix approach of modular scale-out enables customers to select any initial deployment
size and grow in more granular data and compute increments. This removes the hurdle of a
large up-front infrastructure purchase that a customer will need many months or years to grow
into, ensuring a faster time-to-value for the implementation.

MSSQL on Nutanix – Best Practices | 9

|9
Data Tiering and Management
The Nutanix Distributed Filesystem (NDFS) has a built-in Information Lifecycle Management
(ILM) process which will automatically handle data placement. Applying this to the data lifecycle
in MSSQL, all hot data is automatically written to the high performance SSD tier. Both hot and
warm data sit in this tier to be readily accessed and provide the highest performance. An in- 1
memory read cache is leveraged to cache frequently accessed data from all tiers. However, 0
given the sequential nature of some MSSQL files, Nutanix ILM can be configured to
automatically bypass the SSD tier for sequential workloads, increasing SSD endurance.
The Nutanix Elastic Deduplication Engine is integrated with NDFS ILM to allow for efficient
deduplication of data across both the capacity and performance tiers. Upon a sequential write
the I/O can be fingerprinted at a 4K granularity. Duplicate fingerprints, meaning duplicate
pieces of data, can then be deduplicated on HDD as well as when brought up into the Content
Cache which spans both memory and SSD and acts as a read cache. This can greatly increase
utilization in the capacity tiers by removing duplicate data, as well as increase the effective
cache sizes by only bringing one copy of the data into the Content Cache.
In the case of MSSQL, this means any common data (OS, DB, etc.) can be deduplicated
allowing for higher NDFS read cache hits for MSSQL reads and higher capacity utilization.
Given the sequential nature of MSSQL, very little overhead is seen given the larger operation
sizes. NOTE: Random writes will not be fingerprinted as to not impact write latency.
The figure below shows the NDFS I/O path and how fingerprinting and the Content Cache are
integrated:
Compute fingerprint
Read IO
Write IO

Memory
Content
Cache
NDFS EDE

OpLog*
SSD
Drain Cache

Extent Store
HDD
NDFS ILM

Cloud NAS, etc. Extensible

*NOTE: Sequential IO can be configured to bypass

SSD and be directly written to the HDD tier.
Figure 5 NDFS I/O Path

MSSQL on Nutanix – Best Practices | 10

| 10
As MSSQL data becomes less frequently accessed Nutanix ILM will automatically see the data
is cooling down and migrate the data from SSD to the higher-capacity HDD tier in order to free
SSD space for new hot data. As mentioned above, this data can also be configured to
automatically bypass the SSD tier and be written directly to HDD. This data can then be
automatically compressed after a specified period of time to allow for increased storage
capacity. In the event this data becomes accessed again it will automatically become un-
compressed and placed on the appropriate tier.
1
1
This keeps the most highly accessed data in the cache and/or highest performance tier,
meaning indexes and heavily accessed data will have the lowest latency and highest possible
performance.

Data IO Detail

The figure below describes the high-level IO path for VMs and MSSQL VMs running on Nutanix.
As shown, all IO operations are handled by NDFS and occur on the local node to provide the
highest possible IO performance. Data written to the MSSQL VMs occurs locally for all VMs on
the same ESXi node and over 10GbE for VMs and sources hosted on another node or remotely.
VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N

Hypervisor
Hypervisor Hypervisor
Hypervisor ... Hypervisor
Hypervisor

Cache
Cache Cache
Cache Cache
Cache
Cache Cache Cache
CVM
CVM CVM
CVM CVM
CVM
Storage
Storage Storage
Storage Storage
Storage

External
External
10GbE
10GbE Network
Network
Sources
Sources
NDFS
Figure 6 Data IO Detail

The figure below describes the detailed IO path for VMs and MSSQL VMs running on Nutanix.
All write IOs, including data being input to the MSSQL VMs, will occur locally on the local node’s
SSD tier to provide the highest possible performance. Read requests for the MSSQL VMs
occur locally and are served from the high performance in-memory read cache (if cached) or the
SSD or HDD tier depending on placement. Each node will also cache frequently accessed data
in the read cache for any local data (VM data, MSSQL Server data). Nutanix ILM will continue
to constantly monitor data and the IO patterns to choose the appropriate tier placement.
VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N VM
VM 11 ... VM
VM N
N

Hypervisor
Hypervisor Hypervisor
Hypervisor ... Hypervisor
Hypervisor

Cache
Cache Cache
Cache Cache
Cache
Cache Cache Cache
CVM
CVM CVM
CVM CVM
CVM
Storage
Storage Storage
Storage Storage
Storage

External
External
10GbE
10GbE Network
Network
Sources
Sources
NDFS
Figure 7 Data IO Detail Expanded

MSSQL on Nutanix – Best Practices | 11

| 11
Why run MSSQL on Nutanix?

Nutanix enables you to run multiple workloads all on the same scalable converged infrastructure
o Modular incremental scale: With the Nutanix solution you can start small and scale. A
single Nutanix block (four nodes) provides from 20-40+ TB storage and up to 80+ cores 1
in a compact footprint. Given the modularity of the solution, you can granularly scale 2
per-node giving you the ability to accurately match supply with demand and minimize the
upfront CapEx.
o High performance: Up to 100,000 random read IOPS and up to 3 GB/s of sequential
throughput in a compact 2U cluster. ILM keeps indexes and heavily access data in the
high performance SSD and cache tiers.
o Integrated: The Nutanix platform provides full support for VAAI allowing you to leverage
all the latest advancements from VMware and taking your solution to the next level.
o Elastic Deduplication: The Nutanix Elastic Deduplication Engine provides granular
deduplication of data to increase cache efficiency. The engine will utilize the unique
fingerprints of data and only bring one copy up into the Nutanix Content Cache. This
allows for the highest possible cache utilization and higher performance for VM’s
accessing common data - eliminating the issues normally seen with full clones or P2V
migrations.
o Data efficiency: The Nutanix solution is truly VM-centric for all compression and
deduplication policies. Unlike traditional solutions that perform these tasks mainly at the
LUN level, the Nutanix solution provides all of these capabilities at the VM and file level,
greatly increasing efficiency and simplicity. By allowing for both inline and post-process
compression capabilities and cache and on-disk deduplication, the Nutanix solution
breaks the bounds set by traditional solutions.
o Effective Information Lifecycle Management: Nutanix incorporates heat-optimized
tiering (HOT), which leverages multiple tiers of storage and optimally places data on the
tier that provides the best performance. The architecture was built to support local disks
attached to the controller VM (SSD, HDD) as well as remote (NAS) and cloud-based
source targets. The tiering logic is fully extensible, allowing new tiers to be dynamically
added and extended. The Nutanix system continuously monitors data-access patterns to
determine whether access is random, sequential, or a mixed workload. Random I/O
workloads are maintained in an SSD tier to minimize latencies. Sequential workloads
can be automatically placed into HDD to improve endurance.
o Business continuity and data protection: Native snapshotting and replication features
provide an extensive DR and protection capability. VSS provides integration for
application consistent snapshots and a SRA for VMware SRM integration.
o Enterprise-grade cluster management: A simplified and intuitive Apple-like approach
to managing large clusters, including a converged GUI that serves as a single pane of
glass for servers and storage, alert notifications, and bonjour mechanism to auto-detect
new nodes in the cluster.
o High-density architecture: Nutanix uses an advanced server architecture in which 8
Intel CPUs (up to 80+ cores) and up to 1TB of memory are integrated into a single 2U
appliance. Coupled with data archiving and compression, Nutanix can reduce desktop
hardware footprints by up to 4x.

MSSQL on Nutanix – Best Practices | 12

| 12
4. Best Practice Checklist
The MSSQL on Nutanix best practices can be summarized into the following high-level items.
NOTE: the majority of best practice configuration and optimization comes at the MSSQL level.
1
General 3
o Perform a current state analysis to identify workloads and sizing
o Spend time up front to architect a solution that meets both current and future needs
o Design to deliver consistent performance, reliability, and scale
o Don’t undersize, don’t oversize, right size
o Start with a PoC, test, optimize, iterate, scale

Core Components
o MSSQL
▫ Performance and Scalability
o Utilize multiple drives for TempDB Log/Data and Database Log/Data
 Start with a minimum of 2 for small environments or 4 for larger
environments
 Look for PAGEIOLATCH_XX contention and scale number of drives
as necessary
o Utilize a 64KB NTFS allocation unit size for MSSQL drives
o Enabled locked pages in memory for MSSQL Server service account (NOTE:
if this setting is used the VM’s memory must be locked, only applies with
memory > 8GB)
o TempDB Data Files
 Set TempDB size between 1 and 10% of instance database sizes
 If number of cores < 8
 # of cores = # of data files
 If number of cores > 8
 Use 8 data files to being with
 Look for contention for in-memory allocation
(PAGELATCH_XX) and scale 4 files at a time until contention
is eliminated
o Database Data files
 Size appropriately and enable AUTOGROW respective to database
growth
 Do not AUTOSHRINK data and log files

MSSQL on Nutanix – Best Practices | 13

| 13
  At a maximum keep below 80% of disk capacity utilization
 Use multiple data file and drives
 Look for contention for in-memory allocation
(PAGELATCH_XX), if contention increase number of files
 Look for I/O subsystem contention (PAGEIOLATCH_XX), if 1
contention, spread the data files across multiple drives
4
o Trace flags
 Implement trace flag 1118 at startup to remove single page
allocations
 Implement trace flag 834 to enable large pages (for tier-1
performance)
o Utilize the MSSQL Server Best Practices Analyzer (BPA) to identify potential
issues
o Utilize fast file initialization
o Scale number of MSSQL VMs vs. number of MSSQL instances per VM
o More memory = higher performance, if seeing memory pressures, increase
VM memory
o Utilize a dedicated disk for Microsoft Page File
▫ Availability
o In most cases vSphere HA will provide an adequate level of availability and
uptime for non-mission critical/tier-1 applications
o For mission critical/tier-1 applications:
 MSSQL 2012: utilize AlwaysOn availability groups (preferred)
 MSSQL 2008 and prior: utilize log shipping or clustered MSSQL using
MSCS clusters
o Take consistent database snapshots/backups, frequency should be derived
from required RPOs
o Leverage native or 3rd party tools to manage backups (eg. Microsoft System
Center Data Protection Manager (DPM), etc.)
▫ Manageability
o Standardize, monitor and maintain
o Leverage a MSSQL application monitoring solution (eg. System Center, etc.)
o Create standardized MSSQL VM Templates
o Utilize consistent disk quantities and layout schemes for MSSQL VMs
o Join the MSSQL Server to the domain and use Active Directory for
authentication
o Leverage Contained Database Authentication (MSSQL 2012)

MSSQL on Nutanix – Best Practices | 14

| 14
 o Use named instances for MSSQL database instances, even when only
planning a single instance per VM
o For named instances, ensure application compatibility with dynamic ports,
otherwise set instance to use a fixed port
o VMware vSphere
▫ Follow VMware performance best practices
1
5
▫ Avoid CPU core oversubscription (for tier-1 workloads)
▫ For small MSSQL VMs keep vCPUs <= to the number of cores per each physical
NUMA node
▫ For wide MSSQL VMs size vCPUs to align with physical NUMA boundaries and
leverage vNUMA
▫ Keep vCPU numbers easily divisible by NUMA node sizes for easy scheduling
▫ Leave Hyperthreading sharing at the default policy (Any)
▫ Enable ‘High Performance’ host power policy
▫ Lock MSSQL VM memory (for tier-1 workloads)
▫ Size MSSQL VM memory using the following calculation:
o VM Memory = SQL Server Max Memory + ThreadStack + OS Memory + VM
Overhead
o Threadstack = SQL Max Worker Threads * 2MB (for x64)
▫ Use paravirtual SCSI Controllers and VMXNET3 Nics
▫ Use resource pools with correct share allocation
▫ Use DRS anti-affinity rules to keep MSSQL VMs apart
o Nutanix
▫ Use a single container
▫ Utilize appropriate model based upon compute and storage requirements
o Ideally keep working set in SSD and database size within node capacity
o Choose a model which can ideally fit the full database on a single node.
NOTE: for larger databases which cannot fit on a node, ensure there is ample
bandwidth between nodes
o Utilize higher memory node models for I/O heavy MSSQL workloads
▫ Create a dedicated consistency group with the MSSQL VMs and applications
▫ Leverage ‘Application Consistent Snapshots’ on the consistency group to invoke
VSS when snapshotting

Supporting Components
o Network
▫ Utilize and optimize QoS for NDFS and database traffic

MSSQL on Nutanix – Best Practices | 15

| 15
 ▫ Use low-latency 10GbE switches
▫ Utilize redundant 10GbE uplinks from each Nutanix node
▫ Ensure adequate throughput between Nutanix nodes and MSSQL VMs
▫ Check for any pause frames which could impact replication and VM communication
o Active Directory 1
▫ Utilize AD based authentication for MSSQL servers 6
o OS and Application Updates
▫ Schedule updates to be applied outside business hours to avoid performance
impacts
▫ Stagger updates in phases

MSSQL on Nutanix – Best Practices | 16

| 16
5. Best Practice Detail
With the MSSQL on Nutanix solution you have the flexibility to start small with a three nodes
and scale up incrementally a node, a block, or multiple blocks at a time. This provides the best
of both worlds–the ability to start small and grow to massive scale without any impact to
performance. 1
7
5.1. MSSQL Server Sizing

The following are typical scenarios for any database workload:

Table 1: Scenario Overview

Scenario Definition

OLTP – These workloads are transactional in nature and can heavy a good deal
Transactional of updates and inserts depending on the workload. Depending on size
and workload, write I/O and latency is extremely important for the
highest possible application performance. The quantity of transactions
processed per second is a key metrics for OLTP databases.
OLAP – These workloads are analytical in nature and rely a great deal on
Analytics & batched ETL or data loading and batched reporting. These workloads
Reporting are primarily sequential in nature and can require reading great
quantities of data. Sequential write performance is critical for data
ingest and quantity can be determined by load frequency and quantity.
These workloads are traditionally utilized for data warehousing and
reporting.

Below are some initial recommendations for MSSQL Server VM sizing based upon assumed
workload characterizations:

Table 2: Scenario Detail

Scenario vCPU Memory (GB) Disks

OLTP
Light 1–2 4 1 x 40GB (OS) + 2 x n GB (DB)
Medium 4 – 8+ 8 – 16+ 1 x 40GB (OS) + 2-4 x n GB (DB)
Heavy 8 – 16+ 24 – 192+ 1 x 40GB (OS) + 4+ x n GB (DB)
OLAP
Light 2 – 4+ 4 – 8+ 1 x 40GB (OS) + 2 x n GB (DB)
Medium 4 – 8+ 8 – 24+ 1 x 40GB (OS) + 2-4 x n GB (DB)
Heavy 8 – 16+ 24 – 192+ 1 x 40GB (OS) + 4+ x n GB (DB)

Note: These are recommendations for sizing and should be modified after a current state
analysis.

MSSQL on Nutanix – Best Practices | 17

| 17
VM Configuration

Use the following settings when configuring your base VM:

Table 3: VM Configuration

Parameter Configuration 1
8
Network adapter Vmxnet3
Storage adapter Minimum of 2 x PVSCSI (OS + DB), 4 for larger databases
VMware tools Latest installed
Memory Locked (preferred)
VM Logging Disabled
Latency Sensitivity High (vSphere 5.5 only)
Advanced VM Name: numa.vcpu.min
configuration parameters Value: For less than 8 vCPU set to number of vCPUs (eg. 4)

OS Optimization

Configure your Windows image to Microsoft Server best practices, with some of the following
additional recommendations:
o Set display to “Adjust for best performance”
o Enable locked pages in memory
o Enable fast file initialization for MSSQL Server Service Account
o Try to minimize utilization of Windows page files, if necessary, put the windows page file
on a dedicated disk

MSSQL Service Startup Options

Configure applicable startup options for MSSQL Server through the MSSQL Server
Configuration Manager. Below are some commonly utilized trace flags:

Table 4: MSSQL Service Startup Options

Trace Flag Description

-T1118 Removes all single page allocations

-T834 MSSQL will allocate all buffer pool memory at startup

MSSQL on Nutanix – Best Practices | 18

| 18
VM Placement

As mentioned, the MSSQL VMs will co-exist with other VMs running on the Nutanix platform.
DRS anti-affinity is utilized to split all of the MSSQL VMs amongst the ESXi hosts in the cluster.
This is leveraged to keep a 1:1 ratio between MSSQL VMs and ESXi nodes/Nutanix Controllers 1
to ensure the highest possible performance. 9
DRS anti-affinity rules for SQL Server VMs

SQL
SQL 1..N
1..N ... VM
VM 1..N
1..N SQL
SQL 1..N
1..N ... VM
VM 1..N
1..N SQL
SQL 1..N
1..N ... VM
VM 1..N
1..N

Hypervisor
Hypervisor Hypervisor
Hypervisor Hypervisor
Hypervisor

Cache
Cache Cache
Cache Cache
Cache
Cache Cache Cache
CVM
CVM CVM
CVM CVM
CVM
Storage
Storage Storage
Storage Storage
Storage
...

NDFS
Figure 8 MSSQL + VM Node Placement

VM Component Mapping

The figure below shows the mapping for the MSSQL VM’s storage. The quantity of VMDKs will
be driven by the estimated database size and I/O requirements. As mentioned above any
PAGEIOLATCH_XX waits would signify a need for more disks. A minimum of six dives for
MSSQL VMs is recommended, however will vary. More information on disk quantity and sizing
can be found below in the MSSQL Component Sizing section below.

SQL VM
SQL DB Disks
OS + SQL

Pagefile

[DATA|LOG] N
TempDB Data

TempDB Log

DB Data

DB Log

...

PV-SCSI Controller PV-SCSI Controller

nGB VMDK
80GB VMDK

nGB VMDK
nGB VMDK

nGB VMDK

nGB VMDK
nGB VMDK

...

NFS Datastore

RF2 CTR VM CTR N

SP1 (PCIe SSD,SATA SSD, SATA HDD)

Nutanix DFS (NDFS)

Figure 9 VM Component Mapping

MSSQL on Nutanix – Best Practices | 19

| 19
5.2. General Sizing
The following section covers the design decisions and rationale for MSSQL deployments on the
Nutanix platform.

Table 5: General Design Decisions

2
Item Detail Rationale 0
General - Nutanix
Minimum Size 3 x Nutanix nodes (3 ESXi hosts)  Minimum size requirement
Scale Approach Incremental modular scale  Allow for growth from PoC to
massive scale
Scale Unit Node(s)  Granular scale to precisely
meet the capacity demands
 Scale in n x node increments
VMware vSphere
Cluster Size Up to 12-24 ESXi hosts  Isolated fault domains
(Minimum of 3 hosts)
Clusters per vCenter Up to 2x24 or 4x12 host clusters  Task parallelization
 Isolated fault domains
Datastore(s) 1 x Nutanix datastore per cluster  Nutanix handles I/O
distribution/localization
 n-Controller model
Infrastructure  Small deployments: Shared  Dedicated infrastructure
Services cluster cluster for larger
 Large deployments: deployments (best practice)
Dedicated cluster
Nutanix
Cluster Size Up to 24-48 nodes  Isolated fault domains
Storage Pool(s) 1 x Storage Pool  Standard practice
 ILM handles tiering
Container(s)  1 x Container for VMs and  Standard practice
Data

MSSQL on Nutanix – Best Practices | 20

| 20
5.3. MSSQL Storage Sizing

The following section covers the storage sizing and considerations for running MSSQL on
Nutanix. NOTE: It is always a good practice to add a buffer for contingency and growth.
2
Step 1 - Calculate the estimated required storage: 1
Required Storage = (Number of Databases x Average Database Size) x
Average growth rate per year

For example, if there are 10 databases with an average size of 500 GB and a 20% growth rate
per year:

Required Storage (year 1) = (10 Databases * 500 GB) * 1.2 = 6,000 GB = 6 TB

Step 2 - Calculate the total required number of Nutanix nodes by storage capacity:

Total Number of Nutanix Nodes by Storage Capacity = (Required Storage x

Nutanix Replication Factor) / Storage per Node

For example, if there is 20 TB of required storage, the replication factor is 2 (default) and there
is 4 TB of storage per node (varies per model):

Number of Nutanix Nodes by Storage Capacity = (20 TB * 2) / 4 TB = 10 Nodes

Step 3a - Calculate the initial storage capacity of Nutanix nodes:

With Nutanix you have the ability to start with a certain capacity and incrementally scale
capacity as needed. If all nodes are deployed initially this step can be skipped.

Initial Storage Capacity = (Total Required Number of Nutanix Nodes * %

Deployed Initially) * Storage per Node

For example, if there are 10 Nutanix nodes required and 50% of nodes initially deployed and 4
TB of storage per node (varies per model):

Initial Storage Capacity = (10 Nodes * 50%) * 4 TB = 20 TB

MSSQL on Nutanix – Best Practices | 21

| 21
Step 3b - Calculating the necessary duration to scale Nutanix nodes:

If all nodes are deployed initially this step can be skipped.

Initial Storage Data Duration = Initial Storage Capacity / Daily 2

average ingest rate in GB
2
For example, if there are 5 nodes initially deployed and 4 TB of storage per node (varies per
model) providing 20 TB of initial capacity and 500 GB of ingest data daily:

Initial Storage Data Duration = 20 TB / 500 GB/day = 40 days

Meaning that after 40 days the required storage capacity would breach the initial storage
capacity and Nutanix nodes must be scaled to facilitate the demand up to the full 100% of the
required nodes based upon storage capacity.

MSSQL on Nutanix – Best Practices | 22

| 22
5.4. Nutanix

The Nutanix Virtual Computing Platform provides an ideal combination of both high-
performance compute with localized storage to meet any demand. True to this capability, this
document contains zero reconfiguration of, or customization to, the Nutanix product to optimize
for this use case. 2
3
The figure below shows a high-level example of the relationship between a Nutanix block, node,
storage pool and container

Container 1 -
CTR-RF2-VM-01 ... Container N

Storage Pool – SP01

HDD

HDD

HDD

HDD

HDD

HDD
HDD

HDD
SSD

SSD

SSD

SSD

SSD

SSD
SSD

SSD
Nutanix Nutanix Nutanix Nutanix ... Nutanix Nutanix Nutanix Nutanix
Node Node Node Node Node Node Node Node
Nutanix Block Nutanix Block
Figure 10 Nutanix Component Architecture

The table below shows the Nutanix storage pool and container configuration.

Table 6: Nutanix Storage Configuration

Name Role Details

SP01 Main storage pool for all data All Disks
CTR-RF2-VM-01 Container for all VMs and data ESXi – Datastore

MSSQL on Nutanix – Best Practices | 23

| 23
5.5. Network

Designed for true linear scaling, a Leaf Spine network architecture is leveraged. A Leaf Spine
architecture consists of two network tiers: an L2 Leaf and an L3 Spine based on 40GbE and
non-blocking switches. This architecture maintains consistent performance without any
throughput reduction due to a static maximum of three hops from any node in the network. 2
4
The figure below shows a design of a scale-out Leaf Spine network architecture which provides
20Gb active throughput from each node to its L2 Leaf and scalable 80Gb active throughput from
each Leaf to Spine switch providing scale from 1 Nutanix block to thousands without any impact
to available bandwidth

Figure 11 Leaf Spine Network Architecture

MSSQL on Nutanix – Best Practices | 24

| 24
6. Validation & Benchmarking
The solution and testing provided in this document was completed with MSSQL 2008 R2
deployed on VMware vSphere 5.1 on Nutanix Virtual Computing Platform.
2
The SQLIO benchmark was leveraged to detail I/O subsystem performance – the HammerDB
“tpc-c like” benchmark was leveraged to detail the transaction performance on the Nutanix 5
appliance.

6.1. Environment Overview

A Nutanix NX-3450 was utilized as the target environment and provided all MSSQL VM hosting.
The Nutanix block was connected to an Arista 7050S top-of-rack switch via 10GbE.

Test
Test Client(s)
Client(s) Test
Test Target(s)
Target(s)

Sessions

HammerOra Test MSSQL Test Storage Benchmark

VMs Machines VMs (SQLIO/
SQLSIM)

Figure 12 Test Environment Overview

MSSQL on Nutanix – Best Practices | 25

| 25
Test Environment Configuration

Assumptions
o SQLIO size: 2x memory and 2x cache size
o SQLIOSim size: 2x memory and 2x cache size 2
o HammerDB size: 32 warehouses 6
o Disk configurations: 1,2,4,8

Hardware
o Storage/Compute: 1 Nutanix NX-3450
o Network: Arista 7050Q(L3 Spine)/7050S(L2 Leaf) Series Switches

Nutanix
o Version: 3.5

MSSQL Server Configuration

o OS: Windows Serve 2008 R2 x64
o CPU and Memory: 8 vCPU/8 GB
o Disk:
▫ 1 x 40GB (OS)
▫ 8 x 200GB (Data)
o Application Version: 2008 R2

MSSQLIO
o Version: 1.5
MSSQLSIM
o Version: 9.00.1399.05
HammerDB
o Version: 2.14

MSSQL on Nutanix – Best Practices | 26

| 26
6.2. Test Scripts & Configurations
o Add VMDKs to MSSQL VMs

#########################################################

## 2
7
## Script: AddDisks

## Author: Steven Poitras

## Description: Add n VMDKs to VM

## Language: PowerCLI

##

##########################################################

# Inputs

$vmPrefix = "TM4-MSSQL-"

$vms = Get-VM | where {$_.name -Match $vmPrefix}

$numDisks = 8

$diskCapacity = 100

# For each vm in vms add the disks

$vms | %{

$vm = $_

# Add disks

1..$numDisks | %{

$vm | New-HardDisk -CapacityGB $diskCapacity -Persistence

persistent

MSSQL on Nutanix – Best Practices | 27

| 27
o Partition and Format Disks

#########################################################

##

## Script: Partition Windows Disks 2

## Author: Steven Poitras 8
## Description: Partition and format added disks, this script is run
using the following syntax: diskpart /s <File Location\Name>.txt

## Example: diskpart /s X:\Scripts\SQL_DB_Diskpart.txt

## Language: Windows .txt

##

##########################################################

select disk 1

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=F

format fs=ntfs label="TempDB_64K_1" unit=64K quick

active

select disk 2

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=G

format fs=ntfs label="TempDB_64K_2" unit=64K quick

active

select disk 3

online disk

ATTRIBUTES DISK CLEAR READONLY

MSSQL on Nutanix – Best Practices | 28

| 28
create partition primary

assign letter=H

format fs=ntfs label="DB_64K_1" unit=64K quick

active
2
9
select disk 4

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=I

format fs=ntfs label="DB_64K_2" unit=64K quick

active

select disk 5

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=J

format fs=ntfs label="64K_1" unit=64K quick

active

select disk 6

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=K

format fs=ntfs label="64K_2" unit=64K quick

active

select disk 7

MSSQL on Nutanix – Best Practices | 29

| 29
online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=L
3
format fs=ntfs label="64K_3" unit=64K quick
0
active

select disk 8

online disk

ATTRIBUTES DISK CLEAR READONLY

create partition primary

assign letter=M

format fs=ntfs label="64K_4" unit=64K quick

active

MSSQL on Nutanix – Best Practices | 30

| 30
o SQLIO Powershell Function

#########################################################

##

## Script: NTNX-Run-SQLIO 3
## Author: Steven Poitras 1
## Description: Automate SQLIO testing using Powershell

## with a given set of params and output to csv

## Language: Powershell

##

#########################################################

function NTNX-Run-SQLIO {

<#

.NAME

NTNX-Run-SQLIO

.SYNOPSIS

This function does automated SQLIO runs

.DESCRIPTION

This function does automated SQLIO runs

.NOTES

Authors: VMwareDude

.LINK

www.nutanix.com

.PARAMETER Iterations

This parameter specifies the number of iterations to perform the test

.PARAMETER Duration

This parameter specifies the duration of each SQLIO run

.PARAMETER RorW

This parameter specifies if the test is read (R) or write (W).

Applicable values are R or W.

.PARAMETER RandOrSeq

MSSQL on Nutanix – Best Practices | 31

| 31
 This parameter specifies if the test is random or sequential.
Applicable values are random or sequential.

.PARAMETER BlockSize

This parameter specifies the blocksize of the IOs. Applicable values

are 8,64 or 512.
3
.PARAMETER Threads
2
This parameter specifies the number of threads to use

.PARAMETER OutstandingOps

This parameter specifies the number of outstanding ops to use

.PARAMETER TargetFile

This parameter specifies the target file or param file to use (eg.
test.dat)

.PARAMETER TargetType

This parameter specifies the type of target param or file.

Applicable values are param or file.

.PARAMETER OutputCSV

This parameter specifies the CSV file to output (eg. X:\results.csv)

.EXAMPLE

NTNX-Run-SQLIO -Iterations 10 -Duration 30 -RorW R -RandOrSeq random

-BlockSize 512 -Threads $_ -OutstandingOps 1 -TargetFile F:\test.dat -
TargetType file -OutputCSV $gOutputCSV

#>

Param(

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[int]$Iterations,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[int]$Duration,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[ValidateSet("R","W")]

[string]$RorW,

MSSQL on Nutanix – Best Practices | 32

| 32
 [Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[ValidateSet("random","sequential")]

[string]$RandOrSeq,

3
[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
3
[ValidateSet("8","64","512")]

[string]$BlockSize,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[string]$Threads,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[string]$OutstandingOps,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[string]$TargetFile,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[ValidateSet("param","file")]

[string]$TargetType,

[Parameter(Mandatory=$False,ValueFromPipeline=$True)]

[string]$TargetNum,

[Parameter(Mandatory=$True,ValueFromPipeline=$True)]

[string]$OutputCSV

BEGIN {

# Create array to store results data

MSSQL on Nutanix – Best Practices | 33

| 33
 $fResults = @()

# Perform string formatting for params

$fSecs = "-s$Duration"
3
$fRorW = "-k$RorW"
4
$fRandOrSeq = "-f$RandOrSeq"

$fBlockSize = "-b$BlockSize"

$fThreads = "-t$Threads"

$fOps = "-o$OutstandingOps"

$fTarget = if ($TargetType -eq "param") {"-F$TargetFile"}

else {$TargetFile}

$resultsCSV = $OutputCSV

} PROCESS {

1..$Iterations | %{

Write-Host "Running SQLIO with the following params: "

$fSecs $fRorW $fRandOrSeq $fBlockSize $fThreads $fOps $fTarget

$r = C:\SQLIO\SQLIO.EXE $fSecs $fRorW $fRandOrSeq

$fBlockSize $fThreads $fOps -LS -BN $fTarget

# Run command. NOTE: The output varies when using a

param file with multiple files.

if ($TargetType -eq "param") {

# Parse multi file output

if ($TargetNum -eq 2) {

# Formatting for 2 disk

$iops = $r.Split("`n")[14].Split(":")[1].Trim()

$mbps = $r.Split("`n")[15].Split(":")[1].Trim()

$lat = $r.Split("`n")[18].Split(":")[1].Trim()

MSSQL on Nutanix – Best Practices | 34

| 34
} elseif ($TargetNum -eq 4) {

# Formatting for 4 disk

$iops = $r.Split("`n")[18].Split(":")[1].Trim()

$mbps = $r.Split("`n")[19].Split(":")[1].Trim()
3
$lat = $r.Split("`n")[22].Split(":")[1].Trim()
5
} elseif ($TargetNum -eq 6) {

# Formatting for 6 disk

$iops = $r.Split("`n")[22].Split(":")[1].Trim()

$mbps = $r.Split("`n")[23].Split(":")[1].Trim()

$lat = $r.Split("`n")[26].Split(":")[1].Trim()

} else {

# Parse file output

$iops = $r.Split("`n")[10].Split(":")[1].Trim()

$mbps = $r.Split("`n")[11].Split(":")[1].Trim()

$lat = $r.Split("`n")[14].Split(":")[1].Trim()

$fResults += New-Object PSObject -Property @{

File = $TargetFile

NumDisks = $TargetNum

ReadWrite = $RorW

RanSeq = $RandOrSeq

BlockSize = $BlockSize

NumThreads = $Threads

OutstandingOps = $OutstandingOps

IOPS = $iops

Throughput = $mbps

Latency = $lat

Timestamp = $(Get-Date -Format "yyyy-MM-dd HH:mm:ss")

MSSQL on Nutanix – Best Practices | 35

| 35
 }

#Print results to display

write-host "Target: $fTarget IOPS: $iops Throughput: $mbps

Latency: $lat" 3
} 6

} END {

# Append/write results to CSV

$fResults | Export-Csv $OutputCSV -NoTypeInformation -

Append -Force

MSSQL on Nutanix – Best Practices | 36

| 36
o SQLIO Function Driver Script

#########################################################

##

## Script: NTNX-Run-MSSQLIO Driver Script 3

## Author: Steven Poitras 7
## Description: Automate parameter and test execution

## Language: Powershell

##

#########################################################

$gOutputCSV = "X:\TESTING\tm4sql201-results2.csv"

$gIterations = 5

$gDuration = 30

$gVarIterations = 1,8,16,32

$gFiles =
("J:\Testfile.Dat","file",1),("C:\SQLIO\param2d.txt","param",2),("C:\SQ
LIO\param4d.txt","param",4)

$gBlockSizes = "8","64","512"

$gRorW = "R","W"

$gRandOrSeq = "random","sequential"

# For each file

$gFiles | %{

# Set file vars

$tFile = $_[0]

$tFileType = $_[1]

# For each block size

$gBlockSizes | %{

# Set block size

$tBlockSize = $_

MSSQL on Nutanix – Best Practices | 37

| 37
 # For both R and W

$gRorW | %{

$tRorW = $_
3
8
# For random and sequential

$gRandOrSeq | %{

$tRandOrSeq = $_

# Perform test iterations

if ($tBlockSize -eq "512") {

#Run single op test

NTNX-Run-SQLIO -Iterations $gIterations -

Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 8 -OutstandingOps 1 -TargetFile $tFile -TargetType
$tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

NTNX-Run-SQLIO -Iterations $gIterations -

Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 16 -OutstandingOps 1 -TargetFile $tFile -
TargetType $tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

NTNX-Run-SQLIO -Iterations $gIterations -

Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 32 -OutstandingOps 1 -TargetFile $tFile -
TargetType $tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

} else {

$gVarIterations | %{

$tVarInt = $_

#Run var ops test

NTNX-Run-SQLIO -Iterations $gIterations -

Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 8 -OutstandingOps $tVarInt -TargetFile $tFile -
TargetType $tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

NTNX-Run-SQLIO -Iterations $gIterations -

Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 16 -OutstandingOps $tVarInt -TargetFile $tFile -
TargetType $tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

MSSQL on Nutanix – Best Practices | 38

| 38
 NTNX-Run-SQLIO -Iterations $gIterations -
Duration $gDuration -RorW $tRorW -RandOrSeq $tRandOrSeq -BlockSize
$tBlockSize -Threads 32 -OutstandingOps $tVarInt -TargetFile $tFile -
TargetType $tFileType -TargetNum $tFileNum -OutputCSV $gOutputCSV

} 3
} 9
}

MSSQL on Nutanix – Best Practices | 39

| 39
6.3. Benchmarks

SQLIO Benchmark

MSSQLIO is a performance benchmarking tool for benchmarking and comparing storage I/O 4
subsystem performance. NOTE: Make-a-file.exe should be used to generate the files used by 0
SQLIO with random real data to simulate a real workload, by default zeros are used.

The test benchmark consists of a few main types:

1. Write Data: Generate n GB of data (target = 2 x memory/cache size)
2. Read Data: Read n GB of data (target = 2 x memory/cache size)

For more information about MSSQLIO visit the Microsoft SQLIO Subsystem Benchmark Tool
page: http://www.microsoft.com/en-us/download/details.aspx?id=20163

SQLIOSim Benchmark

SQLIOSim is a performance benchmarking tool for simulating database I/O subsystem

workload.

The test benchmark consists of a few main phases:

1. File Creation: Generate n GB of data (target = 2 x memory/cache size)
2. I/O Simulation: Performing random access, bulk reader and read ahead operations
3. Perform Checkpoint and Page Audit: Checkpoint and perform scan of data

For more information about MSSQLIO visit the Microsoft SQLIOSim Utility support site:
http://support.microsoft.com/kb/231619

HammerDB Benchmark

HammerDB is a performance benchmarking tool for simulating database I/O subsystem

workload.

The test benchmark consists of a few main phases:

1. Schema Creation: Creation of the “TPC-C” schema
2. “TPC-C” Workload Simulation: Perform a “TPC-C” like workload simulation

For more information about MSSQLIO visit the HammerDB Sourceforge site:
http://hammerora.sourceforge.net/

MSSQL on Nutanix – Best Practices | 40

| 40
6.4. How to Interpret the Results

SQLIO

Evaluation is quantified using the following metrics: 4

o Read Throughput (MBps): The amount of read data per second 1
o Write Throughput (MBps): The amount of write data per second
o Read I/O (IOPS): The amount of read I/O operations per second
o Write I/O (IOPS): The amount of write I/O operations per second
o Read Latency (ms): The time to perform a read operation
o Write Latency (ms): The time to perform write operation

Based user experience and industry standards, recommend ideal ranges for these values are
kept below the following values:

Table 7: SQLIO Metric Values

Metric Value Rationale

Read Throughput (MBps)
Write Throughput (MBps)
Read I/O (IOPS)
Write I/O (IOPS)
Read Latency (ms)
Write Latency (ms)
Varies Varies by block size and outstanding ops
Varies Varies by block size and outstanding ops
Varies Varies by block size and outstanding ops
Varies Varies by block size and outstanding ops
<1 Acceptable read response time
< 3-5 (varies) Acceptable write response time

SQLIOSim

Evaluation is quantified using the following metrics:

o Read Latency (ms): The time to perform a read operation
o Write Latency (ms): The time to perform write operation

Based user experience and industry standards, recommend ideal ranges for these values are
kept below the following values:

Table 8: SQLIOSim Metric Values

Metric Value Rationale

Read Latency (ms) <1 Acceptable read response time
Write Latency (ms) < 3-5 (varies) Acceptable write response time

MSSQL on Nutanix – Best Practices | 41

| 41
HammerDB

Evaluation is quantified using the following metrics:

o Initial Write (KB/sec): The number of KB written per second
o Rewrite (KB/sec): The number of KB rewritten per second
4
Based user experience and industry standards, recommend ideal ranges for these values are 2
kept below the following values:

Table 9: HammerDB metric values

Metric Value Rationale

Transactions per Minute (TPM) Varies Varies based upon number of vUsers
Number of Orders per Minute (NOPM) Varies Varies based upon number of vUsers

Performance Graphs

The performance graphs show a plot of the data as well as a trend line. Below various aspects
of the graphs are highlighted:

MSSQL on Nutanix – Best Practices | 42

| 42
 4
3

MSSQL on Nutanix – Best Practices | 43

| 43
7. Results
7.1. SQLIO – Single VM

The single VM SQLIO results highlighted the performance for a single VM with 8 vCPU and 8 4
GB memory running on the Nutanix platform. The size of the test data file generated using 4
Make-a-file.exe was 16 GB to double Windows memory and Nutanix Cache sizes.

Over 20,000 SQLIO test runs were completed to stress the environment and eliminate any
variance or performance outliers. The platform showed ample IOPS and throughput based upon
a varying set of block sizes and workloads.

NOTE: the results displayed below are for a single VM running on a single Nutanix node. By
default a Nutanix deployment will have a minimum of 3 nodes and the results below are not
applicable to the full platform’s performance.

Aggregate Test Results

The figure below shows the SQLIO IOPS and throughput performance based upon the block
size. Results showed the single SQLIO VM was able to facilitate ~35,000 random IOPS with a
8 KB block size and ~16,000 IOPS with a 64 KB block size. Sequential I/O peaked with a 512
KB block size at ~1,200 MBps (1.2 GBps) and was ~950 MBps (.95 GBps) with a 64 KB block
size.

SQLIO - Single VM
Aggregate I/O by Block Size

Throughput (MBps)
40,000 1,500
30,000 1,000
IOPS

20,000
10,000 500
0 0
8 KB 64 KB 512 KB
Block Size

Figure 13 SQLIO - Single VM - Aggregate I/O by Block Size

MSSQL on Nutanix – Best Practices | 44

| 44
The figure below shows the SQLIO operation latency for read and write workloads based upon
the block size. Results showed read latency kept consistently under 1ms and in microseconds
for 8 KB and 64 KB block sizes and ~1 ms for 512 KB. Write latency was consistent at 1 ms for
8 and 64 KB block sizes and ~5 ms for 512 KB.

SQLIO - Single VM 4
5
Latency by Block Size
6
5
Latency (ms)

4
3
Read
2
1 Write
0
8 KB 64 KB 512 KB
Block Size

Figure 14 SQLIO - Single VM - Latency by Block Size

The figure below shows the SQLIO operation latency for read and write workloads based upon
the number of outstanding ops. Results showed read latency kept consistently under 1ms and
in microseconds under 16 outstanding ops. Write latency varied between <1 ms and 5 ms
during the tests.

SQLIO - Single VM
Latency by Outstanding Ops
6
5
Latency (ms)

4
3
Read
2
1 Write
0
1 8 16
Oustanding Ops

Figure 15 SQLIO - Single VM - Latency by Outstanding Ops

MSSQL on Nutanix – Best Practices | 45

| 45
The Controller VM (CVM) hosting the VM running the SQLIO workload (CVM-B - highlighted
with * below), peaked at ~91% CPU utilization during the testing. All other CVM CPU
utilizations stayed consistently ~15-20% which is expected during idle operation. Aggregate
cluster CPU utilization peaked at 22%.

This highlights the ability to incrementally scale out the number of SQL servers and maintain
performance as the number of databases/instances scale, as well as the ability to eliminate any 4
noisy-neighbor scenarios. 6

SQLIO - Single VM
Controller VM CPU Utilization
100
CPU Utilization (%)

80 CVM-A
60 CVM-B*
40 CVM-C
20 CVM-D

0
Figure 16 SQLIO - Single VM - Controller VM CPU Usage

MSSQL on Nutanix – Best Practices | 46

| 46
Random I/O Tests

The SQLIO random tests were run to simulate a random I/O workload on NDFS. Random I/O
workloads are important from the random nature of database data file I/Os. Testing showed
ample IOPS numbers for a single VM which will scale linearly with the number of SQL servers 4
or Nutanix nodes. 7

Table 10: SQLIO Random I/O Results – Single VM

Block Size Read I/O Write I/O Read Throughput Write Throughput
(KB) (IOPS) (IOPS) (MBps) (Mbps)
Random
8 35,968 16,437 281 281
64 15,496 6,236 919 919
512 2,433 1,145 1,216 1216

The figure below shows the SQLIO random I/O test performance for read and write workloads
based upon block size. IOPS peaked using a smaller (8 KB) block size.

SQLIO - Single VM
Random IOPS by Block Size
40,000
35,000
30,000
25,000
IOPS

20,000 Read
15,000 Write
10,000
5,000
0
512 KB 64 KB 8 KB
Figure 17 SQLIO - Single VM - Random IOPS by Block Size

MSSQL on Nutanix – Best Practices | 47

| 47
The figure below shows the SQLIO random I/O test performance for read and write workloads
based upon the number of outstanding ops. Results showed that for larger block sizes (64 &
512 KB) only one outstanding operation was necessary to keep peak IOPS. For smaller block
sizes (8 KB) eight outstanding ops were necessary to keep peak IOPS.

SQLIO - Single VM 4
8
Read IOPS by Outstanding Ops
40,000
35,000
30,000
25,000
IOPS

20,000 512 KB
15,000
10,000
64 KB
5,000 8 KB
0
1 8 16
Outstanding Ops

Figure 18 SQLIO - Single VM - IOPS by Outstanding Ops

The figure below shows the SQLIO random I/O test performance for read and write workloads
based upon the number disks. Results showed IOPS increasing with the increased disk
quantity and showed a minimum of 4 disks is necessary to provide ideal performance.

SQLIO - Single VM
IOPS by Disk Quantity
40,000
35,000
30,000
25,000
IOPS

20,000
15,000 Read
10,000 Write
5,000
0
1 2 4 6
Disk Quantity

Figure 19 SQLIO - Single VM - IOPS by Disk Quantity

MSSQL on Nutanix – Best Practices | 48

| 48
Sequential Tests

The SQLIO sequential tests were run to simulate a sequential I/O workload on NDFS.
Sequential I/O workloads are important from the sequential nature of database log file and
TempDB I/Os. Testing showed ample throughput numbers for a single VM which will scale
linearly with the number of SQL servers or Nutanix nodes. 4
9
Table 11: SQLIO Sequential I/O Results – Single VM

Block Size Read I/O Write I/O Read Throughput Write Throughput
(KB) (IOPS) (IOPS) (MBps) (Mbps)
Sequential
8 35,905 15,341 280 280
64 15,496 11,120 968 968
512 2,391 1,741 1,741 1,195

The figure below shows the SQLIO sequential I/O test performance for read and write
workloads based upon block size. Throughput peaked using a larger (512 KB) block size,
however, showed ample throughput using 64 KB block sizes.

SQLIO - Single VM
Sequential Throughput by Block Size
1,400
Throughput (MBps)

1,200
1,000
800
Read
600
400 Write
200
0
8 KB 64 KB 512 KB
Figure 20 SQLIO - Single VM - Sequential Throughput by Block Size

MSSQL on Nutanix – Best Practices | 49

| 49
The figure below shows the SQLIO sequential I/O test performance for read and write
workloads based upon the number of outstanding ops. Results showed that for larger block
sizes (64 & 512 KB) only one outstanding operation was necessary to keep peak throughput.
For smaller block sizes (8 KB) eight outstanding ops were necessary to keep peak throughput.

SQLIO - Single VM 5
0
Read Throughput by Outstanding Ops
1,400
Throughput (MBps)

1,200
1,000
800
512 KB
600
400 64 KB
200 8 KB
0
1 8 16
Outstanding Ops

Figure 21 SQLIO - Single VM - Read Throughput by Outstanding Ops

The figure below shows the SQLIO sequential I/O test performance for read and write
workloads based upon the number disks. Results showed throughput increasing with the
increased disk quantity primarily for write I/O and showed a minimum of 4 disks is necessary to
provide ideal performance.

SQLIO - Single VM
Sequential Throughput by Disk Quantity
1,400
Throughput (MBps)

1,200
1,000
800
600 Read
400
Write
200
0
1 2 4 6
Disk Quantity

Figure 22 SQLIO - Single VM - Sequential Throughput by Disk Quantity

MSSQL on Nutanix – Best Practices | 50

| 50
8. Further Research
As part of its continuous determination to deliver the best possible solutions, Nutanix will
continue to research into the following areas:
o Performance optimizations.
5
o Scale testing. 1
o HammerDB TPC-C testing.
o Detailed use-case application.
o Joint solutions with partners.

MSSQL on Nutanix – Best Practices | 51

| 51
9. Conclusion
The Nutanix platform offers the ability to run both MSSQL and VM workloads simultaneously on
the same platform. Density for MSSQL deployments will be primarily driven by the database’s
CPU and storage requirements. Test validation has shown it is preferred to increase the 5
number of MSSQL VMs on the Nutanix platform, as compared to scaling the number of SQL
instances, to fully take advantage of its performance and capabilities. From an I/O standpoint 2
the Nutanix platform was able to handle the throughput and transaction requirements of a
demanding MSSQL server given NDFS localized I/O and server attached flash.

A single VM running SQLIO was able to get over 35,000 random IOPS and over 1.2 GBps of
sequential throughput. During the tests latencies peaked at 1 ms for read, however were
normally in the microseconds range. Write latencies were at 1 ms for 8 KB and 64 KB block
sizes and peaked at <5 ms for 512 KB.

Sizing for the pods was determined after careful consideration of performance as well as
accounting for additional resource for N+1 failover capabilities.

The MSSQL on Nutanix solution provides a single high-density platform for MSSQL, VM hosting
and application delivery. This modular pod based approach enables these deployments to
easily be scaled.

MSSQL on Nutanix – Best Practices | 52

| 52
10. Appendix: Configuration

Hardware
o Storage / Compute 5
▫ Nutanix NX-3450 3
o Per node specs (4 nodes per 2U block):
 CPU: 2x Intel Xeon E5-2670
 Memory: 256 GB Memory
 SSD: 2x400 GB Intel S3700
 HDD: 4 x 1 TB SATA Drives
o Network
▫ Arista 7050Q - L3 Spine
▫ Arista 7050S - L2 Leaf

Software
o Nutanix
▫ Version: NOS 3.5
o Windows Server
▫ 2008 R2
o MSSQL
▫ 2008
o Infrastructure
▫ ESXi 5.1.0 patch 2
▫ vCenter 5.1.0 patch 2

VM
o Nutanix Controller
▫ CPU: 8 vCPU
▫ Memory: 48 GB
o SQLIO Server Configuration:
▫ OS: Windows Server 2008 R2
▫ 4 vCPU / 8 GB Memory

MSSQL on Nutanix – Best Practices | 53

| 53
11. References
11.1. Table of Figures

Figure 1 Nutanix Node Architecture ........................................................................................... 7 5

4
Figure 2 Nutanix Architecture ..................................................................................................... 7

Figure 3 Elastic Deduplication Engine ........................................................................................ 8

Figure 4 MSSQL on Nutanix Conceptual Arch ........................................................................... 9

Figure 5 NDFS I/O Path ............................................................................................................10

Figure 6 Data IO Detail .............................................................................................................11

Figure 7 Data IO Detail Expanded.............................................................................................11

Figure 8 MSSQL + VM Node Placement ...................................................................................19

Figure 9 VM Component Mapping.............................................................................................19

Figure 10 Nutanix Component Architecture...............................................................................23

Figure 11 Leaf Spine Network Architecture ...............................................................................24

Figure 12 Test Environment Overview ......................................................................................25

Figure 13 SQLIO - Single VM - Aggregate I/O by Block Size ....................................................44

Figure 14 SQLIO - Single VM - Latency by Block Size ..............................................................45

Figure 15 SQLIO - Single VM - Latency by Outstanding Ops ....................................................45

Figure 16 SQLIO - Single VM - Controller VM CPU Usage .......................................................46

Figure 17 SQLIO - Single VM - Random IOPS by Block Size....................................................47

Figure 18 SQLIO - Single VM - IOPS by Outstanding Ops ........................................................48

Figure 19 SQLIO - Single VM - IOPS by Disk Quantity .............................................................48

Figure 20 SQLIO - Single VM - Sequential Throughput by Block Size .......................................49

Figure 21 SQLIO - Single VM - Read Throughput by Outstanding Ops .....................................50

Figure 22 SQLIO - Single VM - Sequential Throughput by Disk Quantity ..................................50

MSSQL on Nutanix – Best Practices | 54

| 54
11.2. Table of Tables

Table 1: Scenario Overview ......................................................................................................17

Table 2: Scenario Detail ............................................................................................................17

5
Table 3: VM Configuration ........................................................................................................18 5
Table 4: MSSQL Service Startup Options .................................................................................18

Table 5: General Design Decisions ...........................................................................................20

Table 6: Nutanix Storage Configuration ....................................................................................23

Table 7: SQLIO Metric Values...................................................................................................41

Table 8: SQLIOSim Metric Values.............................................................................................41

Table 9: HammerDB metric values............................................................................................42

Table 10: SQLIO Random I/O Results – Single VM ..................................................................47

Table 11: SQLIO Sequential I/O Results – Single VM ...............................................................49

MSSQL on Nutanix – Best Practices | 55

| 55
12. About the Author
Steven Poitras is a Solutions Architect and Technology Evangelist on the Engineering team at
Nutanix, Inc. In this role, Steven helps design architectures combining applications with the
Nutanix platform creating solutions helping solve critical business needs and requirements and 5
disrupting the infrastructure space.
6
Prior to joining Nutanix he was one of the key solution architects at the Accenture Technology
Labs where he was focused on the Next Generation Infrastructure (NGI) and Next Generation
Datacenter (NGDC) domains. In these spaces he has developed methodologies, reference
architectures, and frameworks focusing on the design and transformation to agile, scalable, and
cost-effective infrastructures which can be consumed in a service-oriented or cloud-like
manner.

Follow Steven on Twitter at @StevenPoitras

About Nutanix

Nutanix is the recognized leader in the emerging Virtual Computing Platform market. The
Nutanix solution converges compute and storage resources into a single appliance, delivering a
powerful, modular building block for virtual datacenters. It incorporates the same advanced,
distributed software architecture that powers leading IT innovators such as Google, Facebook
and Amazon – but is tailored for mainstream enterprises and government agencies. The
Nutanix solution enables easy deployment of any virtual workload, including large-scale virtual
desktop initiatives (VDI), development/test apps, big data (Hadoop) projects and more. Nutanix
customers can radically simplify and scale out their datacenter infrastructures with cost-effective
appliances that can be deployed in under 30 minutes for rapid time to value.

Follow the Nutanix blogs at http://www.nutanix.com/blog/

Follow Nutanix on Twitter at @Nutanix

MSSQL on Nutanix – Best Practices | 56

| 56
 5
7

MSSQL on Nutanix – Best Practices | 57

| 57