You are on page 1of 38

Technical Paper

60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide

60TB* Microsoft SQL Server Fast Track
Data Warehouse Reference Architecture
(*60TB Maximum User Data, 25TB Certified)

Installation and Configuration Guide
Content
1

2

3

4

5

6

7

8
9

Page 1 of 38

Introduction
1.1
Intended Audience and Document Version
1.2
Document Version
Introduction to Fast Track Data Warehouse
2.1
Microsoft SQL Server Fast Track Data Warehouse
2.2
Fast Track Conceptual Overview
2.3
Paradigm Shift
2.4
Enabling New Scenarios
Performance Overview
3.1
PRIMERGY RX300 S7 and Violin Memory 6232
3.2
Hardware Overview
3.3
Conclusion
Installation and Configuration Instructions
4.1
Installing HBA Cards and Memory Modules
4.2
Connecting Compute Node to Storage Array
4.3
Operating System Installation
Storage Array Installation
5.1
HBA Drivers
5.2
Configuring MPIO
5.3
Storage Array Configuration
5.4
Adding Windows Volumes
5.5
SQL Server Installation
Checklists
6.1
Operating System
6.2
Storage
6.3
SQL Server
High Availability Scenario
7.1
Local High Availability
7.2
Remote High Availability
Bill of Materials
Appendix
9.1
Literature and Links
9.2
Abbreviations

2
3
3
4
4
5
6
7
8
8
9
12
14
14
15
16
19
19
20
21
27
30
35
35
35
35
36
36
36
37
38
38
38

http://www.fujitsu.com/fts/fasttrack

Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide

1

Introduction

A data warehouse (DW) is a large store of data accumulated from a wide range of sources. The stored
data is analyzed for trend analysis, business intelligence reporting, and various types of predictive
analysis. With today’s never ending data growth and complexity, it is becoming a tedious job for IT
professionals to balance capacity and performance within the data warehouse system. A modern
data warehouse has to continually balance growing data volumes, ETL (data load) requirements,
OLAP query complexity with ever growing number of users. DW systems based on disk storage
require a massive number of disk drives to deliver the required performance. Further they usually
require discipline in maintaining disk groups to maintain desired performance. The industry has
responded with “in-memory” DW solutions using column compression. Unfortunately the new
in-memory, No SQL DW solutions require a complete ground up rebuild of the DW solution. They are
unproven in terms of scale and limited by the size of DRAM present in the servers which are hosting
such solutions. When high availability and disaster recovery and persistence come into play, these
solutions tend to fall short. Fujitsu™, Violin Memory™ and Microsoft® deliver an all-memory, fully
persistent and highly available solution in 5U which does not require new platform, training or
re-integration of new software. The result is a lower operating cost, lower datacenter footprint, lower
power consumption and processing speed equal or approaching “in-memory” DW solutions. Fujitsu™
and Violin Memory™ all-memory solutions scale from 20TB (5U) to 240TB (20U) while preserving
most of the infrastructure investment along the way.
Fujitsu™ all-memory SQL DW solutions
address challenges in creating an optimal
balance with CPU, memory, I/O and storage.
At the same time these solutions
dramatically reduce ETL and maintenance
requirements on the system. All of your data
is located in DRAM on the Server, DRAM in
storage or flash memory in storage. There is
no spinning media in the solution. Fujitsu™ ,
Violin Memory™ and Microsoft® jointly
developed guidelines and design principles
to assist customers in designing and
implementing a balanced configuration
specifically for Microsoft SQL Server® Data Warehouse workloads to achieve “out-of-box” scalable
performance.

Page 2 of 38

http://www.fujitsu.com/fts/fasttrack

Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide

1.1

Intended Audience and Document Version

This white paper has two parts. Part one is intended for CIOs, CTOs, IT planners, solution architects,
database administrators, storage administrators, and business intelligence users who are
evaluating, planning, and deploying a next generation of the predicable data warehouse. The
second part is intended for a technical audience and provides step by step setup and configuration
instructions.
1.2

Document Version

Version
1.0
1.1

Page 3 of 38

Date
2013-04-12
2013-05-02

Comments
First Released Version
Minor Corrections

http://www.fujitsu.com/fts/fasttrack

no separation of data. the RX300 will also ensure the demands of Page 4 of 38 http://www. at any time. Throughput will not degrade dependent upon usage patterns or data locality. In a traditional system it is the growth or change in usage patterns that causes the most challenges over the life of the system. delivering easy to follow setup instructions and predictable performance as measured by IOPS and throughput. Each configuration has been thoroughly tested and rated for performance using both throughput certified and capacity certified ratings. tables. This consistency allows CIOs and technical staff to plan for and maintain performance levels stipulated by an SLA with ease. Fast Track is intended to provide an out of the box experience that optimizes the utilization of hardware implemented in a data warehouse solution. Violin solutions overcome these issues by presenting the array as one entire block of flash storage. The Microsoft Fast Track Data Warehouse built by Fujitsu™ and Violin Memory™ is a robust solution to this problem. As users come and go the locality of the access points changes.fujitsu. It supports up to 16 SSDs or Hard Disks. files or LUNs. This causes hot spots and unpredictable performance in production environments. Ongoing maintenance and ongoing troubleshooting is still required.or SSD-based solutions. RAID performance degradation and LUN striping issues. Setup. sustainable performance. still can be affected by data locality. In contrast. The goal is to provide predictable hardware performance and to remove the guess work when choosing a hardware solution for a data warehouse implementation. Each open query to the system could interact with multiple tables. partitions or data files further multiplying the total number of concurrent access points to storage. Violin Memory flash storage arrays enhance the predictability of the Fast Track solution by delivering the same performance (throughput) regardless of the number of threads. There is no LUN striping to architect. configuration and management of storage tier in the Violin architecture is significantly faster and simpler than disk. users. As the databases grow in size so does the administration time and complexity. up to 7 PCIe Gen 3 cards and up 768GB RAM. SSDs. All data is securely balanced across all of the available storage inside of the array and all data is equally accessible.com/fts/fasttrack . Thanks to top performance of the new Intel® Xeon® E5 product family and the available upgrade kits. anywhere in the array. As the number of users increases.1 Introduction to Fast Track Data Warehouse Microsoft SQL Server Fast Track Data Warehouse The Microsoft Fast Track Data Warehouse is a combination of Microsoft SQL Server software running on prescribed hardware configurations that have been specifically tested and approved for data warehouse workloads by Microsoft to meet various levels of sustained throughput. This has lead decision makers to look for alternatives to their current infrastructure in order to ensure the availability of consistent. so does the number of concurrent open threads to storage. optimized for all types of business applications by focusing on versatility and scalability. Combining multiple Violin Memory storage arrays also results in linear scalability of the storage component of the system and DW. log and temp space or tiering software to admin and all LUNs perform the same regardless of locality. The Fujitsu PRIMERGY RX300 S7 used in this solution is a 2U dual socket rack server.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 2 2. like HDDs. Violin’s flash technology goes beyond SSDs and other implementations of flash that are still bound to a hard disk drive (HDD) architecture. which allows administrators to avoid chasing periodic or systemic degradation issues.

This guide focuses on the 60 TB maximum user data / 25 TB certified configuration. LUNs are simply used as a logical organization unit for ease of administration. Violin’s unique offering of an all-flash array is specifically designed for high speed random access to any data at any time which eliminates bottlenecks. Fujitsu also has a smaller (20 TB Max) and larger (240 TB Max) configuration based upon the Violin Memory 6212 and 6232 storage arrays. A core requirement of Fast Track reference architectures is to align CPU bandwidth to that of the storage system. There are no physical moving parts in the compute or storage layer. The whole solution is a 5U design delivering the performance normally measured in racks. This is the first all-memory data warehouse reference architecture. This drastically reduces the chance of failure due to moving parts wearing out over time. Fast Track is a set of hardware configurations known as a reference architecture that have been tested and approved by Microsoft to deliver consistent and predictable performance. hot spots and data segregation requirements. Moving data or rearranging large sections of data results in time consuming tasks and frequently cause a downtime for the system. This makes the time consuming load tasks mostly irrelevant as the array delivers the same high performance for both sequential and random I/O requests.2 Fast Track Conceptual Overview The goal of the Fast Track data warehouse system is to achieve the required performance while remaining balanced at each layer. This eliminates the need to plan for different LUN allocations or leave large portions of purchased space unused. the reference also addresses the staging and loading procedures to allow for predictable results. Rotationally bound systems are also more difficult to maintain over time. 2. SQL Server settings. the array will deliver the same performance if it is hosting one huge LUN or hundreds of smaller LUNs. Page 5 of 38 http://www.fujitsu. While the focus of the Fast Track system is on the report delivery side. The goal now is to import as quickly and simply as possible. This is accomplished through using specific and defined OS configurations. The total aggregate IOPS for 4K blocks will be the same for the life of the array. The Fast Track system then provides a series of load steps and procedures to achieve continued success. It is intended to help avoid the common pitfalls of improperly designed and configured systems and to bring predictability into your data warehouse performance. The Fujitsu™ and Violin Memory™ Fast Track DW is built with no moving parts so that each piece of data is equally accessible at any moment. The goal is no longer to produce physically sequential data on spinning media. Changing one piece of data could cause an entire day to be reloaded. Many users wanting to touch all data from the same day at the same time could cause one set of spindles to degrade in performance. Right sizing seeks to deploy sufficient hardware to achieve performance goals without deploying more hardware than is required to get the job done. regardless of how long that takes.com/fts/fasttrack . This allows customers to take the guesswork out of determining hardware as the approved hardware configurations have already been rated for certain sized data warehouses. loading procedures and storage configurations which will be outlined later in this paper. The RX300 S7 can grow along with the DW solution to certain extend preserving the investment in the server. With this patented design.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide tomorrow are met.

In classic data warehouses this is required to achieve maximum and reliable speeds. Page 6 of 38 http://www. It is internally designed to scale out I/O operations to maximize concurrent usage of internal flash memory through parallelization down to the bit level. then partition switched into the final table which is a meta-data operation. then loaded again to another staging table with just one core (to achieve true sequential writes). Figure 1: Violin Memory Architecture To achieve the optimal performance of storage. Best practices for spinning media provide the best likelihood for success but do not guarantee it. This was created to satisfy the requirement for predictable throughput during range scans (sequential reads) which is the typical workload of a data warehouse system. then ordered. Only an all-flash system can deliver both the quickest and easiest load times and deliver guaranteed bandwidth regardless of the number of users.3 Paradigm Shift Every classic data warehouse requires time consuming and complex steps to sequentially load data into a data warehouse in order to achieve true physically sequential layouts of data. sequential loading of data is not necessary. Violin Memory arrays are able to achieve this by writing each I/O block to all VIMMs (memory modules) that comprise the array. there is no need to minimize movement of read/write heads. These steps included single threaded loading steps.com/fts/fasttrack .fujitsu. An all-flash array removes this requirement and opens up the administrators to take the quickest path with parallel loads and reloading data at will. The result is that the system delivers the same high level of performance for both sequential and random I/O reads and writes regardless of the usage pattern. This is optimal when designing for spinning media. It traded overall loading-speed for a physically sequential data layout to minimize fragmentation and allow for physically sequential reads. when deployed on spinning media. or when usage drifts from pre-defined requirements. Many DW administrators understand this can take hours or days. especially over the multi-year life span of a data warehousing system. data is loaded into a primary staging table with all cores (to fully utilize the cores and LUNs). With Violin storage arrays. Another issue comes when many users need simultaneous access to the same data. The array can handle random patterns and sequential patterns equally well and since there are no moving parts in the underlying storage.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 2. queries or data sources in use. All data gets written twice and once via a single core so that no matter how large the system is it is throttled to the speed of one core. All LUNs are spread evenly over all internal storage components providing maximum speed at all times. The intent was to try to minimize movement of the read/write heads on the disks. Each 4K block is written to all VIMMs within the RAID group. The Violin 6232 has 12 built in RAID groups and each RAID group is made up of 5 VIMMs: 1 for parity and 4 for data. ordering data between steps and utilizing multiple staging tables. in some cases filling their weekend with work.

fujitsu. Violin Memory does not have this problem.com/fts/fasttrack . Thanks to the power supply units with 94% efficiency and the new power management further add to the space and power savings from the storage side. 2. Inside the array every component is doubled up (except flash. migrations can also be a challenge as the administrator has to sequentially unload the data from the old system and perform a full reload into the new Fast Track system to minimize fragmentation. But. reliable and predictable performance. fast and the last step. Fragmentation is not a concern with an all-silicon. SSD drives or Flash based PCIe cards. There are no hot spots to migrate.4 Enabling New Scenarios SQL Server natively allows real time data updates while the DW is in full use. random IO storage device. Getting the data into the database should be simple. Now it is. With spinning media. This hot swap functionality includes rolling updates to the firmware on the system. Page 7 of 38 http://www. focusing on versatility and scalability. the system allows for parallel inserts of the data once and it is done. no tiering tools to configure and maintain and no costly add-on software to purchase. this typically is not done because this will cause significant performance degradation due to logical fragmentation. This will dramatically reduce the complexity of your ETL processes while increasing the speed of loading data by a significant factor. With a DW based on Violin storage. In addition to eliminating the long and complex loading process. With a Violin storage array the logical fragmentation level is irrelevant to throughput and the administrator is free to utilize the full tool in front of them. The Violin Memory storage array allows for any administration to occur at any time while still delivering stable. The Fujitsu PRIMERGY RX300 S7 in this solution is a dual socket rack server. This performance degradation is called “Write Cliff” and is present on all SSDs and PCIe Flash Cards. which is vRAID-protected) and hot swappable while the system is running and delivering full data rate speed.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide Violin storage arrays eliminate the ordering and second (single core) load steps. the data can be loaded directly in parallel into the DW real time without compromising the performance of the read operations. This further lowers the operational costs of the solution. Violin architecture uses patented technology to remove performance degradation normally associated with writing to hard disk. the Violin Memory storage array accommodates easy maintenance and growth. Like a modern storage system should be.

1 Performance Overview Fujitsu PRIMERGY RX300 S7 and Violin Memory 6232 The testing and certification process requires running a benchmarking system that includes a full set of tests that simulate real world test queries and metrics. With the Fujitsu PRIMERGY RX300 S7 and Violin Memory.2 GB/s Average throughput with SQL 2012 columnstore indexes enabled Maximum I/O observed with columnstore indexes enabled Maximum Physical I/O observed This table shows a maximum capacity of 60 TB for this reference architecture.8 GB/s. Metric Rating Rated Database Capacity TB 25 TB Comment Certified for up to 25 TB of queries Maximum User Database 60 TB Maximum amount of user data Total RAW space 35. the throughput or IOPS will not change between 25 TB and 60 TB of user data.2 TB Array Raw space without formatting Total User Usable Space 20. The Violin array’s performance ensures that the index can be dropped and rebuilt rapidly when compared to disk based solutions. which provides additional insight into the performance of the flash storage above and beyond what you Page 8 of 38 http://www. The reason is that the columnstore index does not select all data but subsets from each SQL page and therefore becomes a somewhat random I/O pattern. with columnstore indexes present.fujitsu. This was achieved with a 20 session test. Same aggregate throughput will be available regardless of the size of the database and number of concurrent users.8% of the maximum possible physical IO while under heavy load compared to the maximum baseline IO rates which are measured running SQLIO when the system is idle. the array’s average tested throughput was 7.8 GB/s Peak I/O 5. Columnstore index (CSI) is a new feature introduced with SQL Server 2012 that provides significant performance gains as tested on the Violin storage array. The storage array contains a management and administration portal known as vSHARE. The storage array is able to maintain 97. Averaging the 20 session CSI test. Furthermore. In general the more concurrent queries and users placed on a commodity spinning disk the lower the performance. full HA Fast Track Average I/O CSI 7.1 GB/s.1 GB/s Maximum I/O CSI 7. What this means to end users is they can expect the storage array to provide efficiency under heavy random I/O load that is roughly equivalent to the array’s maximum IO capability.5 GB/s.2 TB After formatting. Fast Track rated I/O was 3. This is a critical advantage as data is loaded into the system and used in day to day operations. This guarantees the system will perform to Fast Track DW expectations while using up to 25 TB of data to run queries. The maximum sustained I/O with CSI enabled was 7.com/fts/fasttrack . active spares. For comparison it is common for spinning media based solutions to drop off to 28-50% or less of the maximum I/O throughput.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 3 3. Spinning disks will not benefit from this feature as dramatically. CSI can only be applied to read-only tables. Here are selected metrics which should provide an understanding of the system performance.

Furthermore. most systems are prepared for the next Intel® Xeon® generation. This is made possible by the new processors.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide can capture with traditional Windows Performance Monitor. which now have 8 cores as well as an improved I/O architecture. the array provides a comfortable margin of additional headroom which can be used to run additional applications or simply provide ample headroom for running mission critical applications during production hours. In the field of high performance computing the growth in performance is up 120%. 3.com/fts/fasttrack . This is significant in the scenario where a DW workload only needs a few hundred thousand IOPS. The PCIe lanes are now directly connected to the processors and no longer to an intermediate I/O hub.000 read IOPS (4K) and sustain that level for the duration of the test. dynamic increase in processor performance. In this case the array was able to push close to 800.not just according to today's standards but also those of the future. The integrated Turbo Boost technology from Intel enables an on-demand. Page 9 of 38 http://www. the new generation of PRIMERGY systems is equipped to meet future requirements.both large and small – and provide a powerful expandable platform for the wide range of critical business applications and solutions used in a data center. An increase in overall performance of up to 70% is achieved with the help of the latest Intel® Xeon® processor E5 family. thus allowing fast and easy processor upgrades without the need to invest in new hardware.fujitsu. In most cases. Here Fujitsu has continued to develop the new generation in the three following areas in particular: Performance The new generation of PRIMERGY Dual Servers enables an important leap forward in performance compared with previous generations. thus reducing latencies and optimizing throughput.2 Hardware Overview Server Overview The new generation of Fujitsu PRIMERGY Dual Servers helps you overcome the challenges of IT and make IT even more powerful and more flexible . Administrators now know exactly how many IOPS they have spare for other workloads. Furthermore. Versatility The Fujitsu PRIMERGY dual-socket servers have many innovations .

low-priced upgrade option to the existing on-board LAN controller. the new generation of PRIMERGY Dual Socket Servers offers an expandable. remote and power management come from a single source. When it comes to I/O throughput. which understands data center structures and processes. high-performance environment for a large number of business applications and data center solutions. for deployment. All ServerView products. which also enables IT to be reorganized without having to invest in new servers.  Modular LAN provides a standardized. This is particularly important for those customers Page 10 of 38 http://www. It is also possible to consolidate LAN and Management LAN via one controller. which enable individual configuration according to requirements with maximum efficiency of up to 94% (80Plus Platinum) and simplified storage.g. which enable the systems to be extended easily and on demand. while on the other hand offering high scalability to enable new components to be added at any time. Fujitsu offers in this regard upgrade kit for hard disks and cages for all new systems as well as for backup devices. the new systems offer up to 100% more RAM and thus support up to 768GB. Here all the components can be freely configured and subsequently extended.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide This is made possible by innovative technologies:  Modular design and high expandability enable maximum adaptability for new areas of application at all times  Accelerating processor-intensive computing thanks to the support of GPGPU cards  Flexible I/O connectivity for a balanced operation of virtual and physical servers  State-of-the-art Intel® Xeon® E5 processors provide top performance  Intelligent and easy to operate server administration via the ServerView® Management Suite sustainably reduces the administration and maintenance work involved In short. thus simplify the cabling. Upgrade Options The modular design makes it possible on the one hand to only configure the components that are needed for today's IT requirements. The new PRIMERGY systems also rely on a completely revised modular design. Scalability The new generation of PRIMERGY Dual Socket Servers sets new standards in the fields of scalability and modularity. this supports technology changes at any time and allows for an individual configuration of 1Gbit or 10Gbit ports.  Modular RAID provides a comprehensive and standardized offer of RAID solutions. an increase of up to 100% can be achieved with the support of the latest PCI-Express standard (Generation 3). monitoring and administration. e. spread over 24 DIMM slots. which allow the user to select the right controller for the application scenario. Management Efficiency The Fujitsu PRIMERGY ServerView Suite provides all the necessary elements for professionally managing server systems during the Lifecycle.  Modular power supply units are standardized power supply units for the new generation of PRIMERGY Dual Socket Servers. Compared with the S6 generation.fujitsu.  The new housing concept now offers the basis of running up to 24 hard disks in tower/rack servers and up to 32 hard disks in blade servers. This is good for the budget and safeguards the investment. which ensures the requirements of tomorrow.com/fts/fasttrack . They also provide maximum versatility. And up to 33% more hard disks or SSDs can also be integrated.

Furthermore. Low-noise mode: This reduces noise to a minimum so that the server can be used in the office environment. improves the lifespan and service quality through optimized deployment. Energy Efficiency . thus providing significantly more performance for applications and databases and at the same time reducing power consumption and heat dissipation levels. State-of-the-art power monitoring and control functions ensure a high level of energy-efficient server operation.com/fts/fasttrack . Figure 2: Fujitsu PRIMERGY RX300 S7 Page 11 of 38 http://www. the seamless integration of PRIMERGY Servers in corporate management solutions ensures full investment protection. Additional pre-defined energy profiles are now available that allow the server power consumption to be adapted individually to the respective load cycles:   Performance mode: This profile provides the PRIMERGY server with maximum performance at all times Minimum power mode: Ensures minimum energy consumption so that enormous savings can be achieved when usage is low. On the one hand. 80+ Platinum). The Fujitsu ServerView® Suite is cost-effective. the new PRIMERGY systems offer 73% more efficiency. highly efficient power supply units are used (94%. It offers generic management functions for server installation. monitoring and administration including basic remote management as well as a range of integration packs enabling easy integration of PRIMERGY Servers in widely-used enterprise management systems or vendor-specific server management solutions free-of-charge. thus ensuring efficient operation of the PRIMERGY servers. On the other hand Power Management is further simplified.73% more performance per Watt The new PRIMERGY server generating sets new standards in energy efficiency.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide aiming to benefit from the latest virtualization and server technologies and thus improve the efficiency and flexibility of their IT. permanent status monitoring and extensive control options – even when there are malfunctions. The Fujitsu ServerView® Suite minimizes downtimes. Compared to the previous generation.fujitsu.

fujitsu. It provides a solid Page 12 of 38 http://www. Hot swap functionality includes rolling updates to the firmware on the system.9 GHz with Hyper-Threading 2 8 16 32 256 GB DDR3 (16 X 16 GB DIMMs) 2x SSD SATA 6G 400GB MLC (400 GB configured in RAID 1) 4x QLogic QLE2562 (dual port) (8Gbps) Violin 6232 Storage Array V-6232-HA64-FC Windows Server 2012 Enterprise SQL Server 2012 Enterprise SP1 (Build 11. The system represents exceptionally reliable and predictable infrastructure. Flash storage (VIMM) is also hot swappable and in vRAID configuration to never lose data. With total required space of 5Us the system saves space and power as compared to many other solutions with the same performance parameters.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide Storage Overview Violin Storage Memory Arrays deliver exceptionally low latency measured in micro seconds and consistent predictable read throughput of up to 3.3000) Conclusion The Fujitsu and Violin Memory Fast Track Reference Architectures provide the best combination of performance and manageability out of the box.0.5GB/s.3 Details Fujitsu PRIMERGY RX300 S7 Intel Xeon E5-2690 2. Each component of the array is doubled and hot swappable while the system is running and delivering full data rate speed. Fujitsu and Violin Memory deliver a comprehensive product line covering the needs of relatively small 20 TB all-memory data warehouse to a large 240 TB (and soon 480) all-memory architecture. Figure 3: Violin Memory 6212 Storage Array Component Server CPU Total Sockets Cores Per Socket Total Physical Cores Total Logical Cores Memory Internal Storage HBA Storage Array Operating System SQL Server 3.com/fts/fasttrack .

reducing complexity and increasing data loading speeds and efficiency by a significant factor.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide infrastructure for data warehousing needs.fujitsu.fujitsu.com  http://www.com/solutions/enterprise-applications/sql-server/ `  http://www.fujitsu.fujitsu.fujitsu. The evolution of enterprise storage has placed flash memory in a strategic market position to provide the most usable storage compared to dollars spent over spinning disks. This is achieved with an easy to use setup program and a minimal learning curve for system administrators.vmem.violin-memory.com/fts/fasttrack  http://www.com/fts/products/computing/servers/  To find out how to buy the solution or get a POC please contact sales at storage-pm@ts.com or sales@vmem.com/fts/products/computing/storage/all-flash-arrays/  http://www. The storage design provides a paradigm changing solution compared to other reference architectures that require up to several hundred hard disks and many racks of space to deliver comparable performance.com Page 13 of 38 http://www. The Violin 6232 storage array comes out of the box ready to use making set up easy and with the web based administration UI.com/fts/fasttrack .  For more information visit the following links:  http://www.

one thing to take into account is the price/usable GB. Page 14 of 38 http://www. Figure 4: PCIe slots to use with Violin Memory Array Violin Memory 6232 storage arrays can also be configured with InfiniBand adapters. 4. The introduction of flash memory arrays is starting to change the way people look at storage as a major bottleneck in their infrastructure. PCIe adapters (built in) and 10Ge adapters. A switch can be utilized when additional compute nodes are introduced. As compared to RAID 10 configuration requirements (and attendant storage enclosures) of commodity hard drives needed to satisfy IOPS requirements. When looking to purchase storage. 5 and 6. Violin is a very desirable solution for achieving the most IOPS with the lowest price/usable GB. This array is already pre-configured with internal RAID groups and hot spare VIMM cards. there is no need for a fibre switch. and with this solution. 4.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 4 Installation and Configuration Instructions The server is a 2U configuration and the storage array is a 3U configuration so minimizing total rack space was taken into account when architecting this hardware solution. finding room in a data center should be an easy task. Storage failover is already accounted for inside the 3U array. 4. The PCIe architecture is designed to accept most standard networking components given there are drivers to run them on the memory. Direct connect was the method chosen for connecting the storage array to the server although nothing prevents using a fibre switch between the server and the storage array if preferred. please validate that the memory modules and HBA cards are installed in the following manner: The QLogic HBA cards will be installed in PCI Slots 3.1 Installing HBA Cards and Memory Modules If the system does not come preconfigured. A lot of the Fast Track reference guide focuses on how to lay out your storage and specify RAID levels for different LUNs. One of the major benefits of Violin storage arrays is the ease of use and simplicity of administration provided.fujitsu. possibly intended for compute fail over or to introduce another application or DW to the same storage array. The storage array and server are connected by 8 OM3 Fibre Channel cables.com/fts/fasttrack . 5 and 6. With only 5U needed to house this hardware. The Fibre Channel cards should be installed in slots 3.

2C. 1G. 1C. 1B. 1D. 2A. 2D. 1E. the storage array comes with 4 dual-port HBA cards installed out of the box. The server should have all 4 HBA cards installed per the previous diagram.fujitsu. 2E. 1H and 2H. 2G.com/fts/fasttrack . 2B.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide The 8 GB memory modules will be installed in the DIMM slots 1A.2 Connecting Compute Node to Storage Array Below is a schematic of the compute node and storage array clearly showing the direct fibre connection. This configuration will deliver the highest possible throughput while eliminating I/O contention on the motherboard. 2F. Figure 5: CPU and DRAM Placement 4. 1F. Page 15 of 38 http://www.

3. 2.fujitsu.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide Figure 6: Direct Attachment of PRIMERGY to Violin Memory Array 4. Power the Fujitsu PRIMERGY Server on. Change the following : After scrolling down additional parameters are visible. 4. Press F2 to enter the BIOS configuration.1 Pre Operating System Installation Tasks There are a number of configuration steps that need to be undertaken within the BIOS prior to installation of the operating system. Page 16 of 38 http://www.com/fts/fasttrack .3 Operating System Installation This section will discuss the installation of the operating system. 1. There are some prerequisite tasks as well as some post installation configuration tasks that need to take place to configure the server for optimal performance in accordance with Fast Track guidelines. 3.

Please follow the instructions for installation by referring to the Fujitsu ServerView Installation Manager Guide.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 4. When prompted. which can be found at the following web address: http://manuals.3.3 Service Packs and Special Instructions Please ensure the latest Windows Updates have been applied to the Windows Server 2012 installation.com/fts/fasttrack . indicate that you want to install the operating system onto the mirrored internal SSDs.pdf. Page 17 of 38 http://www.fujitsu. Please follow normal installation steps for installing the operating system and apply all current patches via Windows Update.2 Operating System Installation Tasks The Operating System drives and Fujitsu system software will all be installed using the ServerView Installation Manager which can be located on the ServerView Suite media that comes with your Fujitsu system. The operating system to be installed is Windows Server 2012. Below is a screenshot showing the version of Windows used for this Fast Track.com/file/8396/sv-install-mgr-en.3.ts. 4.fujitsu.

3. you will need to change the power plan. disable all Windows firewalls in the OS. Please refer your company’s firewall policy or check with the system administrator for further information.4 Post Operating System Installation Configuration Once the Windows OS has been installed.fujitsu. open the OS Power Options (Control Panel) and change the power plan from “Balanced” to “High Performance”. In order to do this.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 4. Page 18 of 38 http://www.com/fts/fasttrack . Additionally. This is not required but firewall rules would need to be set in place to allow SQL Server through the firewall.

com/fts/fasttrack . Make sure they are present in the Storage Controllers section of Device Manager.fujitsu.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5 5. Page 19 of 38 http://www. These will be dual-port QLogic QLE HBAs for 8 total IO paths.1 Storage Array Installation HBA Drivers Prior to connecting the storage array please ensure there are 4 HBA cards installed on both the server and the storage array.

2 Configuring MPIO When configuring MPIO.fujitsu.10. please select “Discover Multi-Paths” and select “VIOLIN SAN ARRAY” and select “ADD”. Please make sure your driver is at least this version for the HBA cards. check your MPIO settings and you should see the storage array present as shown below. 5.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide The current driver as of the time of this writing is 9. Reboot when asked. Page 20 of 38 http://www. After the server has been rebooted.com/fts/fasttrack .1.28 dated 11/26/2012.

Note that unlike when using spinning disks.fujitsu.3. Connect serial port 2. Configure networking 5.1.com/fts/fasttrack . Page 21 of 38 http://www. Creating Initiator Groups 2.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5.3 Storage Array Configuration Connect the storage array to the server using OM3 Fibre Channel cables. SQLLog01 250 GB 4. SQLData02 9.1 Storage Array Configuration You will need to create LUNs using the Admin interface for the Violin Storage array.3. SQLData01 9. SQLStage01 1.706 GB 3.024 GB The process of creating LUNs is done from the admin web interface and is comprised of the following steps : 1. 1. Turn on 3. From this screen please select “Add igroup” from the top right section of the initiator groups.707 GB 2. Creating LUNs 3. You will need 8 of them to connect all the ports on the server to the storage array to account for 4 dual port cards. In our testing we created 4 LUNs with the following names and sizes. Exporting LUNs 5. the number of LUNs with flash memory is not relevant but we did add some logical separation and these sizes may be different in your environment: 1.1 Creating Initiator Groups From the home screen of the admin interface select “LUN Management”  “Manage Initiators”.

Page 22 of 38 http://www.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide From this screen choose a name for the new initiator group.fujitsu. you will associate all 8 World Wide Names (WWN’s) to the new group. Confirm the initiator assignment in the following dialog by clicking OK.com/fts/fasttrack . After the group is created. This is done by selecting the new group created under the initiator groups section and selecting all 8 WWN’s under the manage initiators section and then select “Save” followed by “Commit Changes” at the top centre of the UI.

Next click on “Manage Targets” under “LUN Management” and make sure each target is in a good state. click “Commit Changes” at the top left of the UI: It should look like the following once completed.com/fts/fasttrack . Page 23 of 38 http://www.fujitsu.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide Finally.

2 Creating LUNs For creating LUNs. Page 24 of 38 http://www.1.com/fts/fasttrack . Select “Create LUN” and from here you give it a name. As mentioned earlier one of the major benefits of Violin storage is you should achieve predictable performance irrespective of the number of LUNs present. At this point you should see nothing allocated for the storage array.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide At this point you have set up connectivity of the HBAs between the server and the storage array. exporting. Click “Yes” when you get the warning box that says “Not all client systems support 4096 block size.3. 5. and presenting 1 LUN and this same process can be followed for the number of LUNs you decide to present. Continue to create LUN(s) using 4096?”. and select 4096 bytes for the “Block Size”. This process will take you through creating. size.fujitsu. In the next step we will create LUNs using the admin web tool. select “Manage LUNs” under the “LUN Management” section of the admin interface.

select “Add Export” to bring up the “Add export for LUN” screen. Make sure “All Initiators” and “All Ports” is selected and hit OK.1. 5.3 Exporting LUNs From the right side based on the screen shot above.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide At this point you have a LUN present with a status of “not exported”.3.com/fts/fasttrack . Page 25 of 38 http://www.fujitsu.

3.fujitsu. If you rescan disks on the server using “Disk Management” you will see the new LUN present.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide After this process. screen shots are for illustration only. Page 26 of 38 http://www.1. You have successfully exported the LUN. select “Commit Changes” button in the top middle of the screen. Note: For next steps please follow sizing provided in 5.com/fts/fasttrack .

volumes need to be created and formatted in the NTFS file system in Windows. Under Disk Management format the new Volumes with a 4096 block size. choose GPT (GUID Partition Table) for the partition style. but unallocated. Next you want to right click the disk and select “New Simple Volume”. At this point the new disk is now online.4 Adding Windows Volumes Once the LUNs have been presented to the OS.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5. The LUNs should be presented as mount points. Page 27 of 38 http://www. Once you see the newly presented disk available in Disk Management the first step is to initialize the disk by right clicking and selecting “Initialize Disk”.com/fts/fasttrack .fujitsu.

In this example we are using mount points but drive letters are perfectly acceptable if that is easier to manage in your organization.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide This brings up a wizard to walk you through the process.fujitsu. Page 28 of 38 http://www.com/fts/fasttrack . Remember to follow these same steps to create the multiple LUNs if necessary. Make sure to allocate maximum possible size for the volume. Also make sure to use the NTFS file system with an “Allocation unit size” of 4096. At this point the drive is online and available for use.

fujitsu. Page 29 of 38 http://www.com/fts/fasttrack . For this go to “Properties” of the drive mount point or drive letter and remove the check box “Allow files on this drive to have content indexes in addition to file properties”.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide The last step is to remove content indexing for these drives.

tempdb.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5. in particular the Database Engine and the SQL Server Agent. 5. Perform Volume Maintenance Tasks Below is a screen shot of these settings: Page 30 of 38 http://www. These settings can be found by clicking “Start”  “Administrative Tools”  “Local Security Policy” Once this opens select “Local Policies”  “User Rights Assignment” 1. The user databases.fujitsu.1 Pre SQL Server Installation Tasks Prior to installing SQL Server. Lock Pages In Memory 2. please assign it to the following Local Security Policies in the User Rights Assignment section. and transaction logs will be pointed to a mount point on the storage array.5.com/fts/fasttrack . please create a domain Service Account to run the SQL Server Services.5 SQL Server Installation The SQL Server version used for installation will be SQL Server 2012 Enterprise Edition and will be installed onto the C:\. Once this service account is created.

Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5.5.com/fts/fasttrack . Please perform a normal install and choose to install all components except Reporting Services (SharePoint) components.3 Post SQL Server Installation Configuration There are a number of tasks to perform after the installation is complete to configure SQL Server.fujitsu. Below is the discovery report of all the installed features in the current installation: 5.5. You can choose all the defaults for now as we will change the data and TempDB locations after the installation process. Page 31 of 38 http://www. The build of SQL Server that was installed at the time of the writing of this paper is SQL Server 2012 with Service Pack 1.2 Installing SQL Server The version of SQL Server for this reference architecture is SQL Server 2012 Enterprise.

It is assumed your backups will be to an external file share using non Tier I storage. MAX = 235 GB (240640 MB) Page 32 of 38 http://www. Set the MIN/MAX Memory Settings for SQL Server a. MIN = 200 GB (204800 MB) b.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 1. 2.com/fts/fasttrack .fujitsu. Change the default database file locations to one of the mount points for the storage array. Ensure that the DW database you create has the same number of files on each LUN of the same size with auto growth enabled.

Size each file to ~100GB.fujitsu.com/fts/fasttrack .Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 3. 4. Set the following startup parameters for the SQL Server Service using the SQL Server Configuration Manager a. Move the TempDB log file to the LUN named SQLLog01. –T1117 Page 33 of 38 http://www. –E b. Move TempDB data file to storage array data LUN SQLData01 and create an additional data file on SQLData01 and 2 additional data files on SQLData02.

6.com/fts/fasttrack . Resource Governor Settings. Please see below for checklists related to all settings and configurations. Set the “Memory Grant %” to 12% from the default of 25% by opening “Management” and right clicking “Resource Governor” and selecting “Settings”. Restart the SQL Server services At this point SQL should be configured and ready for use.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 5.fujitsu. Page 34 of 38 http://www. Set MaxDOP to 16 7.

.NET Framework 3.com/fts/fasttrack .fujitsu.204800 MB MAX .2 Storage Area Array LUNs MPIO OS Volumes OS Volumes 6.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 6 6.12.04 Multipath I/O.11.5.3 Description Windows Server 2012 Provided by Fujitsu ServerView Installation Manager release 11.1 Features High Performance (OS and BIOS) Disable all windows firewalls Description 4096 block size VIOLIN STORAGE ARRAY present 4096 allocation unit size Content indexing turned off SQL Server Area SQL Version Startup Parameters Memory Settings MAXDOP Local Security Policy Resource Governor Page 35 of 38 Description SQL Server 2012 Enterprise Edition SP1 -E -T1117 MIN .1 Checklists Operating System Area OS Version Drivers Features Enabled Power Settings Firewall 6.240640 MB 16 Perform Volume Maintenance Tasks Lock Pages In Memory Memory Grant % = 12% http://www.

com/fts/fasttrack . Instead of one server with 4 HBA cards. one for each server. Depending what the HA/DR objective is you can combine these instance level and database level availability features utilizing multiple storage array’s and servers for the highest level of protection. 2 cards from the Violin 6232 should be connected to 2 cards on each node of the WSFC.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 7 High Availability Scenario 7. you would perform a normal SQL Server cluster install. and the LUN would be exported to both initiator groups as well. SQL Server 2012 has some great new features for multi subnet clustering that eliminate the need for a stretched VLAN and a flexible failover policy so the administrator has control to set the levels that would initiate a failover. Below are some links to point you in the right direction to getting started with these new SQL Server 2012 features:     Overview of SQL Server 2012 High Availability Solutions SQL Server Multi-Subnet Clustering Overview of AlwaysOn Availability Groups AlwaysOn Failover Cluster Instances Page 36 of 38 http://www. Below are some links to setting up a SQL Server Failover Cluster:     7. you would need two arrays combined with SQL Server 2012 Mirroring or SQL Server 2012 AlwaysOn Availability Groups for database level protection and/or multi subnet cluster for instance level protection. To use this scenario.1 Local High Availability It is possible to use the Violin 6232 storage array in a high availability scenario combined with Windows Server Failover Clustering (WSFC) and SQL Server failover cluster instances (FCI). the configuration will have 2 servers with 2 HBA cards. Since SQL Server clustering requires shared storage. Once storage has been presented. For this scenario. The Violin 6232 fully supports SQL Server clustering and this section gives an overview of how that configuration needs to be architected. When creating the initiator groups and exporting LUNs there will be 2 initiator groups.2 SQL Server Clustering Prerequisites SQL Server Failover Cluster Installation Create a New SQL Server Failover Cluster Add or Remove Nodes in a SQL Server Failover Cluster Remote High Availability Another HA/DR configuration that is possible with the Violin storage array is cross data centre disaster recovery. Two dual port cards at 8Gb/s/port can deliver 4GB/s total which provides more than sufficient bandwidth to avoid becoming the bottleneck in the storage path. the LUNs created on the Violin 6232 must be presented to both compute nodes in the cluster.fujitsu.

Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 8 Bill of Materials To order a complete solution.com Individual Components are: Qty Part Number Violin Memory 6232 Storage Array Hardware 1 D:V6232HA64-8XFC Service 1 D:V6232-8XFC-VSB1 Fujitsu PRIMERGY RX300 S7 Hardware 1 S26361-K1373-V401 2 S26361-F5225-E400 4 S26361-F3631-E202 2 S26361-F3686-E290 1 S26361-F3669-E1 1 S26361-F1790-E242 2 S26113-F574-E10 16 S26361-F3697-E516 1 S26361-F1373-E420 1 S26361-F2735-E145 8 D:FCKAB-OM3-C05L-L 1 S26361-F2735-E82 2 S26361-F3417-L505 1 S26361-F1452-E100 2 T26139-Y1968-E100 2 S26361-F3694-E10 1 S26361-F4530-E10 Software 1 S26361-F2567-E420 Service 1 FSP:GP3S63Z00DEPX3 Page 37 of 38 Description V-6232 32.5" HDD RackMountKit-F1_DI_CMA_QRL FC-Cable OM3 LWL 5m LC LC Rack Cable Arm 2U LAN-CAT 5 Enhanced.fujitsu.90 GHz 20 MB RAID Ctrl SAS 6G 1GB (D3116) Remote Management iRMC S3 advanced pack Modular PSU 800W platinum hp Memory 16GB (1x16GB) 2Rx4 L DDR3-1600 R ECC Hard Disk Bay Config 2: 4x 2.com/fts/fasttrack . .7TB 64VIMMs 8x4/8Gbit/s FC V-6232-HA24-8xFC Bronze Maint. l=5m Region Kit APAC/EMEA/India Cable Power Cord Rack. 4h Response Time. 4m.5" EP MAIN FC Ctrl 8Gb/s 2 Channel QLE2562 MMF LCLP Intel® Xeon® E5-2690 8C/16T 2.fujitsu. please contact Fujitsu or Violin Memory Sales at: storage-pm@ts.1 Year PY RX300S7 2. 24hx7 http://www. gray Independent Mode Installation Mounting in Symmetrical Racks Windows Server 2012 Standard 2CPU TP 3years OS Service.5" Expandable Chassis SSD SATA 6G 400GB MLC HOT P 2.com or sales@vmem.

fujitsu.8 80807 Munich. Technical data subject to modification and delivery subject to availability. product and service names may be trademarks or registered trademarks of their respective owners.ts. the Fujitsu logo are trademarks or registered trademarks of Fujitsu Limited in Japan and other countries. Other company.Technical Paper 60 TB Microsoft Fast Track Data Warehouse Reference Architecture – Installation and Configuration Guide 9 9.com/file/8396/sv-in stall-mgr-en. Any liability that the data and illustrations are complete. the use of which by third parties for their own purposes may infringe the rights of such owner.pdf Abbreviations Abbreviation CSI DIMM DR DW ETL FCI HA HBA HDD IOPS LUN OLAP OS RAM SLA SSD VIMM WSFC Contact FUJITSU Technology Solutions GmbH Address: Mies-van-der-Rohe Str.2 Comment Link: http://manuals. Fujitsu.1 Appendix Literature and Links Mark Document [ServView User Guide] ServerView Suite ServerView Installation Manager User Guide English Edition December 2012 9. Transform and Load Failover Cluster Interface High Availability Host Bus Adapter Hard Disk Drive Input Output Per Second Logic Unit Number Online Analytical Processing Operating System Random-Access Memory Service Level Agreement Solid-State Drive Violin Intelligent Memory Module Windows Server Failover Clustering Copyright 2013 Fujitsu Technology Solutions. actual or correct is excluded. Germany E-mail: storage-pm@ts.fujitsu.fujitsu.com Website: www.com/fts/fasttrack .fujitsu.com/fts/storage 2013-05-02 EM EN Page 38 of 38 Meaning ColumnStore Index (Feature of Microsoft SQL Server 2012) Dual Inline Memory Module Disaster Recovery Data Warehouse Extract. http://www. The designations may be trademarks and/or copyrights of the respective manufacturer.