SQL Server 2008 Fast Track Data Warehouse 2.

0
Speaker: Phil Hummel of WinWire Technologies Presentation developed by: Bruce Campbell
Western Region Data Warehouse Specialist, Microsoft

Silicon Valley SQL Server User Group February 16, 2009

Mark Ginnebaugh, User Group Leader, mark@designmind.com

Agenda
• DW vs. OLTP • Balanced Architecture Approach for DW Fast Track Defined • Fast Track Reference Architectures • Next Steps

Microsoft DW & BI Stack
DELIVERY

PerformancePoint Services END USER TOOLS

BI & DW PLATFORM (RDBMS, ETL, OLAP, Reporting)

DW
Database

versus
Database

OLTP

• Designed for analytical operations: Strategic focus • Optimized for bulk load and large, complex, unpredictable queries • Fewer concurrent users relative to OLTP

• Designed for operational requirements: Tactical focus • Optimized for transactions: “single row” entry and retrieval • Thousands of concurrent users

Storage
• Primary focus on Read operations • Optimized for disk scan over seek operations • Storage optimization focused on disk scan rate (MB/s)

Storage
• Emphasizes transactional performance • Optimized for disk seek over scan operations • Storage optimization focused on I/O operations/s (IOPs)

Sequential I/O
Sequential I/O • Scans on large data stores are usually read with sequential read patterns and not random read patterns • Scalable, predictable performance • Requires 1/3 or fewer drives to match server I/O consumption capability.

Random I/O
OLTP usually random-read centric. Discrete lookups benefit from index optimization and random read capability. Not as predictable & scalable for data warehousing Requires large number of drives to match server I/O consumption capability.

• •

All databases contain both scans and seeks among with other types of reads and writes, DW workload indicate that the vast majority of reads are sequential – not all

Some SQL Data Warehouses today
Big SAN Big 64-core Server Connected together

What’s wrong with this picture?

Answer: system out of balance
• This server can consume 16 GB/Sec of IO, but the SAN can only deliver 2 GB/Sec
– Even when the SAN is dedicated to the SQL Data Warehouse, which it often isn’t – Lots of disks for Random IOPS BUT – Limited controllers Limited IO bandwidth

• System is typically IO bound • Queries are slow

Result: significant investment, not delivering performance

The Alternative: A Balanced System
• • • • Design a server + storage configuration that can deliver all the IO bandwidth that CPUs can consume when executing a SQL Relational DW workload Avoid sharing storage devices among servers Avoid overinvesting in disk drives
– Focus on scan performance, not IOPS

Layout and manage data to maximize range scan performance and minimize fragmentation

Potential Performance Bottlenecks
DISK DISK

CPU CORES WINDOWS SQL SERVER CACHE

SERVER

A B

FC HBA

A B

STORAGE CONTROLLER

CACHE

FC HBA

FC SWITCH

A B

A A B B LUN LUN
DISK DISK

CPU Feed Rate

SQL Server Read Ahead Rate

HBA Port Rate

Switch Port Rate

SP Port Rate

LUN Read Rate

Disk Feed Rate

SQL Server Fast Track Data Warehouse
Solution to help customers and partners accelerate their data warehouse deployments

• A method for designing a cost-effective, balanced system for Data Warehouse workloads • Reference hardware configurations developed in conjunction with hardware partners using this method • Best practices for data layout, loading and management

Relational Database Only – Not SSAS, IS, RS

Fast Track Data Warehouse Components

Software: • SQL Server 2008 Enterprise • Windows Server 2008 Configuration guidelines: • Physical table structures • Indexes • Compression • SQL Server settings • Windows Server settings • Loading Hardware: • Tight specifications for servers, storage and networking • ‘Per core’ building block

Fast Track Scope
Supporting Systems BI Data Storage Systems Presentation Layer Systems

Integration Services ETL

Analysis Services Cubes
Presentation Data Presentation Data

Data Path

Web Analytic Tools Reporting Services

SharePoint Services Dedicated SAN, Storage Array Microsoft Office SharePoint Data Warehouse Data Staging, Bulk Loading PerformancePoint Excel Services

Reference Architecture Scope (dashed)

Two SQL DW Infrastructure Options: SQL Classic DW or Fast Track SQL DW
SQL Classic DW Architecture Leverages Shared SAN Fast Track SQL DW Architecture Architecture modeled after DW Appliances Teradata, DATAllegro..etc “ Appliance Like” Uses Dedicated SAN arrays and Network
Dedicated Network Bandwidth Dedicated SAN

Enterprise Shared SAN Storage

Shared Network Bandwidth

SQL 2008 Data Warehouse SMP Server

SAN Arrays 1:4 cpu cores 8 Data Disk / Array – 4 Raid 1 Pairs Simultaneous SQL Server Reads 2 Log and 1 Hot Spare EMC AX4 – HP MSA2312 IBM 3400

OLTP Applications

SQL Fast Track DW supports “Scan Centric” DW workloads that are index light

Optimizing storage layout for scan intensive workloads
• LUN configuration is based on RAID1 pairs
– Optimal for scan type access patterns
S P A S P B
RAID GP01 RAID GP02 RAID GP05

01

02

03

04

09

10

LUN1 LUN2
RAID GP03

LUN3 LUN4
RAID GP04

05

06

07

08

LUN5 LUN6

LUN7 LUN8

Observed throughput for a single RAID pair >= 130 MB/s

HOT SPARE

Striping across storage is accomplished via SQL Server data files

LUN0 (Logs)

Storage Layout Implications for SQL Server
LUN 1 LUN 2 LUN 3 LUN16

Permanent FG
Permanant_DB

Permanent_1.ndf

Permanent_2.ndf

Permanent_3.ndf

Permanent_16.ndf

Stage Database

Stage FG

Stage_1.ndf Local Drive 1

Stage_2.ndf

Stage_3.ndf

Stage_16.ndf

TempDB

TempDB.mdf (25GB) TempDB_02.ndf (25GB)

TempDB_03ndf (25GB)

TempDB_16.ndf (25GB)

Log LUN 1

Permanent DB Log Stage DB Log

Creating Sequential Data Layout
• Goal: Align logical and physical ordering of data within a Filegroup • Two primary ways Fast Track optimizes allocation for Sequential Scan
– Minimize Fragmentation – Manage Load processing

Maximum Consumption Rate
Theoretical throughput for IO stack
• Using a 2x quad-core server as a building block / starting point Ensure that the percore data consumption rate can be delivered by all elements of the IO stack Sticker on the new car: “Miles Per Gallon”
Maximum theoretical throughput for IO stack components sized for an 8 CPU core Fast Track system (assumes 200 MB/s per core)

300 MB/s

500 MB/s

300 MB/s 300 MB/s

Fiber Switch

MCR 1.6 GB/s Windows Server OS
CPU Socket (4 Core) CPU Socket (4 Core)

500 MB/s

300 MB/s

Storage Enclosure

300 MB/s

HBA

500 MB/s
Min 2 GB/s Min 2 GB/s

300 MB/s 300 MB/s

HBA

500 MB/s

300 MB/s

Server

Storage Enclosure

Scaling the IO stack
Storage Processor

CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core)

CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core)

Fiber Switch

Storage Processor

Storage Enclosure
Storage Processor Storage Processor

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

Storage Enclosure
Storage Processor Storage Processor

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

Storage Enclosure
Storage Processor Storage Processor

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

HBA HBA HBA HBA HBA HBA HBA HBA
Storage Processor Storage Processor Storage Processor Storage Processor Storage Processor Storage Processor Storage Processor Storage Processor

Storage Enclosure

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

RAID-1 RAID-1

RAID-1 RAID-1

Storage Enclosure

RAID-1

Storage Enclosure

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

Storage Enclosure

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

Server

Storage Enclosure

RAID-1 RAID-1 RAID-1 RAID-1 RAID-1

Fast Track Data Warehouse Reference Configurations
Server
HP Proliant DL 385 G6 HP Proliant DL 380 G6 HP Proliant DL 585 G6 HP Proliant DL 580 G5 HP Proliant DL 785 G6 Dell PowerEdge R710 Dell Power Edge R900 IBM X3650 M2 IBM X3850 M2 IBM X3950 M2 Bull Novascale R460 E2 Bull Novascale R480 E1

CPU
(2) AMD Opteron Istanbul six core 2.6 GHz (2) Intel Xeon® 5500 Series Quad core (4) AMD Opteron Instanbul six core 2.6 GHz (4) Intel Xeon® 7400 Series six core (8) AMD Opteron Istanbul six core 2.8 GHz (2) Intel Xeon Nehalem quad core 2.66 GHz (4) Intel Xeon Dunnington six core 2.67GHz (2) Intel Xeon Nehalem quad core 2.67 GHx (4) Intel Xeon Dunnington six core 2.67 GHz (8) Intel Xeon Nehalem four core 2.13 GHz (2) Intel Xeon Nehalem quad core 2.66 GHz (4) Intel Xeon Dunnington six core 2.67GHz

CPU Cores
12 8 24 24 48 8 24 8 24 32 8 24

SAN
(3) HP MSA2312fc (2) HP MSA2312 (6) HP MSA2312fc (6) HP MSA2312 (12) HP MSA2312 (2) EMC AX4 (6) EMC AX4 (2) IBM DS3400 (6) IBM DS3400 (8) IBM DS3400 (2) EMC AX4 (6) EMC AX4

Data Drive Count
(24) 300GB 15k SAS (16) 300GB 15k SAS (48) 300GB 15k SAS (48) 300GB 15k SAS (96) 300GB 15k SAS (16) 300GB 15k FC (48) 300GB 15k FC (16) 200GB 15K FC (24) 300GB 15k FC (32) 300GB 15k SAS (16) 300GB 15k FC (48) 300GB 15k FC

Initial Capacity*
6TB 4TB 12TB 12TB 24TB 4TB 12TB 4TB 12TB 16TB 4TB 12TB

Max Capacity**
12TB 8TB 24TB 24TB 48TB 8TB 24TB 8TB 24TB 32TB 8TB 24TB

* Core-balanced compressed capacity based on 300GB 15k SAS not including hot spares and log drives. Assumes 25% (of raw disk space) allocated for Temp DB. ** Represents storage array fully populated with 300GB15k SAS and use of 2.5:1 compression ratio. This includes the addition of one storage expansion tray per enclosure. 30% of this storage should be reserved for DBA operations

SQL Server Fast Track Data Warehouse 2.0 for HP – now on G6 Platform
Five AMD and Intel based Reference configurations available for HP: AMD Based Reference Architectures 2 Processor Configuration
– Server: HP ProLiant DL385 G6 with 2 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 4 – 12 TB

4 Processor Configuration
– Server: HP ProLiant DL 585 G6 with 4 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 12 – 24 TB

8 processor Configuration
– Server: HP ProLiant DL 785 G6 with 8 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 24 – 48TB

SQL Server Fast Track Data Warehouse 2.0 for HP – now on G6 Platform
Intel Based Reference Architectures 2 Processor Configuration
– Server: HP ProLiant DL380 G6 with 2 4-core Intel Xeon® 5500 Series CPUs – Storage server: MSA Storage – Scalability: 4 – 8 TB

4 Processor Configuration
– Server: HP ProLiant DL 580 G5 with 4 6-core Intel Xeon® 7400 Series CPUs – Storage server: MSA Storage – Scalability: 12 – 24 TB

New Fast Track Data Warehouse 2.0 for IBM
Three Reference configurations available for IBM: 2 Processor Configuration
– – – – – – – – – Server: IBM System x3650 M2 with 2 Quad-core Intel Xeon CPUs Storage server: IBM System Storage DS3400 Scalability: 4 – 8 TB Server: IBM System x3850 M2 with 4 6-core Intel Xeon CPUs Storage server: IBM System Storage DS3400 Scalability: 12 – 24 TB Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs Storage server: IBM System Storage DS3400 Scalability: 16 – 32TB

4 Processor Configuration

8 processor Configuration

SQL Server Fast Track Data Warehouse 2.0 for DELL
Two Reference configurations available for DELL: 2 Processor Configuration
– Server: Dell Power Edge R710 with 2 Quad-core Intel Xeon processors – 8 CPU Cores – 32GB Memory – Storage server: EMC CLARiiON AX4 – Scalability: 4 – 8 TB

4 Processor Configuration
– Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors – 24 CPU Cores – 96 GB Memory – Storage server: EMC CLARiiON AX4 – Scalability: 12 – 24 TB

SQL Server Fast Track Data Warehouse for BULL
Two Reference configurations available for BULL: 2 Processor Configuration
– Server: Bull Novascale R460 E2 with 2 Quad-core Intel Xeon processors – Storage server: EMC CLARiiON AX4 – Scalability: 4 – 8 TB

4 Processor Configuration
– Server: Bull Novascale R480 E1 with 4 6-core Intel Xeon processors – Storage server: EMC CLARiiON AX4 – Scalability: 12 – 24 TB

Also included in the Rack:
– – – – – SQL Server Analysis Services SQL Server Reporting Services SQL Server Integration Services HA Server Administration Server (with Management Studio, Backup Server

Fast Track Data Warehouse Benefits
• Lower TCO
– Minimizes risk of overspending on un-balanced hardware configurations – Commodity Hardware

• Choice
– HW platform – Implementation vendor

• Reduced Risk
– Validated by Microsoft – Encapsulates best practices – Known performance & scalability

Summary
Fast Track Data offers customers
Faster time to solution High scale: up to 48TB Low TCO with better price performance; industry standard hardware Better performance out of the box and predictable performance Reduced risk through balanced hardware & Best practices Integration with Madison Hub & Spoke Architecture

SQL Server Fast Track Data Warehouse has 2 components

Twelve reference architectures from HP, Dell, Bull, EMC and IBM System Integrators with industry solution templates – Avanade, HP, Hitachi, Cognizant and EMC

Next Steps
• Proof Steps
– Quick Start DW Roadmap Service – Architectural Design Session – Madison Technology Preview (MTP) – Review Madison, SQL Server Classic or Fast Track DW HW/SW configurations and pricing

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Sign up to vote on this title
UsefulNot useful