August Network Announcement v3

IBM Flex System Fabric SI4093 System Interconnect Module
Heather Richardson - Product Marketing Hector Sanchez Product Manager
Aug 2013
2012 IBM Corporation
Agenda
IBM Flex System Fabric SI4093 Module EN4093R Update IBM Flex System EN6131 40Gb Switch
Infrastructure Components
Beyond Blades
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Data Center Challenges

Complexity
Too many layers to manage Virtual or logical management
Multiple Management points

Physical and Virtual Management Manage Server, Storage and I/O as one entity
Increase infrastructure utilization

IT budgets are flat or decreasing Virtualization / Cloud and new applications demand more throughput and performance Manage risk and plan for next decade
Reduce data center foot print space, power and cost

Focus on all aspects to increase hardware utilization Management is one of the biggest area of focus followed by hardware spending and power
Intelligence is moving to the edge of the network

Intelligence is moving from the core to the edge
Opposite of what some of our competitors want!
Why this is happening:

Performance and latency demands Server and networking optimization is critical Convergence of LAN and SAN Virtualization of more east-west traffic High Frequency Trading, Database intensive apps, Messaging (MQ) and Highly Virtualized environments
Software Servers
Core Layer
WAN
Customer Impact:
Aggregation Layer
< 25%
Layer2
Access Layer Layer2
Storage
> 75%
SAN
Network
The Flex System Portfolio Continues to Grow

NEW NEW v 1.3
x222
x220
x240
x440
Storage Expansion
PCiE Expansion
Flex System Manager
NEW POWER 7+
p24L
p260
p270
p460
Flex System V7000
Storwize V7000
NEW
SI4093
5
EN4093R
EN6131
CN4093
EN2092
FC5022
FC3171

Base 10 x 10GbE SFP+
Networking
Unmanaged 10Gb module with 10/40Gb Uplinks

Designed for easy connectivity and management Pre-configured from factory enhances out of box experience for Clients Rapid network provisioning without configuration errors Investment Protection scalable pay-as-you-grow design reduces cost and complexity Intra chassis switching for optimal performance (Ex. vMotion) Virtual Fabric capability reduces cost and improves IO utilization Seamless interoperability with other vendors switches like Cisco, Juniper and Brocade
10Gb to Server 10Gb Uplinks 40Gb Uplinks
Networking Infrastructure
Simplifies chassis connectivity without compromising performance Suited for Clients looking for unmanaged device in the chassis Pay as you grow scalability Seamless interoperability
#2 = 4x10GbE #1 = 2x40GbE
Total Ports
1GbE Mgmt
Base System
Upgrade #1 Upgrade #2
14
28 42
10
10 14
0
2 2
IBM Flex System Fabric EN4093R with OpenFlow Support Software Defined Networking
Networking Increased IT efficiency with programmable networks
Reduce complexity by building the underlying network once Dynamic and rapid application deployment Automate network provisioning and interoperability OpenFlow support enables Software Defined Networks (SDN) Programmable network enables easy customization and innovation Create SDN-ready network architecture based on Open Standards Intelligent and dynamic multipath routing based on business policy Customize network architecture based on business policies and needs Centrally configure and enforce multi-tenant networks Available to new and existing Clients as feature update at no extra cost
IBM Flex System Fabric SI4093 System Interconnect Module Highlights

Simplify connectivity and interoperability challenges with a preconfigured interconnect module
Benefits
Reduced cost and complexity
Key Features
Easy connectivity with preconfigured settings Eliminates the need to configure each device or individual ports thus reducing the number of management points Lower TCO - Up to 42% less expensive than the competition* Lower cabling and power costs Supports up to forty-two downlink 10GbE connections, fourteen uplink 10GbE SFP+ connections plus two 40GbE QSFP+ connections 2-3 X more bandwidth than the competition** Default profile end host mode plug and play just add cables No Spanning Tree, loop free, port aggregation System Interconnect End Host configuration viewed as a large compute node to upstream Network Minimal system admin dependency on network admin for most compute node provisioning actions Can never default into a switch mode protects against network misconfigurations
Pay as you grow scalability
Simple Management
System administrator friendly System Interconnect Module
Base - 95Y3313, ESWA Upgrade 1 95Y3318, ESW8 Upgrade 2 95Y3320, ESW9
Claim * Based on SI4093 base plus upgrade 1 and 2 list price x 2 = $53.194, HP Virtual Connect Flex-10/10D x 6 = $75,450 => 41.8% http://h30094.www3.hp.com/product.aspx?sku=10440121&pagemode=ca **SI4093 data sheet 64x10Gb ports (640Gb) vs HP VC 10 data sheet 26x10Gb ports (260Gb) => 2.1x - http://h30094.www3.hp.com/product/sku/10440121 SI4093 42 Downlinks + 22 Uplinks (64 10GbE) vs HP FEX 16 downlinks + 8 Uplinks (24 10GbE) => 2.6x - http://h18004.www1.hp.com/products/quickspecs/14138_div/14138_div.html

Ethernet Connectivity
Scalable 10Gb Ethernet with 10/40Gb Uplinks

Overview Easy to deploy simple connectivity upgrade in order
Total Ports Base System 10Gb Downlinks 14 28 10Gb Uplinks 10 10 40Gb Uplinks 0 2
Base 10 x 10GbE SFP+
Simple Management Optimized for performance Pay as you grow scalability Lowest TCO Easy interoperability
(95Y3313, ESWA)
I/O Infrastructure
w/ Upgrade #1
(95Y3318, ESW8)
#2 = 4x10GbE
w/ Upgrade #2 42 14 2 Important Features: (95Y3320, ESW9) Exceptional price/performance FCoE convergence as a transit switch Easy connectivity to reduce deployment time Low Touch default mode or VLAN mode Easy interoperability with other vendors switches No need for network administrators to manage Warranty & software upgrade licenses match the chassis Alternative to: Pass-thru, HP Cisco FEX or HP VC Flex
#1 = 2x40GbE 1GbE Mgmt
1. 2.
9
Default transparent end host mode VLAN aware mode for multi-tenancy environments or environments requiring more control of VLAN Layer 2 forwarding. Also enables FCoE FIPS This document is for IBM and local IBM Business Partner use only. in the chassis. Snooping Bridge support enhancing the FCoE operation
It is not intended for client distribution or use with clients.
Connectivity options and distances for the SI4093

Networking Cable and Transceiver Questions
Will your 1Gb connections exceed 100m (less than 550m)? Will your 10Gb connections exceed 5m (less than 300m)? Will your 40Gb connections exceed 3m? Will your 1 or 10Gb connections exceed 300m (less than 10km)? 1000Base-T Transceiver = 1Gb Short Range Transceivers + Optical Cable = 10 Gb Short Range Transceivers + Optical Cable = 40Gb Short Range QSFP+ Trans+ Optical Cable = Long Range Transceiver + Optical Cable 1000Base-LX Transceiver 10KM
1000Base-SX Transceiver 100 Meters
1GbE
RJ-45
550 Meters
Optical Cable
Optical Cable
10GbE
DAC/TwinAx
5 Meters
10GBase-SR Transceiver
300 Meters
10GBase-LR Transceiver
Optical Cable
Optical Cable
40GbE
QSFP+
3 Meters
40GBase-SR Transceiver
QSFP+ MTP Optical Cable
100 Meters
DAC cables are ideal within a rack or adjacent racks (lower cost & power) Transceivers are best for longer distances
10
Portfolio Positioning
11
Flex System Ethernet Module Positioning

EN6131
40Gb Ethernet
Low latency/ high bandwidth apps Telecommunication Enterprise applications high Performance Scalability Virtualization Cloud computing Simple management Simple connectivity and interop with upstream network Infrastructure Applications (file/ print/ collaboration)
EN2092
1Gb Scalable Switch
EN4093R SI4093
Simple Low touch Connectivity Module 10Gb Scalable Switch 40Gb uplinks Ethernet, iSCSI FCoE support
CN4093
10Gb Scalable Converged Switch 40Gb uplinks Native FC ports FCF support for Storage node Ethernet, FCoE
Eighteen uplink High bandwidth Low latency

Ethernet support
CN4093 Convergence within the chassis Connect to existing LAN and SAN network Scalable fabric Easy transition to 40GbE L2/3 function Enhanced Virtual Fabric for reduce I/O cost and complexity
EN4093R High performance 10GbE connection L2/3 function Scalable fabric Easy transition to 40Gb Enhanced Virtual Fabric for reduce I/O cost and complexity OpenFlow SDN Support SI4093 Simple setup and management Scalable fabric Easy transition to 40Gb Virtual Fabric for reduce I/O cost and complexity EN6131 High performance 40Gb Low latency EN2092 1Gb connectivity Easy transition to 10Gb EN4091 10Gb connectivity Unmanaged device
EN4091
10Gb Pass thru
NEW Aug 6th
10Gb performance OpenFlow Enabled Scalable 40Gb uplinks Ethernet, iSCSI Updated Non blocking FCoE support Aug 6th Connectivity to Upstream Network
Unmanaged Device
10Gb Uplinks For easy Ethernet, FCoE Transition Ethernet, iSCSI support
NEW Aug 6th
1Gb Ethernet
10Gb iSCSI, Ethernet FCoE(transit)
10Gb Converged In chassis
40Gb Ethernet
12
Feature Compare of IBM System Networking offerings for IBM Flex System
Features
10Gb Converged Switch CN4093

FCoE Convergence Yes Yes Yes Easy connect Yes Yes Omni ports Yes (No, disabled in Easy Connect) 1280Gbps Yes2
10Gb Ethernet Switch EN4093R

Multi mode flexibility Yes (No L3 in Easy Connect) Yes Yes Easy Connect Yes No Yes (No, disabled in Easy Connect) 1280Gbps Yes
10Gb System Interconnect Module SI4093

Preconfigured end host mode L2 aware Yes Yes default profile Yes No No 1280Gbps No
10Gb Pass Thru EN4091

1:1 direct connect Pass thru only N/A N/A Yes No N/A 280 Gbps N/A
Summary Layer 2/3 Support Multi Tenancy (SPAR) Transparent / End host mode FCoE/CEE support FC ports Spanning Tree Max through put* Stacking
Interoperability
Network Virtualization Max Ports base + upgrades
Open
VMready, Virtual Fabric (vNIC2 or Unified Fabric Port*) 42x downlinks10 GbE 2x uplinks 10 GbE SFP+ 12x uplinks SFP+ Omni 2x uplinks 40 GbE QSFP+ No Yes
Open
VMready, Virtual Fabric (vNIC2 or Unified Fabric Port) 42x downlinks 10 GbE 14x uplinks 10 GbE SFP+ 2x uplinks 40 GbE SFP+ Yes (v7.7)
Open
Virtual Fabric (vNIC2)
Open
N/A
42x downlinks 10 GbE 14x uplinks 10 GbE SFP+ 2x uplinks 40 GbE SFP+ No
14 x downlinks 10GbE 14 x uplinks 10GbE SFP+
Open Flow Support 13 VE Support SDN
N/A Yes
This document is for IBM and IBM BusinessYes Partner use only. Yes
It is not intended for client distribution use clients. This configuration leverages six of the eight ports on the CN4058 adapter available for IBM Power Systems compute or nodes 2 with - Hybrid
Decision Tree: Storage Connectivity Requirements

NAS / iSCSI
Connect to storage?
FCoE
Separate Fibre Channel
Chassis FC directly out of chassis to FC SAN switch or FCoE to storage device
Break out of FC
Top-of-Rack
FCoE to TOR Broken out to IBM, Brocade or Cisco

Compute Node: Ethernet: LOM or CN4054 (with SW upgrade for CNA function) CN4058 (Power)** Chassis: Ethernet Transit: EN4093R*/SI4093 TOR Converged Switch: G8264CS or Cisco/Brocade
Compute Node: Ethernet: LOM CN4054 (with SW upgrade for iSCSI HW offload EN4132/EN4054 for Power nodes Chassis: Ethernet: EN4093R* or SI4093
Compute Node: Ethernet: LOM or CN4054 EN4132/EN4054 for Power nodes Fibre Channel: FC3172 or FC3052 (8Gb) FC5022 (16Gb) Chassis: Ethernet: EN4093R*/SI4093 Fibre Channel: FCXXXX
Compute Node: Ethernet: LOM or CN4054 (with SW upgrade for CNA function) CN4058 (Power)** Chassis: Converged Switch: CN4093*
14
Technical Overview and Use Cases
15
Simplify chassis connectivity using IBM System Interconnect Module

Overview Default profile enables transparent integration
No Spanning tree Built in logic for loop prevention Minimal setup or management required
Key operating modes 1. Transparent mode:
Presents an end host, (compute node) view to the upstream network VLAN communication between compute node NIC and edge switch runs transparently through the module (VLAN agnostic operation) Enables server to server communication for optimum performance (i.e. vMotion) Provides traffic consolidation in the chassis to minimize TOR port utilization Supports FCoE pass-thru to a transit switch or FCoE gateway device
2. VLAN aware mode (Local Domain)
Provides greater security and separation for multi-tenant environments Enables forwarding decisions on client VLANs (ACL support) Can re-use IP addresses in multi-tenancy configurations Supports FCoE transit switch operation, FSB operation
16
Hardware Description
SFP+ Connector (1Gb/10Gb)
QSFP+ Connector (10Gb/40Gb)
Serial Port RJ45 Mgmt Port
17
Switch Partition (SPAR) How it works:

1) 2) 3) 4) 5) 6) 7) 8) Creates separate virtual switching contexts by segmenting the data plane of the switch. The management and control plane partition of a physical switch is NOT addressed within the SPAR context and are still common across the partitions. Implemented as a dedicated VLAN which is generally defined by a set of internal server ports and a single external uplink port or LAG (static or LACP). Multiple uplink ports or LAGs are not allowed within a SPAR, eliminating the possibility of misconfiguring the device and creating a loop with the upstream network. Port memberships are mutually exclusive between SPARs. By default, a SPAR is a private domain that operates as a Layer-2 broadcast network. Hosts on the same VLAN attached to a SPAR are allowed to communicate with each other and with the upstream switch. Hosts attached to the same VLAN but on different SPARs dont communicate with each other within the embedded interconnect. Traffic needs to be bridged by the upstream switch in this situation since there is no communication between SPARs within each broadcast domain.
9)
18
SPAR can be configured to operate in two basic domain processing modes: Transparent and VLAN aware (Local).
SPAR support for SI4093 is a superset containing select features from SPAR Phase 1 & 3 plus customizations specific only to SI4093
Key differences in SPAR on SI4093 Addition of Host Mode
Enables multiple SPARs to connect to the same upstream network Disables MAC learning on upstream ports and applies ACLs to avoid MAC Flapping
FCoE support within a SPAR context
Transparent mode support pass-thru Local mode Transit / FSB

Increase of local VLAN support to 256
19
SI4093 Packet forwarding in Transparent Mode

Packet Data Packet Data
C-VLAN 10
C-VLAN 10
EXT-1 Packet Data C-VLAN 10 S-VLAN 4081
EXT-2
EXT-3 Packet Data C-VLAN 10 S-VLAN 4082
EXT-4
SPAR-1 (Pass-Through) Default VLAN-4081 Port: {INTA1, INTA2, INTA3, EXT1, EXT2}
SPAR-2 (Pass-Through) Default VLAN-4082 Port: {INTB1, INTB2, INTB3, EXT3, EXT4}
Packet Data C-VLAN 10 S-VLAN 4081
INTA1
INTA2
INTA3
INTB1
INTB2
INTB3
Packet Data C-VLAN 10
20
SI4093Packet forwarding in VLAN aware Mode (Local Domain)

EXT-1 Packet Data C-VLAN 10
EXT-2
EXT-3 Packet Data C-VLAN 10
EXT-4
SPAR-1 (Local-Domain) Default VLAN-4081 Port: {INTA1, INTA2, INTA3, EXT1, EXT2} VLAN-20 VLAN-10 {EXT1, EXT2, INTA2, INTA3} {EXT1, EXT2, INTA1, INTA2}
SPAR-2 (Local-Domain) Default VLAN-4082 Port: {INTB1, INTB2, INTB3, EXT3, EXT4} VLAN-10 {EXT3, EXT4, INTB1, INTB2} VLAN-20 {EXT3, EXT4, INTB2, INTB3}
INTA1
INTA2
INTA3
INTB1
INTB2
INTB3
21
SI4093 Multi-Tenancy Host Mode
DC Network
SI4093 SPAR-1 SPAR-2
vSwitch-1
vSwitch-2
vSwitch-1
vSwitch-2
vSwitch-1
vSwitch-2
ITE-1
ITE-2
ITE-3
22
SI4093 Multi-Tenancy with VLAN aware
DC Network
V10, V11, V12
V20, V21, V22 SI4093 SPAR-1 SPAR-2
Each port group within vSwitch is using a specific uplink with VLAN separation among them. Each port group mapped to a SPAR so they can use separate uplink port set from the SI4093. In this kind of topology the SI4093 can run in VLAN aware mode and learn MAC address on uplink port. No dependency on centralize controller for server/VM MAC learning
PG-1 vSwitch-1
PG-2
PG-1 ITE-1
PG-2 vSwitch-1 ITE-2
PG-1 vSwitch-1
PG-2 ITE-3
23
Nexus 5K
Nexus 5K
40 Gb
40 Gb
40 Gb
40 Gb
EXT EXTEXTEXT 1 2 3 4
EXTEXTEXTEXT 5 6 7 8
SI4093 1
Partition 1 MGMT, Migration Storage, Heart Beat
Flex Chassis
EXT EXTEXTEXT 1 2 3 4
EXTEXTEXTEXT 5 6 7 8
SI4093 2
Partition 1 MGMT, Migration Storage, Heart Beat
Partition 2 VM Data
Partition 2 VM Data
14 X 10Gb
INT INT 1A 2A
INT 14A
INT INT 1B 2B
INT 14B
14 X 10Gb
14 X 10Gb
INT INT 1A 2A
INT 14A
INT INT 1B 2B
INT 14B
14 X 10Gb
P0
P1 Hypervisor
P2
P3
P0
P1 Hypervisor
P2
P3
Hypervis or MGMT Heart Beat
Live Migration Storage iSCSI NFS FCoE
Supervisor-Switch Hypervisor-Firewall
Hypervis or MGMT Heart Beat
Live Migration Storage iSCSI NFS FCoE
Supervisor-Switch Hypervisor-Firewall
VM
VM
VM
VM
14 X wide ITE
24
Recommended configuration using end to end IBM offering

IBM Converged ToR
Key value: Integrated FCoE on compute node High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis IBM 10Gb System Interconnect Module aggregates all FCoE traffic and passes to LAN and SAN.
Black line is FCOE Red line is Fibre Channel
V7000
Cisco or Brocade SAN
G8264CS G8264CS
LOM or CN4054 (pNIC, vNIC mode)
SI4093 Flex System Fabric Interconnect
Adapter
Integrated Device
FCoE Top Of Rack Switch G8264CS (IBM Converged ToR) in NPIV mode
SAN Switch
Storage Target
OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes
SI4093 10Gb Module
Cisco & Brocade SAN
FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV
Win2008, ESX 4/5, RHEL 5/6, SLES 10/11
25
Connecting to Cisco network: Customer already owns Nexus ToR
Key value: Integrated FCoE on compute node High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis Connects to existing or new Cisco network.
V7000
Cisco SAN
Nexus 55XX Nexus 55XX
Adapter
Integrated Device
FCoE Top Of Rack Switch Nexus 5548 / 5596
SAN Switch
Storage Target
OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes 26
SI4093 10Gb Module
Cisco SAN
FC: V7000, SVC
Win2008 R2SP1, ESX 5.0 U1, RHEL 6.3
Connecting to Brocade network: Customer already owns Brocade VDX or SAN

Key value: Integrated FCoE on compute node High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis Connects to existing or new Brocade network
V7000
Brocade SAN
Brocade VDX Brocade VDX
Adapter
Integrated Device
FCoE Top Of Rack Switch Brocade VDX
SAN Switch
Storage Target
OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes 27
SI4093 10Gb Module
Brocade SAN
FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV
Win2008, ESX 4/5, RHEL 5/6, SLES 10/11
IBM Flex System EN6131 40Gb Switch IBM Flex System EN6132 40Gb Adapter Matthew Sheard - Mellanox
Aug 2013
IBM Flex System 40Gb Ethernet Switch (EN61310) and adapter (EN6132)
Networking End to end 40Gb enables highest performance
High performance end to end 40Gb connectivity with ultra low latency
Ultra low latency for HPC, High Frequency Trading and Enterprise Clients Up to four 40Gb ports per single wide node Highest performance non blocking architecture drives up to 2.8TB bidirectional throughput
Non blocking throughput that is highly suited for HighPerformance Computing, High Frequency Trading and Enterprise Data Centers.
Enables clients to maximize compute utilization by removing IO bottleneck Reduces cabling by 4 is to 1 for applications requiring 40Gb throughput
29
Mellanox Value Proposition

Financial Clustered Computational Aided Database Engineering Cloud & Web 2.0 Bioscience
University, Academic
Labs, Research
Oil and Gas
Weather
Digital Media
Mellanox heritage in technical computing means that Mellanox switches have some of the lowest port to port latency and jitter, providing a stable high speed interconnect for todays demanding business and virtualization environments.
HPC
Web 2.0
DB/Enterprise
Cloud
Financial Services
Storage
30
Performance and Green Too !
Performance
100+%
Increase
Complete High-Performance Scalable Interconnect Solutions for Server and Storage

TCO Energy Costs Infrastructure
50+% Reduction
65+% Reduction
% 60+ Saving
Efficiency: Reduce components and maximize productivity a data center.
Energy: Reduce power and cooling expenses
Opex: Reduce management overhead and costly real estate.
31
RDMA Technology Becoming Mainstream
Remote Direct Memory Access

Direct access of one computers memory by another computer without involving either ones operating system Allows applications to transfer data directly to the buffer of a remote application Zero-copy, I/O offload, CPU bypass technology for data transfer Near wire speed data transfers with sub 1us latency
Standard RDMA Protocols

RDMA-over-Converged-Ethernet (RoCE)
Up to 40Gb/s
InfiniBand:
Up to 56Gb/s
IBM:Yuqung Gao, Internet Centric Computing WXS RDMA ICPE 2013
32
I/O Offload Boosts Application and Business Performance

Without RDMA With RDMA and Offload
I/O Offload: More Efficient Application Processing
User Space
~53% CPU Available for Application
User Space System Space
~88% CPU Available for Application
System Space
~47% CPU Overhead/Wait
~12% CPU Overhead/Wait
33
Underlying Technologies Enabling Cloud 2.0

Private Clouds Small Scale Clouds Scalability Mobility Serviceability
Virtually Unlimited Tenants Fixed Resource Allocation
Resources Everywhere
Any Service, Any Time
Cloud 1.0
Virtualization Overlay Networks
Cloud 2.0
Interconnect Offload Engines Open Platform
VXLAN, NVGRE
Reduce Application Cost
Open Ethernet, SDN
34
Drive Higher Dollars Per Square Foot for Cloud

Triple the number of Virtual Machines per server: from 20 to 60 VMs Higher storage density: 2X throughput and I/O Operations per Second (IOPS) Atlantic.net achieves 2X cloud I/O performance vs. Amazon with less cost using InfiniBand
Without Mellanox
VM VM
With Mellanox
VM
VM VM
VM VM VM VM
VM
VM
VM
35
RoCE vs. iWARP

RoCE delivers lower latency, lower CPU load and higher bandwidth Latency as low as 1us compared to 2.1us on iWARP Up to 6X in Throughput (64B) compared to iWARP iWARP requires TCP, using packet drops to handle heavy traffic TCP is handled by the CPU or a special and complicated TCP Offload (TOE) NIC RoCE requires NO CPU involvement
36
EN6131 Switch Features Management plane

10/100/1000 ETH RJ45 MNG ports USB Console port for Management Dual SW image Static IP DHCP SSH Telnet RADIUS \ TACACS+ \ LDAP NTP FTP\TFTP\SCP SYSLOG Auto Temperature control Events history Port counters SNMP v1,2,3 Web UI In-band Management
Data plane

Port mirroring sFlow Rapid Spanning Tree (802.1w) Flow control (802.3x) DCB PFC (802.1Qbb) ETS (802.1Qaz) LLDP LAG/LACP (802.3ad) 48K Unicast MAC addresses Cables/transceivers identification IGMP v1,2 VLAN (802.1Q) 4K ACL 12K rules Jumbo frames 9K
37
Shattering Ethernet Switching Performance Records

Leading throughput: 2.5X better 2.88Tb/s throughput on a single chip, running Full Wire Speed at any packet size Leading L2 unicast/multicast latency for L2/L3 switches: 2X better 198-223ns for any packet size Industry record power efficiency: 6X better than competition Sub 0.6Watt per 10GbE throughput with 100% load at Full Wire Speed
1,000
Latency (ns)
800 600 400 200 0 64 128 256 512 206 209 220 216
Zero Jitter!
219 223 222 222 223
L2 Min L2 Average L2 Max
1,024
1,280
1,518
2,176
9,216
38
Packet Size (Bytes)

Performance Driven Applications From Server to Fabric

The Price of Building a Machine to Process 1TB of Data in 1 Minute
Storage Connectivity Big data applications Distributed DB VM Environment (fast VM Migration) HPC Disaster Recovery Add more capacity to strained network
Migration Time Over the Network (sec)
It is not about the number of servers you run It is about how you run them
39
Lowest Latency 40GbE in Trading

E2E 40GbE Lowest Latency Cut through Solution Superior message rate
Exchange
Exchange Entry
TCP/UDP (UC)
UC Ack MC streams
10.10.0.1
10.10.0.0/30 Direct interface (pseudo vlan 2)
SX1036 TOR
10.10.0.2
10.10.10/2 Single VLAN
40GbE NASDAQ connectivity at the lowest latency

40
End-to-End 40GbE Live in US-based Exchange Co-Location

Mellanox provided a 40 Gigabit Ethernet solution that delivered high density, sustainable and high throughput, low latency and low power consumption that was beyond our expectations and in ways no other vendor could. We applaud the Mellanox team for their great support and helping us break new latency performance records.
Ido Rosen, quantitative researcher, Strike Technologies
Breaking New Latency Records

41
IBM Flex System Chassis with Mellanox 40Gb Interconnect

Mellanox ConnectX-3 Dual-Port 40GE NIC and Switch release in Q3/2013 Single wide chassis: 14x ITEs / Blade Servers support 2 adapters per server Double-wide chassis: 7x ITEs / Blade Servers support 4 adapters per server 14 ports Internal I/O; 18 ports External I/O Total System Compute up to 2.24Tbps System Front I/O up to 2.88Tbps
IBM PureFlex System
42
Mellanox Interconnect in IBM PureFlex

Dual Star Architecture
14 Compute Blades each using Single EN6132 Dual Port 40GbE NIC 2 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE Compute I/O 1.12 Tbps @40 Gbps Uplink I/O up to 1.44Tbps @40 Gbps
Dual-Dual Star Architecture

14 Compute Blades each using Dual EN6132 Dual Port 40GbE NIC 4 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE Compute I/O 2.24 Tbps @40 Gbps Uplink I/O up to 2.88 Tbps @40 Gbps
SWITCH
ITE 10GbE 22 x 10GbE 10GbE EN4093R 40GbE 22 x 10GbE EN4093R 40GbE 40GbE 40GbE 40GbE ITE 40GbE
SWITCH
18 x 40GbE EN6131
18 x 40GbE EN6131
18 x 40GbE EN6131
18 x 40GbE EN6131
18 x 40GbE EN6131
18 x 40GbE EN6131 2012 IBM Corporation
43
Summary and Next Steps
Mellanox based technology is available for IBM Blade, Rack, & Flex Communities
Heritage of technical computing being broadly adopted: Cloud, Database & Analytics Big Data Storage Core RDMA Technology Becoming Mainstream Web 2.0, Cloud, Analytics, and Storage Out of Box support for Linux, Windows, Cloud, & Hadoop Distros Storage: Windows Server 2012 with SMB 3.0 IBM has Broad Mellanox based Product Offering Today How can we help to make your business grow?
44
Trademarks
Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: AS/400, DBE, e-business logo, ESCO, eServer, FICON, IBM, IBM Logo, iSeries, MVS, OS/390, pSeries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM The following are trademarks or registered trademarks of other companies Lotus, Notes, and Domino are trademarks or registered trademarks of Lotus Development Corporation Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries LINUX is a registered trademark of Linux Torvalds UNIX is a registered trademark of The Open Group in the United States and other countries. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC. Intel is a registered trademark of Intel Corporation * All other products may be trademarks or registered trademarks of their respective companies.
NOTES:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.
The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.
45

August Network Announcement v3

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

August Network Announcement v3

Uploaded by

Copyright:

Available Formats

IBM Flex System Fabric SI4093 System Interconnect Module

Heather Richardson - Product Marketing Hector Sanchez Product Manager

2012 IBM Corporation

Data Center Challenges

Multiple Management points

Increase infrastructure utilization

Reduce data center foot print space, power and cost

2012 IBM Corporation

Intelligence is moving to the edge of the network

Why this is happening:

Access Layer Layer2

2012 IBM Corporation

The Flex System Portfolio Continues to Grow

Flex System Manager

Flex System V7000

IBM Flex System Fabric SI4093 System Interconnect Module

Unmanaged 10Gb module with 10/40Gb Uplinks

2012 IBM Corporation

IBM Flex System Fabric SI4093 System Interconnect Module Highlights

Pay as you grow scalability

System administrator friendly System Interconnect Module

Base - 95Y3313, ESWA Upgrade 1 95Y3318, ESW8 Upgrade 2 95Y3320, ESW9

2012 IBM Corporation

IBM Flex System Fabric SI4093 System Interconnect Module

Scalable 10Gb Ethernet with 10/40Gb Uplinks

Base 10 x 10GbE SFP+

#1 = 2x40GbE 1GbE Mgmt

2012 IBM Corporation

Connectivity options and distances for the SI4093

1000Base-SX Transceiver 100 Meters

2012 IBM Corporation

Flex System Ethernet Module Positioning

Eighteen uplink High bandwidth Low latency

NEW Aug 6th

NEW Aug 6th

10Gb iSCSI, Ethernet FCoE(transit)

10Gb Converged In chassis

2012 IBM Corporation

10Gb Converged Switch CN4093

10Gb Ethernet Switch EN4093R

10Gb System Interconnect Module SI4093

10Gb Pass Thru EN4091

14 x downlinks 10GbE 14 x uplinks 10GbE SFP+

Open Flow Support 13 VE Support SDN

Decision Tree: Storage Connectivity Requirements

Separate Fibre Channel

Chassis FC directly out of chassis to FC SAN switch or FCoE to storage device

FCoE to TOR Broken out to IBM, Brocade or Cisco

Technical Overview and Use Cases

2012 IBM Corporation

Simplify chassis connectivity using IBM System Interconnect Module

2012 IBM Corporation

SFP+ Connector (1Gb/10Gb)

QSFP+ Connector (10Gb/40Gb)

Serial Port RJ45 Mgmt Port

2012 IBM Corporation

Switch Partition (SPAR) How it works:

Transparent mode support pass-thru Local mode Transit / FSB

2012 IBM Corporation

SI4093 Packet forwarding in Transparent Mode

EXT-1 Packet Data C-VLAN 10 S-VLAN 4081

EXT-3 Packet Data C-VLAN 10 S-VLAN 4082

Packet Data C-VLAN 10 S-VLAN 4081