HP BladeSystem c-Class

connectivity options
Module 6
HP BladeSystem Administration
HE646S I.00

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Objectives

After completing this module, you should be able to:

• Describe the HP BladeSystem c-Class interconnect module architecture
• List the BladeSystem c-Class interconnect modules
− Ethernet

− Fibre Channel
− InfiniBand

− SAS

• Describe the mezzanine cards and slots available in the BladeSystem c-Class server blades
• Describe HP BladeSystem c7000 and c3000 enclosure port mapping

2 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Fabric convergence and c-Class architecture

Fabric convergence of:

• Data networking
• Storage
• Clustering traffic
c-Class interconnect architecture
• Ethernet
− TOE
− iSCSI for storage
− RDMA/TCP for clustering
• Fibre Channel
• InfiniBand
• SAS
• FCoE using FlexFabric or VC Flex10/10D
3 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Interconnect module architecture

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Single-wide interconnect modules
Inta ke Air
(left or right)
Exha ust Air
Connector JP
E45
Alignment Server Server
E1 BL16 BL09
P8 D45
P7
Connector JS
P5

C
(signal ground)
P3 P2
P1 B45
Server Server
A1 A45
BL08 BL01

NOTE: The horizontally adjacent interconnect bays within the c3000 and c7000 enclosures are wired to transfer signals between paired
(redundant) interconnects. Therefore, HP recommends installing identical interconnects in these interconnect bay pairs.

5 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Double-wide interconnect modules

Server BL16 Server BL09

Connector JP2 E45

Connector JP1
P8 Lane-2 & -3 Lane-2 & -3 Lane-0 & -1 Lane-0 & -1 D45
E1
P7

P5 Connector JS2 Connector JS1

C
P3 (signal ground)
P1 P2 Lane-2 & -3 Lane-2 & -3 A1 Lane-0 & -1 Lane-0 & -1
B45
Alignment
A45

Server BL08 Server BL01

6 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Interconnect bay crosslinks

c3000 enclosure c7000 enclosure

 
 
 
 
 
 
 

 

7 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Learning check

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Learning check

1. Because the interconnect bays in the c7000 enclosure are wired to transfer signals between
paired interconnects, HP does not recommend that you mix interconnects within an
interconnect bay pair.
a. True
b. False

9 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Interconnect modules

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Ethernet modules

Ethernet interconnects Picture Form factor

Mellanox SX1018HP Ethernet Switch Double-bay

HP 6125G Ethernet Blade Switch Single-bay

HP 6125G/XG Ethernet Blade Switch Single-bay

Cisco Catalyst 1GbE 3120G Blade Switch Single-bay

Cisco Catalyst 1/10GbE 3120X Blade Switch Single-bay

HP GbE2c Layer 2/3 Ethernet Blade Switch Single-bay

HP 6120XG Blade Switch Single-bay

11 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Ethernet and InfiniBand modules
Ethernet interconnects Picture Form factor
HP 6120G/XG Ethernet Blade Switch Single-bay

HP 1:10Gb Ethernet Blade Switch Single-bay

Cisco Catalyst Blade Switch 3020 for HP c-Class BladeSystem Single-bay

HP Virtual Connect Flex-10 10Gb Ethernet Module Single-bay

HP Virtual Connect Flex-10/10D Module Single-bay

InfiniBand interconnects Picture Form factor

HP 4X FDR InfiniBand Switch for BladeSystem Double-bay
HP 4X FDR InfiniBand Managed Switch

12 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Fibre Channel, FlexFabric, and SAS modules
Fibre Channel and FlexFabric interconnects Picture Form factor
Cisco MDS 8/12c Fabric Switch for HP BladeSystem c-Class Single-bay

Cisco MDS 8/24c Fabric Switch for HP BladeSystem c-Class Single-bay

Brocade 8/12c SAN Switch Single-bay

Brocade 8/24c SAN Switch Single-bay

HP Virtual Connect 8Gb 20-port Fibre Channel Module Single-bay

HP Virtual Connect 8Gb 24-port Fibre Channel Module Single-bay

HP Virtual Connect FlexFabric 10Gb/24-port Module Single-bay

SAS interconnects Picture Form factor

HP Storage 3Gb SAS BL Switch Single-bay
HP Storage 6Gb SAS BL Switch

13 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Pass-thru modules

Pass-thru modules Picture Form factor

HP 1Gb Ethernet Pass-Thru Module Single-bay
(NOTE: No auto-negotiation or 100Mbps supported)
HP 10GbE Pass-Thru Module Single-bay

HP 4Gb Fibre Channel Pass-Thru Module Single-bay

14 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Learning check

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Learning check

1. Which HP BladeSystem c-Class interconnect module occupies two interconnect bays?

a. HP Virtual Connect FlexFabric 10Gb/24-Port Module
b. HP 6Gb SAS BL Switch
c. HP InfiniBand for HP BladeSystem c-Class
d. HP Virtual Connect Flex-10 10Gb Ethernet Module
e. HP 6120G/XG Blade Switch

16 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mezzanine slots and cards

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Mezzanine slots and cards
Pre-Gen8 server blades

Mezzanine slots Mezzanine cards

Same physical size (shape and pins); different Type I mezzanine cards
electronic keying and power • Either four-lane (x4) or eight-lane (x8)
Type I slots • Fit into Type I or Type II slots
• Lower-positioned slots Type II mezzanine cards
• Can be x4 or x8 Type II slots Type I slot
• Eight lanes only (x8)
• Provide up to 15 watts • Fit into Type II mezzanine slots only
• Accept Type I cards only
Type II slots
Type
Type II
II card
• Higher-positioned slots Offset
card
Type
Type II
II slot
slot Type
Type II
II slot
slot
Type
Type II slot
• Are x8 only slot
System board
• Provide up to 25 watts Side view
• Accept Type I or Type II cards
18 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Mezzanine slots and cards
Gen8 server blades

x8 PCIe 3.0 Type A slots PCIe 3.0

• Supports Type A dual-port mezzanine cards • Bit rate of 8 GT/s
• Ports routed to bays 3 and 4 • Backward compatible
x16 PCIe 3.0 Type B slots • Doubles the PCIe 2.0 bandwidth
• Supports Type A and Type B dual-port and quad-port
mezzanine cards
• Dual-port card routing
− Ports routed to bays 5 and 6
• Quad-port card routing
− Ports routed to bays 5, 6, 7, and 8

NOTE: The ProLiant BL465c Gen8 is an exception—it has one x8 PCIe 2.0 Type A slot and one x16 PCIe 2.0 Type B
slot.
20 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
c7000 port mapping

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
Half-height server blades

22 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
Onboard Administrator view

23 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
Full-height server blades
Embedded NIC
LOM Port 3
Port 4 1/2 Bay 1 Bay 2
Ports 3, 4 1 Module Module 2
1 3/4
2
Mezz 1 3
4

1 5/6 Bay 3 Bay 4

Mezz 2 2
3 3 Module Module 4
4
7/8
Embedded NIC
LOM Port 1
1/2 Bay 5
Ports 1, 2 Port 2 Bay 6
5 Module Module 6
3/4

1 5/6 Bay 7 Bay 8

Mezz 3 2
3
7 Module Module 8
4
7/8

24 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
Onboard Administrator view

26 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
BL620c or BL680c G7
System board Interconnect bays
Embedded NIC
Port 3
Ports 3, 4 Port 4 1/2 Bay 1 Bay 2
1 Module Module 2
1 3/4
2
Mezz 1 3
4

1 5/6 Bay 3 Bay 4

Mezz 2 2
3 3 Module Module 4
4
7/8
Embedded NIC
Port 1
Ports 1, 2 Port 2 1/2 Bay 5 Bay 6
5 Module Module 6
3/4

1 5/6 Bay 7 Bay 8

Mezz 3 2
3
7 Module Module 8
4
7/8

27 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c7000 port mapping
BL680c G7
System board side B (left) Interconnect bays
1
Mezz 6 1/2 Bay 1 Bay 2
2
1 Module Module 2
1 3/4
2
Mezz 5 3
4

1 5/6 Bay 3 Bay 4

Mezz 7 2
3 3 Module Module 4
4
7/8
Embedded NIC
Port 5
Ports 5, 6 Port 6 1/2 Bay 5 Bay 6
5 Module Module 6
3/4

1 5/6 Bay 7 Bay 8

Mezz 4 2
3
7 Module Module 8
4
7/8
28 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
c3000 port mapping

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c3000 port mapping notes

• If you install a Virtual Connect

module in interconnect bay 1,
you must either put a matching Virtual
Connect module in bay 2
or leave it empty.
• To achieve full redundancy between
interconnect bays 1 and 2, you must
add a card on mezzanine one.

IMPORTANT: The c3000 enclosure does not support FlexFabric

interconnects.
30 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c3000 port mapping
Half-height server blades

31 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c3000 port mapping
Full-height server blade
Embedded NIC
Port 3
Ports 3, 4 Port 4 1/1 Bay 1 Bay 2
1 Module Module 2
1 2/2
2
Mezz 1 3
4

1 3/4
2
Mezz 2 3
4
3/4
Embedded NIC
Port 1
Ports 1, 2 Port 2 1/1 3 Bay 3 Bay 4 4
3 Module Module 4
2/2

1 3/4
2
Mezz 3 3
4
3/4

32 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c3000 port mapping
BL620c or BL680c G7
System board Interconnect bays
Embedded NIC
Port 3
Ports 3, 4 Port 4 1/1 Bay 1 Bay 2
1 Module Module 2
1 2/2
2
Mezz 1 3
4

1 3/4
2
Mezz 2 3
4
3/4
Embedded NIC
Port 1
Ports 1, 2 Port 2 1/1 3 Bay 3 Bay 4 4
2/2 3 Module Module 4

1 3/4
2
Mezz 3 3
4
3/4
33 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 c3000 port mapping
BL680c G7
System board side B (lower) Interconnect bay
Port 3
Mezz 6* Port 4 1/1 Bay 1 Bay 2
1 Module Module 2
1 2/2
2
Mezz 5 3
4

1 3/4
2
Mezz 7 3
4
3/4
Embedded NIC
Port 1
Ports 5, 6 Port 2 1/1 3 Bay 3 Bay 4 4
2/2 3 Module Module 4

1 3/4
2
Mezz 4 3
4
3/4
34 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Learning check

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 Summary

• The HP BladeSystem c-Class architecture supports − Ethernetand Fibre Channel, including Virtual
the convergence of different protocols in a single Connect for each
enclosure − FlexFabric
• Each single-wide and double-wide interconnect − InfiniBand and SAS
module supports high-speed connections to the
• Mezzanine cards connect server blades to Ethernet,
server blades
Fibre Channel, FlexFabric, SAS, and InfiniBand
− A single-wide module has one 180-pin midplane interconnects
connector; a double-wide module uses two
• Mezzanine cards typically work in conjunction with
− The c7000 enclosure has four pairs of interconnect interconnects
bays; the c3000 enclosure has four interconnect − When installing mezzanine cards in a blade and
bays
interconnects in an enclosure, port mapping must
− Each device bay is connected through the signal be considered to avoid “interconnect mismatch”
midplane to each of the interconnect bays errors

• Interconnect modules available as options include:

36 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.