Composable Infrastructure with HPE Synergy - Part 1

IT organizations need to support a variety of applications running on a

bewildering range of infrastructures. On the one hand, traditional
applications like financial and ERP systems are often running on bare
metal servers. On the other hand, virtual machine environments, with
their own infrastructure requirements, host an increasing share of
applications and services like virtual desktops. On top of that, recent
years have seen the hybrid and private cloud applications that require yet
another type of on-premise infrastructure.

Beyond the sheer complexity of managing multiple physical

infrastructures, it is inefficient, as well. Having hardware dedicated to
specific types of infrastructure requires overbuilding in order to manage
capacity. For example, the servers running your ERP system may have
idle capacity, but it cannot be used to run private cloud workloads. As a
result, excess capacity needs to be built into the private cloud
infrastructure to absorb workload spikes and maintain room for
development activities and new applications.

Composable infrastructure has emerged as a datacenter architecture to

solve these complexity and inefficiency problems facing IT
organizations. In this post, we explain composable infrastructure and
how HPE Synergy has implemented it to provide a compelling next-
generation datacenter computing platform.

What Is Composable Infrastructure?

Ideally, IT organizations would like to manage a single physical
infrastructure that could be dynamically configured to support all types
of workloads. So, for example, the excess capacity on an ERP server
could be reconfigured to the virtual machines, containers, and web
servers needed for a mobile app project that is under development.

This vision, of an adaptable and flexible physical environment that can

be configured to support a variety of logical infrastructure, is known as
composable infrastructure. A composable infrastructure provides pools
of resources - compute, storage, and networking fabric - that can be
automatically configured to support various application needs. An
application’s requirements for physical infrastructure are specified using
policies and service profiles. The composable infrastructure provides
intelligent management software, with deep knowledge of the hardware,
to interpret the specifications and compose a logical infrastructure
meeting the application’s requirements.

In order to engineer a platform to realize this composable infrastructure

vision there are several design principles to observe:

 Software Defined Infrastructure

 Fluid Resource Pools
 Hardware and Software Architected As One
 Physical, Virtual, and Containerized Workloads

Software Defined Infrastructure

A composable infrastructure needs to be built on top of a software

defined infrastructure (SDI). In an SDI, physical resources can be
defined and configured with machine readable code (e.g., configuration
files, templates). Moreover, orchestration software can read these
configurations and make calls to an application programming interface
(API) to configure the underlying hardware as defined. For example, the
specification might indicate that a server needs a 10GbE connection to a
particular database. The orchestration software would then make the
necessary API calls to the appropriate networking switch to allocate the
10GbE ports. To a large extent, IT administrators do not need to get
involved with the configuration process. Intelligent software determines
how to compose resources from the pools of compute, storage, and
network fabric.

Fluid Resource Pools

As mentioned, a composable infrastructure needs to provide fluid pools

of compute, storage, and networking resources. All resources in the
pools need to be immediately available for configuration and
deployment to support a workload. For example, if a web application
experiences a spike in traffic, additional web servers and bandwidth
must be allocated from the pools, configured, and deployed to handle the
additional workload. Likewise, unused resources must be reclaimed and
put back in the pools. So, when the traffic spike has passed, and the
additional web servers and bandwidth are no longer needed, the
underlying physical resources are returned to the pools. This may
involve deleting VMs to free up disk space and releasing networking
ports. The exact implementation details are handled by intelligent
management and orchestration software.

Hardware and Software Architected As One

At this point, you may be wondering how this magic can possibly
happen. The SDI needs a unified API to programmatically manage all
the compute, storage, and networking resource in the pools. In addition,
the management and orchestration software needs a detailed
understanding of the physical properties of all the hardware. And, of
course, the hardware needs to support programmable configuration and
be flexible enough to handle a wide range of scenarios. For example,
gear in the networking pool must support the dynamic configuration of
ports, bandwidth, and security policies.

To make this happen, the software and hardware need to be architected

together. To some extent, the software defines the requirements for the
hardware. The unified API must be supported in the hardware. But,
beyond that, the hardware must meet the definition of composability
specified in the software. For example, if the software layer provides
templates for a virtual desktop infrastructure (VDI), then there need to
be server, storage, and networking resources in the pools that can
support VDI. At a minimum, that means servers where hypervisors can
be installed, data storage that can be attached with low latency and high
bandwidth to the virtual machines, and a network that can be configured
to deliver the bandwidth required to support the expected number of
simultaneous end users.

Physical, Virtual, and Containerized Workloads

As mentioned in the introduction, IT organizations today are managing

separate infrastructures for traditional, virtualized, and cloud-based
applications. Those disparate workloads are not going away anytime
soon, so if IT is going to standardize on a composable infrastructure,
then it must support the composability of bare metal, virtual machines,
and containers (e.g., Docker). Otherwise, the vision of a composable
infrastructure will not have been realized.

What is HPE Synergy?

HPE Synergy is the platform designed and marketed by Hewlett Packard
Enterprise (HPE) to deliver composable infrastructure capabilities to the
datacenter. Released in 2016, HPE Synergy is regarded by many as
the leading platform for composable infrastructure. HPE has been
investing significant engineering effort, over many years, to develop
Synergy. The engineering effort continues today as the Synergy platform
continues to expand and improve. Their efforts are paying off as
Synergy is catching on with customers. In 2018, Synergy was HPE’s
fastest growing new technology category, reaching over 1,600 customers
and generating over $1 billion in annual revenue. In the sections below,
we describe the various components of the HPE Synergy platform and
how they are designed to work together to provide a truly composable
infrastructure.
HPE Synergy 12000 Frame
The HPE Synergy 12000 Frame is a 10U rack-mountable unit that
aggregates compute, storage, networking fabric and management
modules in a single enclosure. The frame provides the foundation of an
HPE Synergy Composable Infrastructure. Uniquely architected to work
with the intelligent software management capabilities provided by HPE
OneView, multiple HPE Synergy Frames can be linked together to scale
the infrastructure while maintaining a single view of the entire network.

A single frame can support a wide variety of different modules and

configurations vary depending on the workloads and software
applications. You can find a variety of reference architectures in
HPE’s Information Library including configurations to support popular
applications such as: SAP HANA, VMware Cloud Foundation, Oracle
Database as a Service, VDI, Citrix XenApp, SAS 9.4, and many more.

Figure 1 below shows the front view of an HPE Synergy 12000 Frame
configured to illustrate the variety of different modules that are
available. Up to 12 2-socket blade servers, or a combination of data
storage modules and servers, can be installed in a single frame. The
various modules, numbered (1) - (8), are discussed in more detail in later
sections.
Figure 1. HPE Synergy 12000 Frame - Front View

The rear view of the HPE Synergy 12000 Frame is shown below in
Figure 2. Here, you can see examples of the various types of fabric
modules that are available. A frame has 6 interconnect bays that can be
configured for up to 3 fabrics (e.g., SAS, Ethernet, and Fibre Channel).
Each fabric typically is supported by a pair of redundant interconnect
modules.
Figure 2. HPE Synergy 12000 Frame - Rear View

In addition to the fabric modules, the rear view shows 10 fans and 6
power supplies. All 10 fans are required and at least 2 power supplies.
The number of power supplies required depends on the requirements of
the installed modules which can be determined by consulting the HPE
Power Advisor Online.

On the left side of Figure 2, notice that there are two frame link
modules. These provide management uplinks and support the multi-
frame ring architecture. Figure 3 shows a closeup view of a frame link
module.
Figure 3. HPE Synergy Frame Link Module

Each frame link module contains a MGMT port for connecting to the
management network and a LINK port for connecting multiple frames in
a ring topology. For redundancy, each frame contains two of these
modules.

Multi-Frame Architecture
A single frame can be managed individually simply by connecting the
MGMT port of both of the redundant frame link modules to the
management network. In this case, the LINK port is not used. Multiple
frames can be managed as a system in this way, with each with an
individual frame connected to the same subnet. However, this
configuration is discouraged because it does not take advantage of
important HPE Synergy management features such as uplink
consolidation and automatic discovery.
Figure 4, below, illustrates the preferred multi-frame architecture -
called a ring topology, This diagram shows 3 frames within a single rack
linked in a ring.

Figure 4. Ring Configuration of Frames

The frames are connected in a ring using the LINK ports on the frame
link modules. As illustrated by the blue lines in Figure 4, Frame 1
connects to Frame 2 which connects to Frame 3 which connects back to
Frame 1. Connectivity to the management network (green lines) is
provided by a single uplink using the MGMT port on Frame 1 with a
redundant uplink provided by Frame 3. Connectivity with the
management network and between the frames is managed automatically
by the frame link module and does not require user configuration.

The actual features that manage the Composable Infrastructure are

delivered by the HPE Synergy Composer (see item 8 in Figure 1) which
is described in the next post in this series.

About IIS (International Integrated Solutions)

This series of posts on Composable Infrastructure and HPE Synergy is
published by International Integrated Solutions (IIS), is a managed
service provider and system integrator with deep expertise in HPE
Synergy. IIS is a distinguished HPE partner, winning HPE Global
Partner of the Year in 2016 and Arrow’s North American Reseller
Partner of the Year in 2017.

As a service provider, IIS brings broad datacenter experience as well as

expertise in HPE Synergy. Having solved a myriad of problems for
hundreds of customers, we bring a holistic view of the datacenter. In
particular, IIS can help with:

 Sizing - providing an assessment methodology and tools to spec

out your workloads.
 Migration Plans - helping you refresh your hardware and migrate
applications to a HPE Synergy.
 Integration - understanding how to integrate your existing
infrastructure with HPE Synergy effectively.
 Managed Services - providing remote monitoring and ongoing
support for your composable infrastructure.
HPE Synergy Composer - Powered by HPE OneView
The HPE Synergy Composer is a hardware appliance powered by the
HPE OneView management software. Each HPE Synergy system,
whether individual or multi-frame, is managed by a pair of redundant
HPE Synergy Composers. The HPE Synergy Composer manages the
system, providing:

 Fluid pools of resources - Compute, storage, and fabric resources

can be configured, released back to the pool, and reconfigured to
support a variety of workloads.
 Software defined infrastructure - Logical infrastructures composed
in the HPE OneView interface are realized nearly instantly through
template-driven operations.
 Unified API access - IT operational processes can be automated
with programming scripts that leverage a single API to manage
every infrastructure element.

As a result, you can deploy, monitor, update, and manage infrastructure,

throughout its life cycle, from a single user interface provided by HPE
OneView.
  Figure 5. HPE Synergy Composer 

A Unified Interface

Traditionally, IT has been managed using separate software tools for

servers, storage, and fabric. These management tools typically had
different APIs, different graphical user interfaces (GUIs), and different
authentication requirements.

Designed to unify these disparate tools, HPE OneView was built from
the ground up to provide a single platform for infrastructure
management. While reducing complexity, HPE OneView improves
productivity and increases flexibility. With a unified interface, it
becomes easier for an individual staff member to manage compute,
storage, and network resources, rather than being siloed into managing a
single resource type. Often referred to as a “single pane of glass”, the
unified interface enables administrators to automate management and
maintenance tasks across all resources in the datacenter.

Introduced in 2013, OneView was originally delivered as a management

platform for BladeSystem c7000 and ProLiant Gen7 and Gen8 servers.
Within HPE Synergy, HPE OneView is delivered by the HPE Synergy
Composer hardware appliance.

Detection and Configuration

When a new frame or module is added to an HPE Synergy system, HPE

OneView automatically detects all the hardware components and
prepares them for monitoring and management. To illustrate the
configuration process, consider what happens when a server module is
added to a frame. Initially, the server hardware has no profile and is
available in a global pool, detected, but in an unconfigured state.
Through HPE OneView, a server profile can be applied, and the
hardware is dynamically configured. Suppose that the profile is for a
web application. Then, a hypervisor is installed, several virtual machines
are deployed, web servers are installed on each VM, a load balancer is
configured, etc. HPE OneView ensures that this server profile will
successfully deploy based on deep knowledge of the underlying
hardware and the software being installed. When the web application is
no longer needed, the profile can be removed and the server capacity
returned to the pool.

Dashboard

From the HPE OneView Global Dashboard, you can manage thousands
of devices from a single user interface. You can view traffic across the
entire network, or drill down to check on the IOPS being consumed by a
single VM. The dashboard provides a unified view of the health of
servers, profiles, storage systems, enclosures, etc. - at the logical and
physical level. In addition, HPE OneView proactively monitors the
health of your entire infrastructure, and alerts you to problems before
they result in downtime.

Figure 6. HPE OneView Global Dashboard

HPE Synergy Image Streamer

HPE Synergy Image Streamer is an optional appliance used to host and
rapidly deploy software to compose an infrastructure. It is show as item
(7) in Figure 1 from the previous post. Within an HPE Synergy system,
the HPE Synergy Image Streamer appliance complements the HPE
Synergy Composer (OneView) by accelerating the deployment and
configuration of software - often the most time-consuming aspect of
composing an infrastructure. The Image Streamer appliance hosts
“golden images” of software artifacts such as VMware ESXi and Red
Hat Enterprise Linux. Golden images can also include your application
stack, so Image Streamer can also speed deployment of databases,
containers, and custom applications.
In a traditional environment, deploying an OS or hypervisor is slow
because it requires building or copying a software image onto the
servers, possibly with multiple reboots required during the installation
process. HPE Image Streamer bypasses this process by maintaining
golden images in a deployed state as bootable image volumes.

When the HPE Synergy Composer provides a specific server profile, the
HPE Image Streamer can read the specifications and create bootable
images for that server profile from the golden image. These bootable
images are then streamed onto compute modules to compose the
specified servers.

HPE Synergy Composer and Image Streamer appliances provide the

intelligence and rapid deployment capabilities required in a software
defined infrastructure. In the next section, we explore the modules that
provide the fluid pool of compute resources.

Compute Modules
An HPE Synergy Frame accommodates both two-socket and four-socket
compute modules. In Figure 1 from the previous post, the four available
compute modules are shown as items (2), (3), (5), and (6). Together,
these modules offer a variety of performance, scalability, and storage
options to power different types of workloads ranging from financial
applications to web infrastructure, to collaboration software, to VDI. All
of the available compute modules can be configured with or without
internal storage. Diskless configurations work with the HPE Synergy
D3940 Storage Module (described below) to provide a pool of direct
attached storage to all the compute modules within a frame.

HPE Synergy 480 Gen10 Compute Module

The HPE Synergy 480 Gen10 is a half-height, single wide, 2-socket

compute module. See item (6) in Figure 1 from the previous post.
Designed for general-purpose enterprise workloads, the 480 is often
considered the workhorse compute module for the Synergy platform. It
supports a variety of server operating systems and hypervisors. In
addition, the 480 is available with optional GPU adapter options that
enable support for VDI (e.g., Citrix XenDesktop) and desktop operating
systems including Windows and Red Hat Enterprise Linux Desktop. 

 Figure 6. HPE Synergy 480 Gen10 Compute Module

HPE Synergy 660 Gen10 Compute Module

Designed for demanding workloads, high-availability, and virtualization

density, the HPE Synergy 660 Gen10 is a full-height, single wide, 4-
socket compute module. See item (2) in Figure 1 from the previous post.
Delivering essentially double the capabilities of the 480, the 660 can
accommodate up to 61.2 TB of internal data storage (4 x 15.3 TB Hot
Plug SFF SAS SSD). By comparison the 480 has 2 drive cages and can
accommodate up to 30.6 TB of internal data storage (2 x 15.3 TB Hot
Plug SFF SAS SSD). The 660 has 48 DIMM slots (12 per processor x 4)
compared with the 480 which has 24 DIMM slots (12 per processor x 2).

Figure 7. HPE Synergy 660 Gen10 Compute Module

HPE Synergy 620 Gen9 Compute Module

The HPE Synergy 620 Gen9 Compute Module is designed for

applications that are more memory than CPU intensive. See item (5) in
Figure 1 from the previous post. It is a is a full-height, single wide, 2-
socket compute module that provides the twice the number of DIMM
slots (24) per processor compared with the 660 (12). As of this writing, a
Gen10 version of the 620 is not yet available.

Figure 8. HPE Synergy 620 Gen9 Compute Module 

HPE Synergy 680 Gen9 Compute Module

The HPE Synergy 680 Gen9 Compute Module is a full-height, double

wide, 4-socket compute module. See item (3) in Figure 1 from
the previous post. The 680 is a mission-critical compute module built to
meet the needs of demanding enterprise workloads. It is designed for
financial, insurance, healthcare, manufacturing, retail, and other
organizations requiring high availability and real-time performance. The
680 provides 24 DIMM slots per processor and up to 61.2 TB of internal
data storage (4 x 15.3 TB Hot Plug SFF SAS SSD). As of this writing, a
Gen10 version of the 680 is not yet available. 
Figure 9. HPE Synergy 680 Gen9 Compute Module

Those are the four compute modules currently offered for the HPE
Synergy platform. Next, we look at the shared data storage options that
are available.

Data Storage
The HPE Synergy platform supports a variety of storage options, from
direct-attached SAS within the frame, to Fibre Channel-attached Tier 1
storage, to Ethernet-attached predictive flash solutions. There is support
for file, block, or object-based storage. No matter which options you
choose, all data storage within the HPE Synergy framework is
composable and managed by the HPE Synergy Composer with HPE
OneView.

HPE Synergy D3940 Storage Module

A direct attached storage solution (DAS) for compute modules within

the same frame, the HPE Synergy D3940 Storage Module provides 40
Small Form Factor (SFF) drive bays. It can provide composable storage
for up to 10 compute modules within a frame via the HPE Synergy 12Gb
SAS Connection Module. The D3940 is shown in the front view of the
frame as item (1) in Figure 1 from the previous post.

Up to 5 D3940 modules can be accomodated in a single frame, scaling

up to 200 SFF drives. This storage module can slide out from the frame
to service drives or I/O adapters without interrupting the operation of
other drives within the module. Optimized for SSDs and leveraging
sixteen 12 Gb/s SAS lanes, a single Synergy D3940 module can deliver
up to 2M IOPs.
 Figure 10. HPE Synergy D3940 Storage Module

The HPE Synergy D3940 Storage Module supports a wide variety of

workloads and is included by HPE in numerous reference architectures
including VMware Cloud Foundation and Microsoft SharePoint and
Exchange.

HPE Synergy D3940 with HPE StoreVirtual VSA Software

An HPE Synergy D3940 Storage Module can be shared with compute

modules on other HPE Synergy Frames using HPE StoreVirtual Virtual
Storage Appliance (VSA) software. The HPE StoreVirtual VSA
provides the services of a shared storage array using the underlying
direct-attached storage. You get the benefits of a scalable, composable
storage array without the complexity and additional cost of an external
dedicated storage network. 
The HPE StoreVirtual VSA has been optimized to support virtualized
environments running a variety of hypervisors, including VMware
vSphere, Microsoft Hyper-V, and Linux KVM.

HPE 3PAR StoreServ Storage

Although DAS solutions based on the D3940 are great for many
workloads, many IT organizations require the performance, reliability,
and scalability of SAN-based solutions. For those larger-scale enterprise
applications requiring Tier-1 service, HPE 3PAR StoreServ Storage
integrates seamlessly into the HPE Synergy platform. HPE 3PAR
StoreServ flash arrays can scale up to 24 petabytes of usable capacity
per Synergy system, provide three million+ IOPS and sub-millisecond
latency. But, best of all, within the HPE Synergy framework these 3PAR
arrays can be managed via HPE OneView through the HPE Synergy
Composer.

Traditionally, one of the biggest drawbacks of SAN-based storage is

manageability. SAN systems typically have their own network
administration tools and require a dedicated administrator. But the HPE
3PAR StoreServ system is a core part of the Synergy Storage family, a
fully composable resource that can be managed much like the Synergy
D3940 DAS module. Administration happens through HPE OneView,
just like with everything else in an HPE Synergy system. LUN
management, a frequent headache for SAN administrators, can be
handled with the templates and profiles provided by HPE Synergy
Composer.
 Figure 11. HPE 3PAR StoreServ 8000

In addition, HPE InfoSight, a predictive analytics platform originally

offered with HPE Nimble, is now available for HPE 3PAR StoreServ as
well. HPE Infosight can predict storage issues and resolve them
proactively, often before IT administrators are even aware of them.

HPE Nimble Storage

In addition to HPE 3PAR, HPE Nimble Storage can also be incorporated

into an HPE Synergy system. HPE Nimble provides an easy-to-use, all-
flash, 99.9999% available storage solution at an attractive price-point.
High

Typically targeted to smaller datacenters than the HPE 3PAR product

line, nevertheless HPE Nimble provides an excellent storage solution for
moderately sized virtualized environment. In fact, this guide describes
an architecture that combines workhorse HPE Synergy 480 Gen10
Compute Modules with HPE Nimble Storage arrays and Mellanox
switches (described below) to reliably deploy and run a virtualized
infrastructure. 
  Figure 12. HPE Nimble Storage All Flash Array

In this post, we’ve reviewed the composability, compute, and data

storage options available with HPE . In the next and final post, we
consider the networking modules that support the composable fabric.

About IIS (International Integrated Solutions)

This series of posts on Composable Infrastructure and HPE Synergy is
published by International Integrated Solutions (IIS) is a managed
service provider and system integrator with deep expertise in HPE
Synergy. IIS is a distinguished HPE partner, winning HPE Global
Partner of the Year in 2016 and Arrow’s North American Reseller
Partner of the Year in 2017.

As a service provider, IIS brings broad datacenter experience as well as

expertise in HPE Synergy. Having solved a myriad of problems for
hundreds of customers, we bring a holistic view of the datacenter. This
experience makes us well suited to helping you realize the benefits of
composable infrastructure HPE SimpliVity. In particular, IIS can help
with:
  Sizing - providing an assessment methodology and tools to spec
out your workloads.
 Migration Plans - helping you refresh your hardware and migrate
applications to a HPE Synergy.
 Integration - understanding how integrate your existing
infrastructure with HPE Synergy effectively.
 Managed Services - providing remote monitoring and ongoing
support for your composable infrastructure

This is the third post in a 3-part series about composable infrastructure

and how it is supported by the HPE Synergy platform. In the first part of
the series, we examined the benefits of composable infrastructure and
looked at the The HPE Synergy 12000 Frame - a 10U rack-mountable
unit that provides the foundation of an HPE Synergy system. In
the second post we look at the management and composability support
modules, the compute modules, and the data storage modules that can be
configured with the HPE Synergy 12000 Frame.

In this post, we look at the composable fabric and the various hardware
modules and some external storage systems that can be integrated into
an HPE Synergy system.

HPE Synergy Composable Fabric

The HPE Synergy Composable Fabric brings everything together to
form a composable infrastructure. It creates a pool of networking
capacity that can be rapidly configured to provision connectivity for a
broad range of applications. Moreover, the HPE Synergy Composable
Fabric employs a master/satellite architecture to connect multiple frames
in an HPE Synergy system. The master networking module provides the
intelligence and extends connectivity to satellite frames through an HPE
Synergy Fabric Interconnect Link Module. This design consolidates
network connections, reduces complexity, and eliminates the need for
top-of-rack switches. In this manner, all traffic between frames is
east/west through the master. There is no north/south traffic with the risk
of bottlenecks to worry about.

When you add a new frame to an HPE Synergy system, and connect it to
the master frame using an HPE Synergy Fabric Interconnect Link
Module, the frame is automatically discovered. Because the new
connection is east/west directly with the master, the system scales with
no performance penalty on the existing workload.

Figure 13. HPE Synergy Master/Satellite Network

Figure 13 illustrates the master/satellite architecture for an HPE Synergy

system containing 3 frames in a single rack. In this illustration, Frame 1
is the master. You can see that is has two networking modules - one
Virtual Connect Module and one Interconnect Link Module. Frames 2
and 3 are satellites and they connect to Frame via their Interconnect Link
Modules. For redundancy, Frame 2 also has a Virtual Connect Module.
If Frame 1 should go down, or Frame 1’s Virtual Connect Module
should go down, then Frame 2 can take over as the master. Frame 3 has
only interconnect Link Modules, but notice that the top one is connected
to Frame 1 and the lower one is connected to Frame 2 for redundancy.

In addition to this fabric for networking among frames, HPE Synergy

provides support for two other fabrics. These three fabrics are supported,
with full redundancy, by 6 bays in the rear of the frame.

Figure 14. HPE Synergy Fabric Bays

HPE recommends that, as a best practice, these fabrics be used for the
following purposes:

 Fabric 1 primary use—Storage

 Fabric 2 primary use—Storage or networking
 Fabric 3 primary use—Networking

Looking back at Figure 2 from the first post, you can see an example of
these best practices. In Figure 2, slots 1 and 4 contain redundant HPE
Synergy 12Gb SAS Connection Module devices. These provide the
fabric for the in-frame HPE Synergy D3940 Storage Module. Slots 2 and
5 contain redundant Brocade 16Gb Fibre Channel SAN switches. These
provide fabric for an external SAN storage system - perhaps based on
HPE StoreServ technology. Slots 3 and 6 contain an Interconnect Link
Module and Virtual Connect Module. These provide fabric for inter-
frame networking. The arrangement of modules is the same as shown in
Figure 13 for Frame 2, so Figure 2 from the first post illustrates a
satellite frame that can server as a failover master.

In the following sections, we look at some of the modules that provide

support for these three fabrics.

HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy

The HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy is the
master module supporting the composable infrastructure networking
fabric across frames. As described above, each multi-frame HPE
Synergy system needs a master frame with this module, and one of the
satellite frames also needs this module for redundancy.

HPE Virtual Connect is a proven and popular technology from HPE that
decouples the network addresses of the compute modules from external
networks so that changes in compute or network infrastructure do not
require complex coordination among LAN and SAN administrators.
HPE Virtual Connect is widely used in c7000 and other HPE
BladeSystem installations. It is also the default master module in HPE
Synergy environments.
Figure 15. HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy

HPE Synergy Interconnect Link Modules

The HPE Virtual Connect master module works with HPE Synergy
Interconnect Link Modules in the satellite frames to provide intelligent
networking capabilities across an HPE Synergy system. There are two
versions of the Interconnect Link Module: a 10G and 20G. The 10G
version provides 12 x 10Gb Ethernet downlinks to the compute modules
in its host frame and can connect up to 4 satellite frames to a master
frame. Likewise, the 20G version provides 12 x 20Gb Ethernet
downlinks to the compute modules in its host frame and can connect up
to 2 satellite frames to a master frame.

Figure 16. HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy

HPE Synergy 40Gb F8 Switch Module

If you have an existing datacenter environment, with traditional switch

functionality and administration, then you may want to integrate your
HPE Synergy into it. In that case you can use the HPE Synergy 40Gb F8
Switch Module - an Ethernet switch that provides manual control to
network administrators from a command-line interface (CLI). While still
part of the HPE Synergy Fabric, this module provides high-speed
switching capabilities and allows a network administrator to
independently manage the switch within the frame.

Like the HPE Virtual Connect SE 40Gb F8 Module, the HPE Synergy
40Gb F8 Switch Module supports the master/satellite frame architecture
through the HPE Interconnect Link Modules. It can also be monitored
by HPE OneView, although it is not controlled by the HPE Synergy
Composer.

Mellanox SH2200 Switch Module

The Mellanox SH2200 Switch Module for HPE Synergy is a high-

performance, low-latency Ethernet switch offering layer 2 and layer 3
features. It provides 25/50GbE connectivity to each of the Synergy
compute nodes and 40/100GbE to upstream network switches. It is
designed for demanding data center environments requiring advanced
features such as high-performance, deterministic low-latency network
fabric, hardware-based virtualization support, or storage offloading. The
Mellanox SH2200 can support workloads such as high-frequency
trading, network functions virtualization (NFV).

The Mellanox SH2200 Switch Module is managed by industry standard

management resources that allow network functions to controlled by
network administrators. It can be used to connect a Synergy Frame into
an existing, high-performance, network, but does not support direct
connections between frames like the HPE Synergy 40Gb F8 Switch
Module.

Figure 17. Mellanox SH2200 Switch Module for HPE Synergy

The HPE Synergy 10Gb Pass-Thru Module

Customers who want to maintain an existing network can use the HPE
Synergy 10Gb Pass-Thru Module to connect each compute module in a
frame to that network. This pass-through module allows one-to-one
connectivity between a compute module’s network adapters and a top-
of-rack Ethernet switch and requires a port for each compute module
connected to the top-of-rack switch.

Figure 18. HPE Synergy 10Gb Pass-Through ModuleHPE Virtual

Connect SE 16Gb FC Module

The HPE Virtual Connect SE 16Gb FC Module is a Fibre Channel

Storage Area Network (SAN) interconnect that simplifies the
administration of compute module connections with the SAN. It works
in conjunction with the HPE Synergy Composer to enable administrators
to add, replace, and recover Synergy Compute Modules resources on-
the-fly. This module appears as a pass-thru device to the Fibre Channel
network, yet provides integrated switching including 16Gb uplinks to
the SAN.

Figure 19. HPE Virtual Connect SE 16Gb FC Module

Brocade 16Gb Fibre Channel SAN Switch for HPE Synergy

HPE Synergy also supports the Brocade 16Gb Fibre Channel switch for
high-performance, low-latency networking with cut-through mode FC
SAN capabilities. This switch is ideal for financial services, trading
applications, medical imaging, and rendering.

For management, The Brocade 16Gb SAN Switch Module integrates

with HPE OneView, Storage Operations Orchestration Manager,
Brocade BNA, and Fabric Vision.

Figure 20. Brocade 16Gb Fibre Channel SAN Switch for HPE Synergy

The modules discussed above support Ethernet or Fibre Channel

networking with devices external to the frame in which they are
installed. Next, we look that the module that supports Serial-attached
SCSI (SAS) connectivity between the compute modules in a frame and
the data storage modules on that same frame.

HPE Synergy 12Gb SAS Connection Module

The HPE Synergy 12Gb SAS Connection Module supports the SAS
interconnection fabric for the HPE Synergy D3940 Storage Module (see
Figure 10 from the second post). This module can connect up to 10
compute modules in a frame with up to 40 SFF drive bays in the D3940.
The result is a storage fabric, managed by HPE Synergy Composer, that
can compose storage resources to meet the needs of a wide variety of
workloads.
 Figure 21. HPE Synergy 12Gb SAS Connection Module

This nonblocking SAS fabric allows full utilization of flash storage and
can support up to two million IOPs across 10 compute modules. The full
architecture is shown below from the SSDs in the D3940 Storage
Module, to the SAS Connection Modules to the HPE Smart Array
P542D Controller connected to each of the compute modules.

Figure 22. HPE Synergy SAS Infrastructure

Composable Infrastructure Is Ready Today

Particularly if your organization is tired of the complexity of managing
multiple physical infrastructures, you should take a close look at HPE
Synergy. Brining the management of compute, storage, and networking
fabric under the single management umbrella of HPE OneView is a huge
simplification. In addition, you reap the benefits of composability that
enable your datacenter to support a wide variety of workloads from a
single platform. Certainly, before you invest more money upgrading a
variety of disparate, specialized, infrastructure, you should investigate
this alternative. HPE Synergy has caught on with more than 1,600
customers already.

As you’ve seen in this series of blog posts, HPE Synergy is a

comprehensive solution. To help you define the configuration that is
right for you organization, we invite you to talk with the experts here at
IIS.

IIS - Your Partner for HPE Synergy

International Integrated Solutions (IIS) is a managed service provider
and system integrator with deep expertise in HPE Synergy. IIS is a
distinguished HPE partner, winning HPE Global Partner of the Year in
2016 and Arrow’s North American Reseller Partner of the Year in 2017.

As your service provider, IIS brings broad datacenter experience as well

as expertise in HPE Synergy. Having solved a myriad of problems for
hundreds of customers, we bring a holistic view of the datacenter. This
experience makes us well suited to helping you realize the benefits of
composable infrastructure with HPE Synergy. In particular, IIS can help
with:

 Sizing - providing an assessment methodology and tools to spec

out your workloads.
 Migration Plans - helping you refresh your hardware and migrate
applications to a HPE Synergy.
 Integration - understanding how integrate your existing
infrastructure with HPE Synergy effectively.
 Managed Services - providing remote monitoring and ongoing
support for your composable infrastructure
HP Virtual Connect Basics
August 9, 2012 21  By ERIC SHANKS
HP Virtual Connect is a great way to handle network setup for an HP
Blade Chassis.  When I first started with Virtual Connect it was very
confusing for me to understand where everything was, and how the
blades connected to the interconnect bays.  This really is fairly simple,
but might be confusing to anyone that’s new to this technology. 
Hopefully this post will give newcomers the tools they need to get
started.

Downlinks
The HP interconnect modules have downlink and uplink ports.  The
uplink ports are pretty obvious, as they have a port on them that can be
connected to a switch or another device.  The downlink ports are less
obvious.  The downlinks exist between the interconnects and the blade
bays.  For example, in a c7000 chassis there are 16 server bays so an HP
Flex-10 interconnect would have 16 downlink ports, one for each blade.
In the picture below of an HP VC Flex-10 Enet Module, there are 8
uplink ports, which are visible, as well as 16 downlink ports which are
not visible, for a total of 24 ports.

Blade Mapping
Now that we’ve seen that each blade has connections to the interconnect
via the downlink ports, lets take a closer look at how we see what NICs
are mapped to which interconnect bay.  HP Blades have two Lan On
Motherboard (LOM) ports as well as room for two mezzanine cards. 
The mezzanine cards can contain a variety of different types of PCI
devices, but in many cases they are populated with either NICS or
HBAs.
The LOMs and Mezz Cards map in a specific order to the interconnect
bays.
LOM1 – Interconnect Bay 1
Lom2 – Interconnect Bay 2
Mezz1 – Interconnect Bay 3 (and 4 if it’s a dual port card)
Mezz2 – Interconnect Bay 5 (and 6 if it’s a dual port card, 7 and 8 if it’s
a quad port card)
The picture below should help to understand how the HP Blades map to
the interconnect bays.  This example uses dual port mezzanine cards.
LOM Ports with Flex-10
An additional thing can happen if you’ve got LOM FlexNICs as well as
a Flex-10 Ethernet Module or Flex Fabric interconnect module.  You
can subdivide the LOM NICs into 4 Logical NICs.  From here, your
hypervisor or operating system will see 8 NICs instead of the original 2
NICs that would normally be there.  This is an especially nice feature if
you’re running virtualization, as you should now have plenty of network
cards for vMotion, Fault Tolerance, Production Networks, and
management networks.
As you can see from the following screenshot, the LOM NICs will be
seperated into 4 Logical NICs labled 1-a, 1-b … 2-d.

I should also mention that if the interconnect modules are Flex Fabric,
the LOM-1b and LOM-2b could be either an HBA or a NIC,  your
choice.
I know that these concepts seem fairly straight forward now, but to a
beginner this is some very useful information to get started with HP
Virtual Connect.  I hope to have some more blog posts in the future
about configuring networking with Virtual Connect.
Share this:

Introduction
HPEs - Virtual Connect FlexFabric networking, can take a while to wrap your head around. So I
thought I would take a moment to explain what HPE Virtual Connect is, and also share some
knowledge and also resources that you may find useful.
What is Virtual Connect?
Virtual Connect is an HPE based technology that provides the ability to configure and customize
how each of the blade network connections are virtualized/mapped.

Or as HPE describes,

"…a technology to simplify networking configuration for the server administrator using an HP
BladeSystem c-Class environment. The baseline Virtual Connect technology virtualizes the
connections between the server and the LAN and SAN network infrastructure. It adds a hardware
abstraction layer that removes the direct coupling between them. Server administrators can
physically wire the uplinks from the enclosure to its network connections once, and then manage
the network addresses and uplink paths through Virtual Connect software."
In addition to virtual connect, you will be sure to see the name FlexFabric. In short, Flexfabric is
a range of HPE networking hardware products, ranging from blade NICs, ToR switches, and
Blade Chassis switch modules.

Components
Virtual Connect InterConnect Modules - The Interconnect modules plug directly into the
interconnect bays in the rear of the HP BladeSystem c Class enclosure. The modules connect to
the server blades through the enclosure midplane.[1] Each Interconnect bay is numbered from 1 to
8.

FlexFabric Adapters - There are two types of FlexFabric adapter - FlexibleLOM and
Mezzanine. Each type is installed into its relating blade system board port/connector.
Figure 1 - C7000 Overview.[2]

Server Blade Connection/Interconnect

Bay Mappings
The system board of each blade allows for FlexibleLOM (to the onboard LOM/NIC) or
Mezzanine adapters to be installed. Each Interconnect bay is mapped to either the FlexibleLOM
or Mezzanine of each blade, like so:

Figure 2 - Interconnect Mappings.[3]

To further visualize these mappings, please see below:

Figure 3/4 - Interconnect Bays within chassis.[4][5]

Network Connection Partitioning

Virtual Connect provides the ability to partition the ports of the FlexibleLOM adapter into a
number of smaller NICs, which are called FlexNICs.

Below provides an overview of the network connection virtual connection mappings:

Interconnect Bay Adapter Type Adapter Port Network

LOM1:1a 1 FlexibleLOM (FLB) 1 Network A

LOM1:1b 1 1 Network B

LOM1:1c 1 1 Network C

LOM1:1d 1 1 Network D
LOM1:2a 2 2 Network A

LOM1:2b 2 2 Network B

LOM1:2c 2 2 Network C

LOM1:2d 2 2 Network D

With regards to the Mezzanine adapters, each port of the adapter is presented to the system/OS
as a single port (shown below):

Figure 5 - Network Allocation.[6]

Bandwidth Allocation

RMC Commands to trouble-shoot

HPE SuperDome Flex
 
 
 
 
 
 
Rate This
1. Display current complex information

RMC cli> show complex

2. Display current attached chassis

RMC cli> show chassis list

3. Display current health status

RMC cli> show health

4. Display current partition inside the server

RMC cli> show npar

5. Display components that have been indicted

RMC cli> show indict

6. View event in Core Analysis Engine

RMC cli> show cae

7. Acquit indicted hardware

RMC cli> acquit all

8. To Power On partition
RMC cli> power on

9. To Power Off partition

RMC cli> power off