FB12 Bowman

Gen-Z Technology:
Enabling Memory Centric

Architecture
genzconsortium.org
Why Gen-Z?
genzconsortium.org
genzconsortium.org © Gen-Z Consortium 2017 2
Why Gen-Z?
genzconsortium.org
Why Gen-Z?
Businesses’ Need to ‘Monetize’ Data
Big Data
AI
Machine Learning
Deep Learning
BI
genzconsortium.org
Why Gen-Z?
genzconsortium.org
Challenge: Multiple Memory Types Need Support
Storage Class
Memory Operations Memory (SCM)Block Operations
Processor SRAM DRAM Flash Hard Drive

1 ns 5 ns 100 ns 200,000 ns 10,000,000 ns
Tape
40,000,000,000 ns
• Byte Addressability • Data Protection methods (RAID, EC)
• High Bandwidth/Low Latency • Device management: Hotplug-ability
• Simple & Efficient Load/Store operations • Rich Data Services (ie deduplication,
replication, thin provisioning, etc)
genzconsortium.org
Challenge: Memory Bandwidth Per Core Is Shrinking
# Processor Pins & DDR Channels

7.0
4500 Memory Bandwidth per Core
6.0
4000
8CH
3500
5.0
6CH
3000
4.0
GB/s
2500
2000 3.0
4CH
1500
2.0
1000 3CH
500 1.0
Courtesy: Ideas International

0
0.0
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
I 2012 2013 2014 2016 2017 2019
I I I (8) (12) (18) (22) (28) (~60)
(64)
1366 pins 2011 pins 3647 pins ~ 4200 pins
(Projected)
Pin limitations lead to memory channel limitations that leads to memory bandwidth limitations.
genzconsortium.org
Challenge: We Are Out Of Real Estate & Resources
Power, Cooling, and Physical Limitations
• CPU power consumption is increasing
• DIMM power consumption is increasing
• Number of memory channels is slowly increasing
• Physical size of CPU is increasing
• Physical size of DIMM is increasing
• Speed of Memory Channel is increasing
• Number of Cores in each Socket is increasing
• Dataset Memory Requirements are increasing
CPU1 CPU2
genzconsortium.org
Challenge: IO Bandwidth Per Core is Shrinking
Memory Bandwidth: HBM Memory Bandwidth:
100 GB/s 732+ GB/s
DDR4 6.0
HBM
64 GB/s
PCIe Bandwidth per core
5.0
x16
CPU PCIe
GPU
4.0
GB/s
3.0
x16
x16
• Cores have limited bandwidth as 2.0

the number of cores increases
1.0
Storage NIC • Bottleneck for heterogeneous
compute environment
0.0
2012 2013 2014 2016 2017 2019
(8) (12) (18) (22) (28) (64)
(~60)
(~60)
SAS Ethernet (Projected)
genzconsortium.org
What Can Address These Challenges?
A Memory Semantic Fabric!

CPUs Accelerators
What is a Memory Semantic Fabric?
• A communication protocol that speaks the SoC SoC FPGA FPGA GPU GPU
same language the CPU speaks: Memory Memory Memory Memory
load/store, put/get, and read/write
• Connectivity that extends beyond the Memory Semantic Fabric

server to the rack
Network Storage
Pooled Memory
I/O
Communication at the speed of memory

genzconsortium.org
Gen-Z: A New Data Access Technology
CPUs Accelerators
High Bandwidth, Low Latency
SoC SoC FPGA FPGA GPU GPU
Packetized Memory Semantic Protocol Memory Memory Memory Memory
Abstracted Hardware Interface Gen-Z Direct Attach, Switched, or Fabric Topology
Simplify HW/SW Boundary
Network Storage
Pooled Memory
Compatible & Economical
I/O
Universal democratized communication
genzconsortium.org
Gen-Z Allows Memory Innovation
Processor Media Module
DRAM
Controller
Memory Bus
Media
DRAM
DRAM
DRAM
4-8 Memory Channels

17-25 GB/s / Channel
288pins / DIMM
Synchronous Interface
genzconsortium.org
Gen-Z Allows Memory Innovation
DRAM
Gen-Z Logic
Gen-Z Logic
DRAM
Controller
Media
Gen-Z Memory DRAM
Semantic Fabric
DRAM
2-8 High-speed Serial Links

Low Latency, High performance
Split Memory Controller
Asynchronous Interface
Processor is media agnostic
genzconsortium.org
Gen-Z is Media Agnostic & Composable
Media Module
Processor Media Module Processor
SCM
SCM
Controller
Gen-Z Logic
SCM
Gen-Z Logic
Media
Controller
Gen-Z Fabric SCM CPU
Media
SCM
SCM
SCM
SCM
Storage Class Memory
Gen-Z Switch
GPU
CPU or
FPGA
Media Module Media Module
Media DRAM
Gen-Z Logic
Gen-Z Fabric GPU
Controller
Media
Controller
or
Media
DRAM
Media
FPGA Media DRAM
Media DRAM
Accelerator
Multiple resources enabled by
Universal Interconnect
genzconsortium.org
Gen-Z Creates More Real Estate
• Gen-Z can be used to connect to devices inside and outside the server enabling a composable system
• Gen-Z’s pin efficient interface and ability to multiplex with PCIe pins allows for an overall increase in
memory bandwidth
Rack-Level Pooled
Server
Server Local Memory Memory
Server
Memory
Channels
DRAM, NVM, Processor

SCM Gen-Z Fabric
On Accelerators Pooled
Package
Memory
Switch Accelerators
genzconsortium.org
Fully Composable Through Disaggregation Using Gen-Z
genzconsortium.org
Example Gen-Z System with Gen-Z & PCIe® I/O
Host System
Gen-Z Interface Logic PCIe Root Complex
Physical
Gen-Z
Switch
Gen-Z
Switch
Gen-Z LPD PCIe I/O PCIe Switch Configuration
Component Device
Gen-Z LPD ... Gen-Z LPD Gen-Z LPD ... Gen-Z LPD PCIe I/O ... PCIe I/O
Component Component Component Component Device Device
• Gen-Z systems may contain a mix of Gen-Z I/O components and PCI Express® I/O devices
• PCIe worked immediately with unmodified OSs by appearing to SW as conventional PCI devices & bridges
• Gen-Z is using a similar strategy via the concept of logical PCI devices (LPDs)
genzconsortium.org
I/O Component Sharing
• With suitable Gen-Z fabric management SW,
a single Gen-Z I/O component can be shared
among multiple Gen-Z systems
• Component sharing offers significant availability
benefits, e.g., system failover for continued
operation and avoidance of stranded resources
• Each Gen-Z system running an unmodified OS
only “sees” LPDs assigned to it by Gen-Z fabric
management SW
• Each Gen-Z system’s BDF space for LPDs is
orthogonal to the BDF space in other systems
• A Gen-Z I/O component designed to be shared by multiple systems must ensure
that the virtual device instances do not interfere with each other
• Gen-Z can deliver the major benefits of PCIe MR-IOV* with less cost & complexity
*MR-IOV: Multi-Root I/O Virtualization and Sharing
genzconsortium.org
More Gen-Z Benefits
• Gen-Z’s 112GT/s Connector DDR

 The connector supports vertical, horizontal, right angle and cabled 25-50GB/s
solutions
 Same connectors for memory, accelerators, I/O, etc.
 Gen-Z defined the connector and is donating it to SFF Gen4
• Leverage High Volume Industry PHYs 64GB/s
Gen-Z 4C
 Derivative of the IEEE 802.3 PHY
100-448GB/s
 PCIe PHY
Gen-Z 2C
 Lower development and product costs
50-224GB/s
• Unmodified OS Support Gen-Z 1C
 Minimize cost & effort to adopt Gen-Z solutions 25-112GB/s
• Enables Right Sized Solutions

 Enables the right amount of compute, memory, storage,
acceleration, and networking to be applied to each workload
genzconsortium.org
Open With Broad Industry Support
Consortium Members
• Alpha Data • IBM • PLDA Group
• AMD • IDT • Red Hat
• Amphenol • IntelliProp • Samsung
• ARM • Jabil • Seagate
• Avery Design • Lenovo • SK hynix
Systems • Lotes • Smart Modular
• Broadcom • Luxshare-ICT • Spin Transfer
• Cadence • Mellanox Technologies
• Cavium • Mentor Graphics • TE
• Cray • Micron • Tyco Electronics
• Dell EMC • Microsemi • VMware
• Everspin • Molex • Western Digital
• FIT • NetApp • Xilinx
• HP Enterprise • Nokia • Yadro
• Huawei • Numascale
genzconsortium.org
Gen-Z Summary
• Scalable system interconnect and protocol
• Optimized for memory-semantic communications
• Breaks processor-memory innovation dependency
• Opportunity to simplify software overhead and complexity
• Unmodified OS support
• Common modular connector and mechanical form factors
• Visit the Gen-Z booth, #739, for more information and see the demonstrations
• Attend “Chat with the Experts” tonight to speak with Mike Krause about Gen-Z
at the Fabrics Table
• Visit us at genzconsortium.org
genzconsortium.org
genzconsortium.org

FB12 Bowman

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

FB12 Bowman

Uploaded by

Copyright:

Available Formats

Gen-Z Technology:

Enabling Memory Centric

Processor SRAM DRAM Flash Hard Drive

• Byte Addressability • Data Protection methods (RAID, EC)

• High Bandwidth/Low Latency • Device management: Hotplug-ability

# Processor Pins & DDR Channels

Courtesy: Ideas International

• Cores have limited bandwidth as 2.0

A Memory Semantic Fabric!

same language the CPU speaks: Memory Memory Memory Memory

load/store, put/get, and read/write

• Connectivity that extends beyond the Memory Semantic Fabric

Communication at the speed of memory

Packetized Memory Semantic Protocol Memory Memory Memory Memory

Abstracted Hardware Interface Gen-Z Direct Attach, Switched, or Fabric Topology

Simplify HW/SW Boundary

Universal democratized communication

Processor Media Module

4-8 Memory Channels

Processor Media Module

2-8 High-speed Serial Links

Gen-Z Fabric GPU

DRAM, NVM, Processor

Gen-Z Interface Logic PCIe Root Complex

*MR-IOV: Multi-Root I/O Virtualization and Sharing

• Gen-Z’s 112GT/s Connector DDR

• Enables Right Sized Solutions

You might also like