Mark Olson

olsonm@us.ibm.com
February 2009

Active Memory Expansion

2010IBM Corporation

IBM Power Systems

Active Memory Expansion

True
memory

True
memory

True
memory

Expand
memory

Expand
memory

Expand
memory

True
memory

True
memory

True
memory

Expand
memory

Expand
memory

Expand
memory

Effectively up
to 100% more
memory

POWER7 advantage for AIX 6.1

Expand memory beyond physical limits
More effective server consolidation
Run more application workload / users per partition
Run more partitions and more workload per server

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Basic Concept

120 GB LPAR1

96 GB LPAR1

MM MM
MM MM
MM MM
MM MM
MM MM
MM MM

M
G

y in
r
o
m
d me
n
a
p
: ex ion
1
o
i
r
a
tit
Scen rained par
const 20 GB
1
96

MM MM G
MM MM G
MM MM G
MM MM G
MM MM G
MM MM G

Assumes
25%
expansion

Physical/true memory
Gained memory capacity
2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Basic Concept

120 GB LPAR1

96 GB LPAR1

MM MM
MM MM
MM MM
MM MM
MM MM
MM MM

M
G

y in
r
o
m
d me
n
a
p
: ex ion
1
o
i
r
a
tit
Scen rained par
const 20 GB
1
96
Scen
a
capa rio 2 H
c
o
capa ity, use ad ld LPAR1
cit
d
96 y for LPAR itional
96+4
8=1 2
44 G
B

Physical/true memory
Gained memory capacity

MM MM G
MM MM G
MM MM G
MM MM G
MM MM G
MM MM G
96 GB LPAR1

MM G G
MM G G
MM M G
MM M G
MM M G
MM M G

Assumes
25%
expansion

48 GB LPAR2

MM
MM
MG
MG
MG
MG

Assumes
50%
expansion

2010 IBM Corporation

IBM Power Systems

Sample SAP ERP Workload, Single Partition

(DataBase + AppServer)
With Active Memory Expansion

Without Active Memory Expansion

Partition utilization

Partition utilization

Memory: 100% (18 GB true)

CPU: 88% (12 cores in LPAR)

Memory: 100% (18 GB)

CPU: 46% (12 cores in LPAR)

Note: Most of
the CPU increase is due to additional work done on
server

Memory capacity is the bottle-neck

CPU is under-utilized
Handles 1000 simulated users

Higher throughput enabled with the same

amount of physical memory
Gain 37% memory capacity
Handles 1700 simulated users

Max Partition throughput: 99 tps

12-core POWER7 partition
18 GB Memory
18 GB true
.
0 GB expanded

+ 65%

Max Partition Throughput: 166 tps

12-core POWER7 partition
24.7 GB Memory
18 GB true
.
6.7 GB expanded

Expanded Memory
Note: This is an illustrative scenario based on using a sample workload. This data represents
measured results in a controlled lab environment. Your results may vary.
2010 IBM Corporation

IBM Power Systems

Sample SAP ERP Workload, Enabled Additional

Application Partition
With Active Memory Expansion

Without Active Memory Expansion

System Utilization

System Utilization

Memory: 100% (48 GB)

CPU: 94% on 32 cores

Memory: 100% (48 GB)

CPU: 76% on 24 cores
25% (8 core) unused

Note: Majority CPU increase is due to additional work.

Higher throughput enabled

Memory Capacity is the bottle-neck

Enabled unused CPU resources (25% of server)

with no addl physical memory.
Gain 30% in application server memory capacity
Handles 5000 simulated users

CPU is under-utilized
Handles 2900 simulated users
Partitions have reached physical memory
limitations by showing moderate paging

+ 60%

System Throughput: 286 TPS

3 x 8-core POWER7 partitions

System Throughput: 460 TPS

4 x 8-core POWER7 partitions

LPAR 4 (AppServer)

LPAR 3 (AppServer)

LPAR 2 (AppServer)

LPAR 1 (DB + App)

48 GB true
.
14 GB expanded
LPAR 4 (IDLE)

LPAR 3 (AppServer)

LPAR 2 (AppServer)

LPAR 1 (DB + App)

48 GB true
.
0 GB expanded

Note: This is an illustrative scenario based on using a sample workload. This data represents
measured results in a controlled lab environment. Your results may vary.

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion - CPU & Performance

% CPU
utilization
for
expansion

no free lunch,
but can be a
really good deal

Amount of memory expansion

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion - CPU & Performance

2
1

% CPU
utilization
for
expansion

1 = Plenty of spare
CPU resource
available

Very resource effective

2 = Constrained CPU
resource already
running at significant
utilization

Amount of memory expansion

There is a knee-of-cure relationship for CPU resource required for memory
expansion
Busy processor cores dont have resources to spare for expansion
The more memory expansion done, the more CPU resource required

Knee varies depending on how compressible memory contents are

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Deployment Steps

2

60-Day Trial

Planning Tool

A. Permanently enable
Active Memory
Expansion
B. Deploy workload into
production
C. Continue to monitor
workload using AIX
performance tools

Memory Expansion

Performance

App. Performance

Actual Results

Estimated Results

Memory Expansion

Deploy into Production

A. One-time, no-charge
temporarily
enablement
B. Config LPAR based
on planning tool
C. Use AIX tools to
monitor Act Mem
Exp environment
D. Tune based on
actual results
CPU Utilization

CPU Utilization

A. Part of AIX 6.1 TL4

B. Calculates data
compressibility &
estimates CPU
overhead due to
Active Memory
Expansion
C. Provides initial
recommendations

Memory Expansion

Time

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Product Structure

One-time, 60-day trial - No charge
Request via Capacity on Demand Web page
www.ibm.com/systems/power/hardware/cod/

Permanent Enablement - Chargeable

Power 750
#4792 Act Mem Exp Enablement Feature
Power 770 & Power 780
#4791 Act Mem Exp Enablement Feature
ONE feature per server no matter how many partitions choose to
use it

Note: Enablement does not mean function has to be used. Enablement allows Act Mem Exp to
be used on any or all of the AIX partitions selected by the client
2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Decision Making

Start off with adequate true physical memory
Not all AIX 6.1 applications/data allow effective expansion
Active Memory Expansion less effective when CPU utilization is high.
Be sure to consider peak workloads.

Best use: Active Memory Expansion as a way to expand beyond

physical memory limits
Before using to reduce memory purchase, need to understand how
well current and future application data compresses and how much
processor resource available for memory expansion
(compression/decompression)
Additional processor resource part of AIX 6.1 partition and requires
licensing like any other active core

If unsure of your expansion value, consider purchasing true physical

memory initially and then using Active Memory Expansion as a
future MES order after analyzing its use.

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Economics

Three Scenarios
Scenario A: Server has maximum possible true memory and CPU utilization is
relatively low. Memory limits constrain server from doing more work.
Active Memory Expansion is a no brainer with even modest memory expansion

Scenario B: Server is not at maximum true memory and CPU utilization relatively
low. Choosing between adding more true memory and Act Mem Exp.
Calculate gained memory: true memory x expansion factor = gained memory. For
example, 96 GB true memory with a 30% expansion = 29 GB gained.
Compare cost of additional true memory versus cost of Active Memory Expansion. If
costs are equal, then chose true memory. But if memory expansion offers savings, then
order it with an appropriate amount of true memory.
As appropriate, include additional software licensing, memory activations, processor
activations in the analysis.

Scenario C: Server is full of smaller DIMMs, but more memory is needed.

Removing smaller memory DIMMs and purchasing larger DIMMs not desirable.
Analysis same as to Scenario B. Need to project an expansion factor.

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion - Planning Tool

Active Memory Expansion Modeled Statistics:
----------------------Modeled Expanded Memory Size :
8.00 GB
Expansion
Factor
--------1.21
1.31
1.41

1.51
1.61

True Memory
Modeled Size
-------------6.75 GB
6.25 GB
5.75 GB

5.50 GB
5.00 GB

t
u
p
t
ou

le
p
m
Sa CPU
Memory

Modeled
Gain
----------------1.25 GB [ 19%]
1.75 GB [ 28%]
2.25 GB [ 39%]

2.50 GB[ 45%]

3.00 GB [ 60%]

Usage
Estimate
----------0.00
0.20
0.35

0.58
1.46

Active Memory Expansion Recommendation:

--------------------The recommended AME configuration for this workload is to configure
the LPAR with a memory size of 5.50 GB and to configure a memory
expansion factor of 1.51. This will result in a memory expansion of
45% from the LPAR's current memory size. With this configuration,
the estimated CPU usage due to Active Memory Expansion is
approximately 0.58 physical processors, and the estimated overall
peak CPU resource required for the LPAR is 3.72 physical processors.

This sample partition

has fairly good
expansion potential
A nice sweet spot for
this partition appears to
be 45% expansion
2.5 GB gained memory
Using about 0.58 cores
additional CPU
resource

Tool included in AIX 6.1 TL4 SP2

Run tool in the partition of interest for memory expansion.
Input desired expanded memory size. Tool outputs different real memory
and CPU resource combinations to achieve the desired effective memory.
2010 IBM Corporation

IBM Power Systems

Act Mem Exp Turning a Partition On or Off

With HMC, check Active Memory
Expansion box and enter
true and max memory
memory expansion factor
To turn off expansion, unclick box
Partition IPL required to turn on or
off

Active Memory Expansion Modeled Statistics:

----------------------Modeled Expanded Memory Size :
8.00 GB
Expansion
Factor
--------1.21
1.31
1.41
1.51
1.61

True Memory
Modeled Size
-------------6.75 GB
6.25 GB
5.75 GB
5.50 GB
5.00 GB

output
e
l
p
m
Sa

Modeled Memory
Gain
----------------1.25 GB [ 19%]
1.75 GB [ 28%]
2.25 GB [ 39%]
2.50 GB [ 45%]
3.00 GB [ 60%]

5.5 true
8.0 max

CPU Usage
Estimate
----------0.00
0.20
0.35
0.58
1.46

Active Memory Expansion Recommendation:

--------------------The recommended AME configuration for this workload is to
configure the LPAR with a memory size of 5.50 GB and to configure
a memory expansion factor of 1.51. This will result in a memory
expansion of 45% from the LPAR's current memory size. With this
configuration, the estimated CPU usage due to Active Memory
Expansion is approximately 0.58 physical processors, and the
estimated overall peak CPU resource required for the LPAR is 3.72
physical processors.

2010 IBM Corporation

IBM Power Systems

Act Mem Exp Operations Considerations

Active Memory Expansion is transparent to applications
A server using Active Memory Expansion needs an HMC on the server.
Enabling Active Memory Expansion: does NOT require a server IPL
Turning Active Memory Expansion on/off for a partition DOES require an
IPL of that partition.
However, changing the expansion factor (leaving Active Memory Expansion
on) does NOT require an IPL. If you set expansion factor to 1.0 , it is
effectively the same as memory expansion turned off with one caveat, Active
Memory Expansion uses the smaller memory page sizes, not some of the
larger page sizes. For a small percentage of clients this page size may be a
factor.

Hardware & software requirements

POWER7 servers running in POWER7 mode (Power 750, 770, 780)
HMC: V7R7.1.0.0 or later
Firmware: 7.1 or later
AIX 6.1 TL4 SP2 or later

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Monitoring Capability

After Active Memory Expansion enabled and turned on

Single LPAR monitoring

Live monitoring of metrics for a LPAR (compression ratio, number of compression
ops, actual amount of CPU used for compression, amount of memory that is
compressed
Provides indication of actual memory expansion levels achieved
System level monitoring (across multiple LPARs)
Live monitoring of expansion metrics for multiple LPARs on a system
Record metrics over time for archiving and historical analysis

Monitor Active Memory Expansion occasionally

to see how much CPU you are consuming for compression and
to monitor whether the workload is compressing well enough to meet the memory
expansion targets.

Monitoring tools provided as no-charge enhancements to existing

performance tools already shipped with AIX and familiar to AIX
administrators.
Examples: lparstat, vmstat, topas, svmon

Run as stand-alone utility

Included with AIX 6.1 and TL4 SP2 at no additional charge

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion - Documentation

AIX publications in Infocenter at
http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp
Sections/books with Active Memory Expansion content:
AIX Commands Reference
Detailed information about the AIX commands/utilities/tools including various
performance tools that support Active Memory Expansion
Information on Active Memory Expansion planning tool (amepat)
AIX Performance Management Guide
Overview describing Active Memory Expansion and how to use it.

White papers
Active Memory Expansion: Overview and Users Guide by David Hepkin
www.ibm.com/systems/power/resources/index.html (then click on white papers)
or www.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs
Active Memory Expansion: Performance Considerations by Dirk Michel
www.ibm.com/systems/power/resources/index.html (then click on white papers)
or www.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs

Movie on introduction, technology, use, installation, operation

(18 min)

www.ibm.com/developerworks/wikis/display/WikiPtype/Movies by Nigel Griffiths

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion & Active Memory Sharing

Active Memory Expansion
Effectively gives more memory
capacity to the partition using
compression / decompression of
the contents in true memory
AIX partitions only

Active Memory Expansion

Active Memory Sharing

Moves memory from one partition
to another
Best fit when one partition is not
busy when another partition is
busy
AXI, IBM i, and Linux partitions

Active Memory Sharing

Supported, potentially a very nice option

Considerations
Only AIX partitions using Active Memory Expansion
Active Memory Expansion value is dependent upon compressibility of
data and available CPU resource
2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Miscellaneous

If using Live Partition Mobility and if using Active Memory Expansion, both
the from and the to servers must have Active Memory Expansion in
place.
If a server gets really busy, memory expansion wont stop. It just means
that the workload will slow down, just as it would if Act Mem Exp was not
in-use and the workload maxed out the CPU.
You can use Active Memory Expansion in the shared processor pool if the
server is enabled. Each partition in the pool turns Act Mem Exp on or off
independently. Each partitions expansion factor is independent.
At the end of the 60-day trial period (assuming no permanent enablement),
then:
no additional partitions can be enabled for memory expansion
An existing partition already enabled for memory expansion may not be able to
adjust its expansion value except to change it to 1.0 (no expansion)
When an existing partition is rebooted, it loses memory expansion capability
Note permanent memory expansion capability activation can be added
dynamically before the end of the trial period. This preserves the existing
partitions ability to use memory expansion without a reboot.

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion Summary

Innovative POWER7 technology
For AIX 6.1 or later
For POWER7 servers

Uses compression/decompression to effectively expand the true

physical memory available for client workloads
Often a small amount of processor resource provides a
significant increase in the effective memory maximum
Processor resource part of AIX partitions resource and licensing

Actual expansion results dependent upon how compressible the

data being used in the application
A SAP ERP sample workload shows up to 100% expansion,
Your results will vary
Estimator tool and free trial available

2010 IBM Corporation

IBM Power Systems

Active Memory Expansion

True
memory

True
memory

True
memory

Expand
memory

Expand
memory

Expand
memory

True
memory

True
memory

True
memory

Expand
memory

Expand
memory

Expand
memory

Effectively up
to 100% more
memory

POWER7 advantage
Expand memory beyond physical limits
More effective server consolidation
Run more application workload / users per partition
Run more partitions and more workload per server

2010 IBM Corporation

IBM Power Systems

Special notices
This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available
in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM
offerings available in your area.
Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources.
Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give
you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk,
NY 10504-1785 USA.
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives
only.
The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or
guarantees either expressed or implied.
All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the
results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations
and conditions.
IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions
worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment
type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal
without notice.
IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.
All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are
dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in
this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generallyavailable systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document
should verify the applicable data for their specific environment.

2010 IBM Corporation

IBM Power Systems

Special notices (cont.)

IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/OS, i5/OS (logo), IBM Business Partner
(logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pSeries, Rational, RISC
System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xSeries, z/OS, zSeries, AIX 5L, Chiphopper, Chipkill, Cloudscape, DB2
Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Purpose File System, , GPFS, HACMP, HACMP/6000, HASM,
IBM Systems Director Active Energy Manager, iSeries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power
Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2,
POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, System i, System p, System p5, System Storage, System z, Tivoli Enterprise, TME 10,
Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other
countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols
indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law
trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml

The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.
UNIX is a registered trademark of The Open Group in the United States, other countries or both.
Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.
Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.
Intel, Itanium, Pentium are registered trademarks and Xeon is a trademark of Intel Corporation or its subsidiaries in the United States, other countries or both.
AMD Opteron is a trademark of Advanced Micro Devices, Inc.
Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries or both.
TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).
SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are
trademarks of the Standard Performance Evaluation Corp (SPEC).
NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.
AltiVec is a trademark of Freescale Semiconductor, Inc.
Cell Broadband Engine is a trademark of Sony Computer Entertainment Inc.
InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association.
Other company, product and service names may be trademarks or service marks of others.

2010 IBM Corporation