3PAR Thin Provisioning Best Practices

3PAR Thin Provisioning Best Practices
Trevor Kelly
Document Abstract:
While 3PAR Thin Provisioning software is simple to use, a certain level of planning is
advantageous to maximize the benefits. This white paper documents these best practices and is
intended for administrators looking to get the most out of their 3PAR Utility Storage deployment.
Part Number: 3PAR-tpbest-wp-09.0

Last saved on: 2/11/2009 10:40:31 AM
-2-
Table of Contents
1 Introduction ............................................................................................................................4
2 Overview of 3PAR Thin Provisioning....................................................................................4
3 When to Use Thin Provisioning.............................................................................................5

3.1 Avoiding Storage Capacity Addition to Servers............................................................. 5
3.2 Accelerating Time to Market ........................................................................................... 5
4 Database Considerations......................................................................................................5
5 Chargeback Models...............................................................................................................6
6 When Not to Use Thin Provisioning......................................................................................6
7 TPVV Sizing Guidelines ........................................................................................................7
8 Common Provisioning Groups..............................................................................................7
9 When to Use Separate CPGs for Different VVs...................................................................8

9.1 Use Different CPGs for VVs that Require Different Logical Disk Characteristics........ 8
9.2 Use Different CPGs for Logical Separation or Chargeback Models ............................ 8
10 CPG Space............................................................................................................................9
10.1 CPG Automatic Growth................................................................................................... 9
10.2 Determining CPG Growth Increment ............................................................................. 9
10.3 CPVVs Space Within CPGs ......................................................................................... 10
10.4 Changing LD Characteristics Within a CPG ................................................................ 10
11 CPG Space Usage ..............................................................................................................10

11.1 How TPVVs Use CPG Space (LD Regions) ............................................................... 10
11.2 How CPG Space Is Allocated to TPVVs and CPVVs ................................................. 10
12 Managing CPGs and TPVVs: Warnings and Limits...........................................................11

12.1 Allocation Limits............................................................................................................. 12
12.2 Allocation Warnings....................................................................................................... 13
12.3 Used Physical Capacity ................................................................................................ 13
12.4 Available Free Space Alerts.......................................................................................... 13
12.5 Host System-Based Quotas ......................................................................................... 14
12.6 User Recommendations ............................................................................................... 14
© 2009 3PAR Inc. All rights reserved. 3PAR, the 3PAR logo, Serving Information, InServ, InForm, and InSpire are all registered
trademarks of 3PAR Inc. All other trademarks and registered trademarks are the property of their respective owners.
-3-
13 Meeting Space Limitations ..................................................................................................15

13.1 Running Out Of Space on a TPVV............................................................................... 15
13.2 Running Out of Space with a CPG............................................................................... 15
13.3 Running Out of Space on the System.......................................................................... 15
14 File System Limitations and Considerations.......................................................................15

14.1 Fragmentation and De-Fragmentation......................................................................... 16
15 Reclaiming Snapshot Space...............................................................................................16
16 Reclaiming CPG Space ......................................................................................................16
17 Migration to and from TPVVs..............................................................................................17

17.1 Migrating to TPVVs Using Block-Based Migration ...................................................... 17
17.2 Using 3PAR Full Copy with 3PAR Thin Provisioning .................................................. 17
17.3 Using 3PAR Remote Copy with 3PAR Thin Provisioning........................................... 17
18 CPG Performance Considerations .....................................................................................18
19 Tracking Volume Space Usage ..........................................................................................18

19.1 Base and CPVVs........................................................................................................... 18
19.2 TPVVs ............................................................................................................................ 18
19.3 CPGs.............................................................................................................................. 18
19.4 System Space................................................................................................................ 18
19.5 Total Used Space.......................................................................................................... 19
20 Usage Reporting and Trend Analysis.................................................................................19
21 Licensing..............................................................................................................................19
22 Conclusions .........................................................................................................................20
-4-
1 Introduction
Balancing new project demands and limited IT resources is the challenging yet often thankless
job of IT management. IT managers must constantly evaluate the efficiency of tools and
processes used to serve and manage corporate data. Of the many proposed solutions for
improving efficiency, few are actually implemented. Of those, fewer still achieve demonstrable
success. 3PAR® Thin Provisioning is a simple solution that succeeds by saving organizations
millions of dollars while enabling IT to initiate new projects on demand.
While 3PAR Thin Provisioning software is simple to use, a certain level of planning is
advantageous to maximize the benefits. This paper documents these best practices and is
intended for administrators looking to get the most out of their 3PAR Utility Storage deployment.
2 Overview of 3PAR Thin Provisioning
3PAR Thin Provisioning allows a large Virtual Volume (VV) to be created and made available as
a LUN on a host server without the need to dedicate physical storage until it is actually written.
The 3PAR InForm® Operating System divides physical disks into uniform-sized, 256-MB
“chunklets.” A traditional VV uses Logical Disks (LDs) assembled from chunklets. With 3PAR Thin
Provisioning, LDs are allocated to a Common Provisioning Group (CPG) which can provide
storage as-needed to Thin Provisioned Virtual Volumes (TPVVs). (See Figure 1.) Thin
Provisioning simplifies storage management by allowing large volumes to be created without
paying for more storage than is actually being used.
Figure 1: Overview of thin-provisioned virtual volumes
-5-
3 When to Use Thin Provisioning
3PAR Thin Provisioning can be used with all virtual volumes to dramatically improve storage
utilization; however, to realize the highest value from this feature, you should consider the use
cases discussed in this section.
3.1 Avoiding Storage Capacity Addition to Servers
Storage administrators often allocate large amounts of storage to accommodate longer term
growth requirements. This reduces the number of times that storage has to be added to a
particular server, which can be time consuming and cause system downtime. With 3PAR Thin
Provisioning it is possible to allocate the large amounts of storage but only consume the physical
space as used. The characteristics associated with this type of storage may include:
 Large capacities of provisioned storage to eliminate change at a later time, with relatively
small initial use.
 Relatively slow growth rates or unpredictable growth over time (for example, if an
administrator provisions 1 TB of storage to a system and this capacity is actually used
within a month, then little value is gained from the use of Thin Provisioning)
Examples of the above include: large file systems used for group shares, mailbox database
space, and general database space that grows slowly over time or is expected to grow
significantly but at an unknown time in the future.
3.2 Accelerating Time to Market
During an application’s development stage, there is a requirement for that storage to be in place
and ready to go—in some cases supporting the full capacity of the application—when the
application goes live. With 3PAR Thin Provisioning it is possible to present the storage almost
immediately so that it is ready for the developers to work on without the requirement for the full
amount of physical capacity being in place. Because Thin Provisioning gives storage
administrators the ability to place limits on TPVVs and Common Provisioning Groups (CPGs), the
admin can ensure that the developers’ work does not affect other production systems by using
the free space within the 3PAR system. Once the application is ready to go live, then the required
initial storage can be placed into the system and the limits removed.
4 Database Considerations
Databases normally initialize their storage space and write a pattern into the database before it is
used. In this case, Thin Provisioning will allocate the same physical space as a traditional or “fat”
virtual volume. To make sure that the full value of Thin Provisioning is realized in database
environments, it is important that the database be created with a small initial size, then manually
extended over time or be placed into some form of “auto-extend” mode (for example, with Oracle
Database).
-6-
5 Chargeback Models
Thin Provisioning is ideal for Service Providers and those wishing to deploy a Utility Storage
offering where usage chargeback is an important component of service. Thin Provisioning allows
Service Providers to provision storage very quickly and, as long as the appropriate applications
are selected (as described above), then this technology allows service providers to accomplish
the following:
 Decoupling the charge for storage from the limitations of actual presented storage and
the exposure to disruption for future capacity expansions.
 Increased profit margins based on the savings resulting from the difference in cost for
provisioned storage compared to actual physically used storage.
Charging can be completed at either the individual volume level (usage shown by the showvv
-tpvv and showvv –r -tpvv commands) or at the CPG level (usage shown by the showcpg
and showcpg –r commands).
It s recommended that unique names describing the relevant line of business or departments be
included in the volume and/or CPG names to simplify reporting. The InForm OS allows up to 19
characters for volume names.
6 When Not to Use Thin Provisioning
Thin Provisioning may be used with all LUNs provisioned from the InServ array; however, there
are certain workloads or applications where it does not necessarily make sense to use Thin
Provisioning. This specifically relates to the fact that, for thin volumes, the Thin Provisioning
license charge will be incurred in addition to the physical disk capacity used. The following cases
are examples to illustrate situations where the use of Thin Provisioning does not necessarily
make the most sense:
 Rapidly growing storage areas. If a large TPVV is presented to a host system and the
space is all allocated within three months, for example, the use of Thin Provisioning may
not return optimal ROI.
 Systems with high file system utilization. If the file systems are full, then the benefits
of Thin Provisioning are reduced. Consider any file systems with utilization rates above
80% as inappropriate for use with Thin Provisioning. In this case, it may be more efficient
to use standard virtual volumes to hold this data.
 Databases not in “auto-extend” mode (or the equivalent). Many databases can
initialize the used storage when they are created (Oracle, MS SQL, Sybase, etc). This
has the same effect as provisioning file systems with high utilization and may be
inefficient depending upon the ratio of provisioned storage to database size.
 Small capacity requirements. Thin Provisioning really shows its value for large-scale
consolidation. For a small number of virtual volumes with modest size requirements (256
megabytes up to 10s of gigabytes) then the growth space allocated by the CPGs may
mean that minimal benefit is realized. Use care in selection of the CPG growth increment
in this case (as discussed in Section 10).
-7-
 Customers who are not prepared to maintain their storage. Thin Provisioning
requires only a minimal amount of management, but it does require a willingness to
upgrade the physical storage and Thin Provisioning licenses as needed on an ongoing
basis. The advantage of this is that the customer can closely track their actual storage
usage with the capacity in the InServ, thus minimizing costs. Customers who are not
prepared to dedicate any resources whatsoever to monitoring and managing their 3PAR
systems are not a suitable match for the use of Thin Provisioning.
 Storage requiring large amounts of snapshots space. Snapshot space used by Thin
Provisioned volumes is included in the license usage calculations until InForm OS 2.3.1.
Prior to InForm 2.3.1, if the TPVV has Virtual Copy snapshots and a high update rate,
then the amount of snapshot space consumed—and the amount of the TP license
consumed—may become significant. Again, the efficiency of Thin Provisioning will be
reduced as the combined TPVV and snapshot space increases above 80% of the TPVV
size. Also, with snapshots, it is possible for snapshot space to take N times the TPVV
volume size (where N is the number of snapshots), i.e., much greater than 100%.
7 TPVV Sizing Guidelines
What’s the ideal size for a TPVV? There is no definite answer to this, but you should consider the
following when deciding on the size for thin volumes:
 The minimum size for a TPVV is 256 megabytes and the maximum is 2,096,128 MB
(2 TB minus 1 GB).
 Thin Provisioning demonstrates the most value in situations involving large scale
consolidation. For a small number of virtual volumes with modest size requirements (256
megabytes up to tens of gigabytes), the growth space allocated by the CPGs may mean
that minimal benefit is realized. For this reason, use caution when selecting the CPG
growth increment (as discussed in Section 10).
 It is possible to increase the size of a thin volume. This provides a significant amount of
flexibility. However, the impact of growing volumes for host-based operating system
needs to be considered.
 It is not possible to shrink a TPVV at this time, but Thin Reclamation technology—the
ability to reclaim freed capacity from a TPVV—is planned for a future 3PAR Thin
Provisioning software release in 2009.
 When faced with a choice, it is preferable to make volumes larger than needed over
making them too small. If the volumes that are presented are too small for the ongoing
requirements, then TPVV growth or additional volumes will be required in the future,
which is something that would need to be managed.
 If you create large volumes, then the space is there for the host system or users to use.
Using quotas on the host system is an elegant way to manage potential over use.
8 Common Provisioning Groups
CPGs are policies for how free chunklets within the 3PAR array should be used. CPGs are given
parameters such as growth increment and RAID level. CPGs automatically grow according to the
-8-
stated parameters (see Section 10) and are used specifically for TPVVs and virtual volumes with
3PAR Virtual Copy snapshot space (CPVVs) unless the snapshot space is pre-allocated.
9 When to Use Separate CPGs for Different VVs
There is no limit to the number of CPGs that can be created on the InServ. However, it is most
efficient to keep the number of CPGs as low as possible, as each CPG requires allocation of a
certain amount of growth space.
This section discusses the major issues to consider when planning CPGs for Thin Provisioning
and Virtual Copy snapshots and recommends a set of best practices.
9.1 Use Different CPGs for VVs that Require Different Logical
Disk Characteristics.
All TPVVs associated with a CPG allocate space from a shared pool of LDs. The storage
administrator cannot explicitly specify the use of an LD within a CPG. When auto-grow is used
(this is the default), the LDs are created using the uniform LD characteristics (configured for CPG
creation with the ability to change growth parameters at any time). Therefore, VVs associated
with a single CPG should require identical LD characteristics. VVs that require different LD
characteristics must use different CPGs.
For example:
 Define and use different CPGs for differing service qualities (e.g., RAID 1 or RAID 5 VVs;
Nearline or Fibre Channel disks).
 Define and use different CPGs for VVs restricted to specific controller nodes within the
InServ array, (e.g., VVs associated with nodes 0 and 1 and VVs associated with nodes 2
and 3).
 Define and use different CPGs for VVs for which it is desirable to use different disks to
minimize interaction with each other for performance or availability reasons (e.g., to
separate databases and their transaction logs).
Please also note that it is possible to move TPVVs between CPGs, thereby changing their
underlying physical layout. The tunetpvv command is useful for this purpose.
9.2 Use Different CPGs for Logical Separation or Chargeback

Models
For reporting and management purposes, it may be beneficial to create separate CPGs for
different lines of business, departments, or customers. This allows a logical separation of
resources and may help with chargeback models, as chargeback could be based on CPG space
usage rather than usage at an individual TPVV level.
-9-
10 CPG Space
Fundamental requirements for efficient thin provisioning of storage are fine granularity of
allocation of storage to volumes and fast allocation of additional storage on demand, all without
any administrator intervention. CPGs provide the framework which InForm OS uses to meet
these requirements.
10.1 CPG Automatic Growth
CPGs dynamically allocate storage in increments specified by the CPG’s growth increment,
which defaults to 32 GB. This on-demand allocation serves two roles: it allows a faster response
to the growth demands of TPVVs and is large enough to ensure wide-striping of the CPG’s
storage. Growth is triggered when the CPG’s available space falls below 75% of the growth
increment, or 24 GB for the default growth increment. The CPG growth increment can be
changed at any time, which also changes the threshold, but does not alter space which has
already been allocated.
10.2 Determining CPG Growth Increment
The minimum growth increment for a CPG is 8 GB; the default, as described above, is 32 GB.
Here are the things to consider when selecting a CPG growth increment:
 The CPG growth increment and other parameters (RAID level, which disks to use, etc.)
can be changed at any time. This affects new Logical Disks (LDs) that are allocated for
the CPG to grow into but does NOT impact any LDs already allocated to the CPG.
 TPVVs can be moved to another CPG with 3PAR Dynamic Optimization and the
tunetpvv command.
 A CPG always has a certain amount of growth space allocated as described in Section
10.1. Because of this, there is a compromise with respect to the CPG growth increment
based on these three points:
o The CPG growth increment should be as small as possible to minimize the

amount of growth space.
o For optimal performance, the growth parameter of the CPG needs to be large
enough to create LDs across all disks within the 3PAR Storage System. An 8-GB
growth size using RAID 1 will use 64 chunklets (each on a separate disk, if
possible). If the 3PAR system has 480 disks, then the optimum CPG growth size
would be 60 GB for RAID 1 to use all disks.
o If the environment is performance-intensive (or, more specifically, write-

intensive), the rate of use of CPG space might be significant. In this case it is
recommended that the growth increment be set to a minimum of 8 GB per pair of
Controller Nodes (32 GB for eight Controller Nodes).
- 10 -
10.3 CPVVs Space Within CPGs
In addition to TPVVs, Snapshot Data (SD) space for traditional base VVs can be associated with
a CPG. This is referred to as a CPVV. This association eliminates the planning required to
estimate and dedicate logical capacity required for Virtual Copy copy-on-write snapshots of a VV.
Instead, the snapshot space can grow on-demand as needed.
10.4 Changing LD Characteristics Within a CPG
LD space within a CPG is automatically allocated to VVs as needed. The storage administrator
has no direct control over which LDs are used for a particular allocation. Therefore, in general, it
makes little sense to have LDs with different characteristics in the same CPG. Although the
capability exists to modify the LD characteristics for new LDs within a CPG, its use is advisable
only in an emergency where different LD characteristics enable the CPG to utilize the remaining
available physical capacity.
11 CPG Space Usage
11.1 How TPVVs Use CPG Space (LD Regions)
LDs within a CPG are divided into 128-MB regions. Each TPVV or CPVV is allocated to these
regions as needed. Host writes are mapped to LD regions in 16-KB pages to optimize capacity
usage.
TPVV LD allocations occur at a minimum increment of 128 MB per controller node the CPG uses.
An initial LD allocation of 128 MB (per node) will be mapped from CPG to the TPVV SD space
upon TPVV creation. Subsequent allocations will vary based on the rate of capacity consumption.
By default, an allocation of 128 MB (per node) is mapped from the CPG unless the last LD
allocation amount is consumed within one minute. Anticipating a quick rate of storage
consumption, subsequent LD space allocations will grow progressively by 128 MB (per node) up
to the maximum of 1 GB (per node). As the TPVV exported size or allocation limit is reached, the
last allocation will map 256 MB (per node) over TPVV exported size to ensure that full exported
TPVV address space is usable.
11.2 How CPG Space Is Allocated to TPVVs and CPVVs
Each VV associated with a CPG can allocate space from the following sources in the following
order:
 Unused SD space. Though LDs from the CPG are mapped to the SD space of an
associated VV and its snapshot descendents as needed, this space can grow to be quite
large since the LD capacity of the SD space used by removed snapshots is marked free
but left allocated to the SD space, i.e., SD space of a base VV does not shrink.
For CPVVs or normal Base Volumes (but not TPVVs), the freespace command allows
an administrator to remove previously allocated snapshot space. The only way that free
SD space for a TPVV can be released for use by other VVs in a CPG is to remove the
- 11 -
TPVV. Unused SD space can be displayed using the showvv -s command (the FAdMB
and FSnpMB columns show the free administration and snapshot data space).
 Unallocated CPG LD space. This space is shared by all the VVs associated with the
CPG. Unallocated LD space can grow to be of considerable size since the SA and SD
space of a base VV associated with the CPG are returned to the CPG as unallocated LD
space upon VV removal (or when the freespace command is issued for a CPVV).
CPGs do not automatically remove unused LDs; however, the returned, unallocated LDs
are remapped to the SD space of associated TPVVs or CPVV snapshots as needed over
time. To free this space for other uses (for example, for use by other CPGs), the unused
LDs must be manually removed from a CPG. (Note that only LDs that are entirely unused
can be removed using the removeld command). Unallocated LD space for a CPG can
be displayed using the command showld –cpg cpgname. Please note that the system
will automatically create new LDs, which will show as unused initially, based on the CPG
growth size parameter.
 Free Chunklets. Free Chunklets available to a CPG for creating new LDs may be limited
by the LD creation parameters for the CPG. It is important to understand that if different
CPGs can draw from the same pool of chunklets, chunklets allocated to one CPG can
reduce the pool of storage available to other CPGs. It is recommended that storage
administrators implement the following strategies to stay abreast of available free space:
o Set the CPG allocation warnings (and limits if necessary)
o Monitor free space reduction rate using the command showspace –cpg
<CPGNAME> –hist
o Set the free space capacity warning with the setsys RawSpaceAlert
command (as detailed below)
o Monitor available free space alerts
o Maintain a buffer stock of physical capacity
12 Managing CPGs and TPVVs: Warnings and Limits
The 3PAR Storage System provides multiple categories of alerts that notify storage
administrators of important events. Figure 2 illustrates the categories on which warnings can be
set to ensure the smooth running of a 3PAR Storage System. These include:
 Allocation limits and warnings for TPVVs and CPGs
 Used physical capacity alerts
 Available free space alerts
 Host system-based quotas
- 12 -
Figure 2: Thin Provisioning Usage Management
12.1 Allocation Limits
Allocation Limits provide a mechanism to prevent “run away” applications (applications that are in
an abnormal state and continuously write data to the storage device) from consuming logical
capacity beyond the specified logical capacity threshold. Allocation limits can be specified
independently for each TPVV (and CPVV snapshot space) and each CPG. For a TPVV, once the
allocation limit is reached, the logical capacity allocated to the TPVV stops growing and new
application writes fail. Similarly, for a CPG, once the allocation limit is reached, the automatic
creation of new Logical Devices (LDs), if configured, is disabled. New applications writes to
TPVVs mapped to the CPG receive a write failure as the remaining logical capacity in the CPG is
utilized over time. Relevant commands are:
 setcpg –sdgl <num>[g|G]: sets the CPG growth limit in megabytes (add “g” for
gigabytes, e.g., ”10g”)
 setvv al <percent>: sets the TPVV or CPVV allocation limit as a percentage of the
specified VV size
Note: If a TPVV or CPG reaches an allocation limit, then the 3PAR system will stop writes into
the relevant TPVVs. This can cause unpredictable application and/or host system behavior, so
should be avoided. It is recommended that TPVV, CPG and free space warnings be set to
sensible levels and managed when they are triggered. Please use allocation limits with care.
It is important to note that the allocation limit is a hard limit and the CPG will not grow beyond it.
Any VVs that require more space from the CPG once the hard limit is reached will not be able to
- 13 -
grow and will eventually present write errors to host systems unless the CPG allocation limit is
raised. The CPG warning limit should be set sufficiently below the CPG allocation limit so that it
alerts the storage administrator with ample time to react before the CPG allocation limit is
reached. The CPG allocation limit and CPG warning limit also provide thresholds for alerts on
available free space for a CPG.
12.2 Allocation Warnings
Allocation warnings provide a mechanism for warning storage administrators when a specified
logical capacity threshold is reached. An allocation warning can be specified independently for
each TPVV (and each CPVV) and each CPG. It is recommended that allocation warnings be
used, at least at the CPG level, and managed when they are triggered. Relevant commands are:
 setcpg –sdgw <num>[g|G], which sets the CPG allocation warning (add “g” for
gigabytes, e.g., ”100g”)
 setvv aw <percent>, which sets the TPVV or CPVV allocation warning as a

percentage of specified VV size
 setsys RawSpaceAlert <value>, where value is the remaining capacity in

gigabytes
These alerts and limits can be changed at any time.
12.3 Used Physical Capacity
As available physical capacity across the 3PAR InServ Storage Server is utilized by traditional
Virtual Volumes (VVs) and/or TPVVs, multiple pre-configured alerts are generated which identify
the used physical capacity as a percentage of total system capacity (50%, 75%, 85%, 95%).
Furthermore, it is possible for the storage administrator to use the following command to set a
warning level for when the available space within the 3PAR Storage System falls below a defined
capacity point:
setsys RawSpaceAlert <value>, where value is the remaining capacity in

gigabytes
For CPGs configured with default LD characteristics, these alerts serve as an advance warning to
the storage administrator to plan for and add necessary physical capacity. In the unlikely scenario
that effective physical capacity becomes used, the 3PAR InServ Storage Server prevents new
writes from occurring until more capacity is added.
12.4 Available Free Space Alerts
A history of used and free CPG space is available. The InForm OS samples CPGs and the space
available to the CPGs for growth once per day. This history is stored in a table and can be
displayed using the -hist option in the showspace and showscpg commands. An alert is
automatically generated if the sampled available free space for a CPG falls below the point at
which the CPG cannot grow to either the CPG warning limit or the CPG allocation limit.
- 14 -
12.5 Host System-Based Quotas
Host system-based quotas can be used to complement TPVV and CPG limits. Quota limits may
be handled more gracefully by the applications and operating systems.
12.6 User Recommendations
Monitoring of 3PAR Thin Provisioning alerts and available capacity by a storage administrator
and business processes internal to the customer are a critical component of Thin Provisioning
management/administration strategy. In priority order, the recommended level of process
responsibility for customers to monitor 3PAR Thin Provisioning is:
 Customer should nominate a primary and backup storage administrator for each
customer site. Nominated storage administrators’ role will be to:
o Proactively monitor free space availability per CPG.,
o Proactively monitor consumption rates for TPVVs and CPGs.
o Proactively monitor used TPVV capacity as compared to licensed thin

provisioning capacity.
o Ensure adequate purchasing and installation of additional physical disk buffer

stock and thin provisioning license upgrades in a timely manner.
 Customer should nominate an escalation contact who will have proper authority to drive
the customer responsibilities outlined in this document if the nominated storage
administrators fail to carry out their responsibilities (see first bullet point above).
 Customer is responsible for implementing a process to monitor and for acting upon all
TPVV and CPG alerts, especially those that warn when thin provisioning license and
physical capacity thresholds are reached. Customer should implement at least one out of
the following two processes:
o Process to monitor and act upon all TPVV and CPG alerts (see first bullet point
above) available through the 3PAR CLI and GUI.
o Process to monitor and act upon all TPVV and CPG alerts (see first bullet point
above) that generate an SNMPv2 trap into the customer's configured enterprise
management system.
In the event that the customer implements all of the recommendations indicated above and has a
network connection with 3PAR Central via the 3PAR Service Processor, then the customer can,
at the customer’s request, sign up to have Thin Provisioning alerts forwarded from 3PAR Central
via email to a specified email address. The customer retains responsibility for managing 3PAR
Thin Provisioning and CPGs. 3PAR is not be responsible for any failure by 3PAR to notify the
customer when thresholds are met or exceeded.
- 15 -
13 Meeting Space Limitations
The following sections describe the severity of the situation when space limitations are met, from
least critical (for example, running out of space on a single TPVV), to more severe (for example,
running out of space on a CPG or even an entire InServ Storage Server).
13.1 Running Out Of Space on a TPVV
If a limit is set for a TPVV, then the InServ will stop all writes coming in to that TPVV once the
limit is reached. Depending on the operating system and application, the effect can range from
the host server application stalling to the host server crashing. Reaching a TPVV limit purely
affects that volume and the resident application.
If a limit is reached and the host server continues to issue write requests, the 3PAR Storage
System will return SCSI hardware errors for those IOs.
TPVVs allocate additional space depending on the incoming write rate, so reaching the TPVV
limit will not stop the application writing to the volume the instant that the limit is reached.
TPVV limits can be changed at any time with the setvv –al command. Writes will then continue
to be processed.
13.2 Running Out of Space with a CPG
Reaching a CPG limit has the same effect as reaching a TPVV limit; however, many more TPVVs
and CPVVs may be impacted. 3PAR recommends a maximum of 32 TPVVs or CPVVs (or
combination) per CPG.
With CPGs, there may be a significant amount of space available into which the TPVVs and
CPGs can grow before host system writes are no longer accepted.
CPG limits can be changed at any time with the setcpg –sdgl command. Writes will then
continue to be processed.
13.3 Running Out of Space on the System
Running out of space on an InServ will affect all TPVVs and CPVVs being written to. Separate
usage warnings are automatically generated when 50%, 75%, 85%, and then 95% of space is
used within the system. In addition to this, it is possible to set a warning level that is triggered
once free space falls below a specified capacity (setsys RawSpaceAlert <value>, where
value is the remaining capacity in gigabytes). These alerts should be monitored and promptly
acted on to prevent all free space being used.
14 File System Limitations and Considerations
Thin volumes are block-based and do not “understand” file systems. This means that if a large file
is written then deleted, 3PAR Thin Provisioning has no knowledge of this and will leave the space
- 16 -
allocated. Fortunately, most file systems re-use the physical blocks that have just been freed
straight away, thus minimizing the impact of deleting large files. In addition, Thin Reclamation
technology is planned for an upcoming 3PAR Thin Provisioning release which will allow freed
capacity from deleted files to be automatically reclaimed. Thin Reclamation is intended to allow a
TPVV to stay thin over time, even in light of rapid or large writing/deleting of files.
14.1 Fragmentation and De-Fragmentation
Data written into thin volumes is allocated in 16-kilobyte increments as data is written into the
128 MB LD regions as described earlier. LD regions may be interleaved for multiple TPVV and
CPVV data within the same CPG. As a result, data may be fragmented to some extent on disk,
independent of how the file system has the data mapped. The InServ uses high-performance,
table-based mapping algorithms to track the data and spreads the load across all available disks.
This means that file system-based fragmentation is much less of a concern compared to
traditional storage systems where “head thrashing” on individual disks can dramatically affect
performance.
It is recommended that de-fragmentation only be run against file systems with severe problems;
for example, where the number of inodes or extents affects the file system operation at the host
level.
15 Reclaiming Snapshot Space
TPVV and CPVV snapshot space grows but never shrinks. Fortunately, when snapshots are
removed, the free space will be re-used for snapshot space but only for the particular volume
(TPVV or CPVV). CPVV snapshot space can be reclaimed using the freespace <VV_name>
command.
This command cannot be used for TPVVs and their associated snapshots. To reclaim TPVV
snapshot space, the base TPVV must be deleted. Currently it is not possible to distinguish
between TPVV user space and snapshot space. This is due to be addressed in the next version
of the InForm OS, scheduled for later this year. At that time, deleting a single snapshot will allow
the Thin Reclamation of space associated with it.
16 Reclaiming CPG Space
CPG space grows but never shrinks. You can manually remove LDs allocated with a CPG so
long as they do not have any data on them once TPVVs and/or CPVVs have been removed. Use
the showld -cpg <cpgname> command to achieve this. The LDs with an N in the MapV column
(or an O in the VV column for the LD view in the GUI) can be removed. Make sure that you do
not remove physical disk IDs (pdlDs) and logging LDs, as these have N under the MapV columm
but are actually being used!
The system will automatically recreate LDs if the CPG is below the growth threshold.
- 17 -
17 Migration to and from TPVVs
There are several approaches to consider when migrating data to TPVVs. With InForm OS 2.3.1
and the InServ T-Class arrays featuring Thin Built In™ and the 3PAR Gen3 ASIC, Fat-to-Thin
block-based migrations are possible with your preferred data migration tool. T-Class arrays are
available today and InForm OS 2.3.1 is scheduled for release in 2009. More information will be
made available once this software release is Generally Available.
Prior to InForm OS 2.3.1, the ideal tool used for migration should be file-based and not block-
based; thus, when writing to the TPVVs, only the actual files are written. Using a block-based
migration tool such as a host-based volume manager or a SAN-based virtualization appliance will
copy all blocks to the TPVV, whether they are used or not. This means that the customer may not
benefit at all from Thin Provisioning. Exceptions to this are listed below.
Using file based tools such as rsync in UNIX, Quest in Windows, or backup-and-restore will
allow the TPVVs to grow only to the capacity of data contained within the file system, therefore
making efficient use of physical disk capacity.
17.1 Migrating to TPVVs Using Block-Based Migration
Data can be migrated to TPVVs using block-based migration. An example is a database that
currently resides on raw partitions but does not use all of that partition. If the partition is on a
VERITAS Volume Manager volume, then the volume can be shrunk to the database size and
moved to the TPVV using VERITAS Volume Manager. The VERITAS volume could then be
extended, if required.
Alternatively, a dd from the existing partition with a block size and count that just transfers the
database to the TPVV should also work.
17.2 Using 3PAR Full Copy with 3PAR Thin Provisioning
3PAR Full Copy recognizes when a source VV is a TPVV. If so, it will only copy written data to
the target VV whether this is a TPVV or a standard base VV.
If the source VV is a standard base VV (CPVV), then Full Copy will copy all blocks to the target
VV whether this is a TPVV or standard base VV. If the target VV is larger than the source, only
the size of the base VV will be copied, which means that some TPVV efficiency could be realized
with 3PAR Full Copy from a standard base VV.
17.3 Using 3PAR Remote Copy with 3PAR Thin Provisioning
As with 3PAR Full Copy, 3PAR Remote Copy recognizes when a source VV is a TPVV and will
only replicate written data to remote VVs whether this is a TPVV or standard base VV.
If the source VV is a standard base VV (CPVV), then Remote Copy will copy all blocks to the
target VV whether this is a TPVV or standard base VV.
- 18 -
18 CPG Performance Considerations
CPG performance consideration can be split into three components:
 CPG layout
 CPG growth size
 Amount of CPG interleaving
19 Tracking Volume Space Usage
The following sections provide commands to find out how much space is used within the system
based on volume and/or group type.
19.1 Base and CPVVs
For base volumes and CPVVs, the raw space used is shown in the RUserMB column when using
the showvv –r command. Command output shows base VVs with pre-allocated snapshot space
(the VV shows as “Base” and not as “CPVV”).
19.2 TPVVs
Use the showvv –r –tpvv command to see how much space is used by TPVVs. The UserMB
column shows how much space is presented to host systems and the RAdmMB and RSnapMB
columns show the physical space used by TPVVs.
19.3 CPGs
The raw space for snapshots and snap admin space may be useful. However, CPGs will always
use more space. The showcpg –r command shows the raw space used by CPGs. You can also
work out the unallocated space within the CPGs by subtracting the used space from the CPG
space. Adding the RTotMB for both the Sa and Sd space will give the amount of physical disk
used.
The total of RAdmMB and RSnapMB in the output for the command showvv –r –cpg
<CPG_NAME> should be the same as the total used space in CPGs (add Sa and Sd RUseMB total
for the CPGs) minus showcpg –r.
19.4 System Space
There are Preserved Data LDs (PDLDs) and Logging LDs. PDLDs should be the same size as
cache (showsys); logging LDs should be 40 GB per controller node. The command showld -d
will show the raw space used for log and pdsld logical disks.
- 19 -
Furthermore, there is a RAID 1 admin Virtual Volume used for storing system logs. This volume
has 10 GB of usable size (20 GB raw) and can be seen with the showvv -r command.
19.5 Total Used Space
To obtain the total raw space used within the system, add the raw space used by base volumes,
CPGs, PDLDs, and logging LDs. This is the total raw space used.
Next, you can correlate this information with the chunklet usage. Use the showpd –c command.
The total chunklets are listed under the Total column. Take away all failed and spare chunklets,
multiply by 256 MB, and you have the space available for logical disks. If you take this number
and subtract the raw usage as calculated above you should have the correct free space. This
should correlate to the total number of Free and Un-initialized Chunklets listed under
Normal Chunklets multiplied by 256 MB.
Finally, the showspace command can be used for further verification by checking the RawFree
column.
This information can also be determined from the Space Reports in the 3PAR System
Reporter utility.
20 Usage Reporting and Trend Analysis
There is a showcpg -hist command that gives a daily account of CPG usage. The
showspace -cpg <cpgname> -hist also shows this information.
Additionally, 3PAR System Reporter software also has the ability to track TPVV and CPVV usage
for comprehensive usage and trend analysis.
21 Licensing
3PAR Thin Provisioning is comprised of software that runs on the 3PAR InServ® Storage Server
and is licensed per terabyte of physical space written, including space used by snapshots and
RAID overhead. If the license space is exceeded, this will not affect the use of existing Thin
Provisioned Virtual Volumes (TPVVs).
To determine how much capacity has been used used towards an existing 3PAR Thin
Provisioning license on an InServ, use the showvv –r –tpvv command. The total Thin
Provisioning capacity used is the sum of the columns RAdmMB and RSnapMB. The admin space is
normally a very small fraction of the snapshot data space used.
The only way to reduce the amount of Thin Provisioning license space used is to remove TPVVs.
As soon as TPVVs are deleted, the 3PAR® InForm Operating System will recalculate the amount
of license space used. TPVV data can be migrated to standard provisioned volumes using 3PAR
Full Copy; however, a brief disruption to service is required to change the host mapping.
- 20 -
22 Conclusions
3PAR Thin Provisioning is a simple yet powerful tool for improving the efficiency of storage.
Following the best practices outlined in this paper will allow IT staff to maximize the benefit of
using 3PAR Thin Provisioning to do more with less.
- 21 -
About 3PAR
3PAR® (NYSE: PAR) is the leading global provider of utility storage, a category of highly
virtualized, tightly clustered, and dynamically tiered storage arrays built for utility computing.
Organizations use utility computing to build cost-effective virtualized IT infrastructures for flexible
workload consolidation. 3PAR Utility Storage gives customers an alternative to traditional arrays
by delivering resilient infrastructure with increased agility at a lower total cost to meet their rapidly
changing business needs. As a pioneer of thin provisioning—a green technology developed to
address storage underutilization and inefficiencies—3PAR offers products designed to minimize
power consumption and promote environmental responsibility. With 3PAR, customers have
reduced the costs of allocated storage capacity, administration, and SAN infrastructure while
increasing adaptability and resiliency. 3PAR Utility Storage is built to meet the demands of open
systems consolidation, integrated data lifecycle management, and performance-intensive
applications. For more information, visit the 3PAR Website at: www.3PAR.com.

3PAR Thin Provisioning Best Practices

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3PAR Thin Provisioning Best Practices

Uploaded by

Copyright:

Available Formats

3PAR Thin Provisioning Best Practices

Part Number: 3PAR-tpbest-wp-09.0

2 Overview of 3PAR Thin Provisioning....................................................................................4

3 When to Use Thin Provisioning.............................................................................................5

6 When Not to Use Thin Provisioning......................................................................................6

7 TPVV Sizing Guidelines ........................................................................................................7

8 Common Provisioning Groups..............................................................................................7

9 When to Use Separate CPGs for Different VVs...................................................................8

11 CPG Space Usage ..............................................................................................................10

12 Managing CPGs and TPVVs: Warnings and Limits...........................................................11

13 Meeting Space Limitations ..................................................................................................15

14 File System Limitations and Considerations.......................................................................15

15 Reclaiming Snapshot Space...............................................................................................16

16 Reclaiming CPG Space ......................................................................................................16

17 Migration to and from TPVVs..............................................................................................17

18 CPG Performance Considerations .....................................................................................18

19 Tracking Volume Space Usage ..........................................................................................18

20 Usage Reporting and Trend Analysis.................................................................................19

2 Overview of 3PAR Thin Provisioning

Figure 1: Overview of thin-provisioned virtual volumes

3 When to Use Thin Provisioning

3.1 Avoiding Storage Capacity Addition to Servers

3.2 Accelerating Time to Market

6 When Not to Use Thin Provisioning

7 TPVV Sizing Guidelines

8 Common Provisioning Groups

9 When to Use Separate CPGs for Different VVs

9.2 Use Different CPGs for Logical Separation or Chargeback

10.1 CPG Automatic Growth

10.2 Determining CPG Growth Increment

o The CPG growth increment should be as small as possible to minimize the

o If the environment is performance-intensive (or, more specifically, write-

10.3 CPVVs Space Within CPGs

10.4 Changing LD Characteristics Within a CPG

11 CPG Space Usage

11.1 How TPVVs Use CPG Space (LD Regions)

11.2 How CPG Space Is Allocated to TPVVs and CPVVs

o Set the CPG allocation warnings (and limits if necessary)

o Monitor available free space alerts

o Maintain a buffer stock of physical capacity

12 Managing CPGs and TPVVs: Warnings and Limits

 Allocation limits and warnings for TPVVs and CPGs

 Used physical capacity alerts

 Available free space alerts

 Host system-based quotas

Figure 2: Thin Provisioning Usage Management

12.1 Allocation Limits

12.2 Allocation Warnings

 setvv aw <percent>, which sets the TPVV or CPVV allocation warning as a

 setsys RawSpaceAlert <value>, where value is the remaining capacity in

These alerts and limits can be changed at any time.

12.3 Used Physical Capacity

setsys RawSpaceAlert <value>, where value is the remaining capacity in

12.4 Available Free Space Alerts

12.5 Host System-Based Quotas

12.6 User Recommendations

o Proactively monitor free space availability per CPG.,

o Proactively monitor consumption rates for TPVVs and CPGs.

o Proactively monitor used TPVV capacity as compared to licensed thin

o Ensure adequate purchasing and installation of additional physical disk buffer

13 Meeting Space Limitations

13.1 Running Out Of Space on a TPVV

13.2 Running Out of Space with a CPG