You are on page 1of 10

Technical Report

Reallocate Best Practices
Chris Wilson, NetApp June 2012 | TR-3929

Summary
The reallocate, read reallocate, and free space reallocate tools offer functionality to optimize ® ® the data layout of NetApp storage systems running Data ONTAP . This paper introduces the basic functionality and best practices associated with using the reallocate toolset.

....................................... 6 Read Reallocation and Reallocate .......................................4 Overall System Performance ......................... 8 Volume Snapshot and SnapMirror ......................................................................................................................................................................... 5 Developing a Reallocate Schedule .....................................................1 2.........................................................................................................................1 4....................................................................................................................................4 Read Reallocate.......................................... 5 Forced Reallocation ..............................................................................................................................6 2.................................................................................................TABLE OF CONTENTS 1 2 Introduction.2 2.......................3 3..................... 5 Aggregate Reallocation ................................................................................................................................................................................................4 2..... 3 Reallocate ....................................................................... 9 5 More Information ..............................3 2.............7 Reallocation Methods.................................................................... 3 2.....................................................5 2............................ 4 Execution Requirements ........1 3......................................................................................................... 7 Free Space Reallocate............................................................... ................... 8 FlexClone Volumes ............................... 8 4..... 3 When to Use Reallocate ..............................................................................................................................................................................................................................................................................................3 4............................................... 9 LIST OF TABLES Table 1) reallocate start execution options.................................................2 3................................................................... 6 3 Continuous Reallocation Features ......................................................................................................... 9 Deduplication and Compression ............................................................................................... 7 4 System and Feature Interactions .........................................................................................................................................................................2 4........... 4 Reallocate Options ............................................................ 6 3.................. 7 Free Space Reallocate and Read Reallocate ..................................................................................................................................................... 9 Version History ......................................................................................................................... 4 2 Reallocate Best Practices .....

stopped. Reallocate internally uses the following process: 1. You can also use the same command to check the status of the reallocation and measure the current layout optimization. read_realloc. The benefit of using physical reallocation is that no additional space is required for Snapshot copies. no data ™ is moved. free_space_realloc. additional space is required to maintain the copies. If the optimization score is less than the threshold. 3. Physical reallocation follows the same process as traditional reallocation. and in a special case. however.” Physical Reallocation The reallocate tool also provides a physical reallocation option. and what the various options perform.” Best Practice When possible. Reallocate rechecks the current layout and repeats the process until the layout is optimal. compared to using traditional reallocation. 2. The implications of using physical reallocation are discussed in section 4. Reallocate can be started. see section 2. aggregates.1 Introduction Data ONTAP provides a set of tools to help customers optimize the layout of data on disk for sequential read access.” 2. If the optimization score is greater than the threshold. if Snapshot copies are used. LUNs. 3 Reallocate Best Practices . The impacts of using reallocate are discussed in section 4. “System and Feature Interactions. instead of completely rewriting data to the disks. If no improvement is predicted. “System and Feature Interactions. individual files. quiesced. “When to Use Reallocate. reallocation is not needed.2. use physical reallocation to reduce the space requirements of Snapshot copies. optimized locations. Reallocate performs data reallocation by using traditional or physical reallocation. 2 Reallocate Reallocate is used to optimize the layout of data on disk. It works on volumes. the volume option. This guide introduces the reallocate tools in Data ONTAP. including reallocate. the data blocks are moved by changing the physical block location while maintaining the logical block location ® within the FlexVol volume. and scheduled from the command line interface (CLI) by using the reallocate command. For information about thresholds and measurements. This section describes how reallocate works. Reallocate measures the current layout and produces an optimization score. The System Administration Guide for the appropriate version of Data ONTAP might answer any additional questions not covered in this guide. when to consider using reallocate. and the aggregate option. This guide also describes some things to consider when using the reallocate tools. Because data is rewritten to disk. NetApp Snapshot data is not moved even when active file system data has been moved to new.1 Reallocation Methods Traditional Reallocation The reallocation process progresses through the file system and moves data blocks by rewriting them when Data ONTAP determines that the layout can be improved. proceed to step 2.

-f -o -n 4 Reallocate Best Practices . Table 1 describes the available options to the reallocate command. The reallocate measure command provides insight into the current layout by providing an optimization value of the layout as it is when reallocate measure is run. Generally not recommended. Typical applications that see the most benefit from using reallocate include: • • • Online transaction processing databases that perform large table scans E-mail systems that use database storage with verification processes Host-side backup of LUNs If the workload isn’t well known. Option -p Description Executes reallocate by using physical reallocation. the measurement output might produce a hotspot value. Reallocate includes measurement functionality to help users quantify if a reallocation is needed. The default threshold that reallocate uses is 4. Occasionally. a volume can have an overall measurement value of 3. The workload that most benefits from the use of reallocate is sequential reads after random writes. You can use this optimization value as a threshold setting for reallocate. “Execution Requirements. but contain a hotspot of 28. toaster> reallocate start –n /vol/volX 2. Generally recommended. see section 2. it might be difficult to determine whether a reallocation is necessary. For more information. other workloads might also show some improvement depending on the workload characteristics. however. you can change this default by using the -t flag when executing a reallocate start or reallocate measure command.2.4. The output values can be used to determine whether on-disk layout is a potential cause of decreased performance. toaster> reallocate start –t 5 /vol/volX or toaster> reallocate measure –t 5 /vol/volX It is also possible to skip the scanning portion of reallocate by using the –n option. Valid threshold values are from 3 through 10 (not optimal). Table 1) reallocate start execution options. This option ignores any thresholds and simply begins data reallocation. Actual measurement values produced by reallocate can exceed 10.3 Reallocate Options You can execute and schedule reallocate by using the Data ONTAP CLI. “Forced Reallocation. For instance.” Executes a forced reallocation.2 When to Use Reallocate Reallocate optimizes sequential read performance. see section 2. For more information.5. The following command shows how to measure the layout of a volume: toaster> reallocate measure –o /vol/volX Note: Use the -o flag to measure layout only and not schedule a measurement.” Executes reallocation one time only. The hotspot value indicates that a portion of the scanned object has a layout that is less optimal than the remainder of the object. Executes reallocation without measuring the layout first.

and consistently better performance can be achieved. Full reallocation requires the physical reallocate option (-p) if Snapshot copies are present. Existing Snapshot copies must leave at least 10% of the Snapshot reserve or 5% of the volume size free.Option -t -A Description Forces reallocate to use a custom threshold. executing reallocate start without any parameters causes reallocate to run once daily. the time required to reallocate. Best Practice Use forced reallocation after adding disks to an aggregate so that existing data is spread onto the new disks. you must meet various requirements. Physical reallocate (and aggregate reallocate) cannot be run on data stored in mirrored aggregates.0 or 7. ® • • • • • • 2. Forced reallocation ignores the optimization thresholds and completely rewrites the data to disk. Also. To change this default.4 Execution Requirements To run reallocate.5 Forced Reallocation A forced reallocation scan optimizes all the data in the volume. Developing a schedule for reallocate depends on the reallocation method. reducing the time required to balance the load across all the disks. Executes aggregate free space reallocation. Physical reallocate cannot be run on aggregates created with Data ONTAP 7. Although this improves the layout.6 Developing a Reallocate Schedule Reallocate is most effective when it is executed regularly. 2. Physical reallocate cannot be run on RAID 0 aggregates. An asterisk (*) is a valid 5 Reallocate Best Practices . 2. and the workload applied to the system. routine use of reallocate –f is not a best practice. The reallocate schedule command can be used only on reallocation scans that have already been created. the amount of work that reallocate must complete during each iteration can be reduced. You can use df to check free space. The following list can help determine why reallocate might not run: • • • The object to be reallocated must be writable (no SnapMirror destinations). ® Physical reallocate cannot be executed on FlexClone volumes. use the reallocate schedule command after a reallocation scan has been created or use the –i flag when creating the reallocate scan. forced reallocation cannot be run against volumes that have existing Snapshot copies unless the physical reallocation method (-p) is also used. With regular reallocation. or file regardless of current optimization level. The volume or volume containing an object undergoing reallocation must be at least 5% free. LUN. Physical reallocate can be executed only on data in FlexVol volumes. By default.1. because all of the data is optimized. unlike the normal reallocation process. This requirement does not apply to physical reallocation. A schedule in the form of “minute hour day_of_month day_of_week” is required as a parameter for the command. whichever is less.

7 Aggregate Reallocation To optimize an aggregate. If the read was less than optimal. This reallocation method reallocates blocks within an aggregate to improve contiguous free space. use the –d flag: toaster> reallocate schedule –d /vol/volX To stop and delete a job. Read reallocation uses the normal workload reads along with the read-ahead engine to determine the current layout optimization.1 Read Reallocate Read reallocate is a volume option that performs opportunistic reallocation on data to improve performance. 3 Continuous Reallocation Features The following sections introduce two features in Data ONTAP that perform continuous. Enabling Read Reallocation Read reallocate is a volume option that is enabled by using one of the following CLI commands: 7-Mode: toaster> vol options volX read_realloc [on / space_optimized] Cluster-Mode: Cluster::> volume modify –vserver vs1 –volume volX –read_realloc [on / space-optimized] Simply enabling read reallocation by using the “on” function tells Data ONTAP to use the traditional reallocation method. If you have questions about using aggregate reallocation contact your NetApp representative. Also. use the reallocate stop command: toaster> reallocate stop /vol/volX 2. use the –i flag: toaster> reallocate start –i 7d /vol/volX To delete a reallocate job. it does not require additional scanning or scheduling. the data will be reallocated to improve the next read of this data. on-the-fly data layout optimization: read reallocate and free space reallocate. 3. This has the same side effects as traditional reallocate. The space_optimized option is synonymous with the physical reallocation method. These features are different from the reallocate command in that they operate continuously without the need to schedule scans.” The following example creates a weekly reallocate schedule that runs every Saturday at midnight for volX: toaster> reallocate schedule “0 0 * 6” /vol/volX To create a scan with a weekly interval at creation time. It does not introduce the Snapshot space 6 Reallocate Best Practices . because read reallocate uses the existing read workload. It should not be used to improve sequential read performance. use the –A option with the reallocate command.option that represents “any. The –A option does not reallocate all of the data in the aggregate following the normal reallocation method. Read reallocate offers both the traditional and physical reallocation methods associated with the reallocate command. Aggregate reallocation uses the physical reallocation method to move blocks to create contiguous free space. therefore the impacts of using physical reallocate still apply.

depending on the workload applied to the system. 7 Reallocate Best Practices .2 Read Reallocation and Reallocate Reallocate and read reallocation are complementary. free space reallocation will not move blocks that are kept in aggregate Snapshot copies. An example of when to employ both is in a database system with weekly large table scans. Enabling Free Space Reallocation Free space reallocation can be enabled on a per-aggregate basis using the following commands: 7-Mode: toaster> aggr options aggr1 free_space_realloc on Cluster-Mode: Cluster::> storage aggregate modify –aggregate aggr1 –free-spacerealloc on For best results. similar to physical reallocation. Enabling read reallocate maintains an optimal layout of frequently accessed data. aggregate reallocation. and space-optimized read reallocate. If enabling the option on an existing aggregate with data already stored on it. If the performance of the storage system is limited by CPU. The two are complementary technologies that help maintain optimal layout.requirements associated with the traditional reallocation method. while free space reallocate will optimize for writes. Read reallocate will optimize the system for sequential reads on the fly. 3. Both features can be employed together to accomplish the same goal of improving spatial layout. however. there might be a period of time when Data ONTAP will be performing additional work to optimize free space and might affect system performance temporarily. which performs opportunistic free space reallocation to maintain an optimal free space layout. introduced in Data ONTAP 8. if Data ONTAP detects that free space is not optimal. enabling free space reallocation is not recommended.4 Free Space Reallocate and Read Reallocate When enabling free space reallocation on an aggregate. while a scheduled reallocate execution prior to the large table scan might help improve the table scan speed. the option should be enabled when creating a new aggregate. Some additional CPU utilization should be expected since Data ONTAP will be doing additional work to manage the movement of data. it will physically move data to produce areas of contiguous free space. The space_optimized method does.3 Free Space Reallocate Free space reallocation is an aggregate option. free space reallocate will still move blocks stored in volume Snapshot copies.1.1. 3. This does not apply to volume Snapshot copies. Optimized free space improves the efficiency of WAFL® (Write Anywhere File Layout) and can reduce overall disk utilization. Read reallocate can help reallocate by reducing the amount of work that reallocate needs to do in each iteration. 3. introduce the Snapshot read performance impact. Similar to physical reallocate. When enabled. also consider enabling read reallocate space_optimized for the volumes in the aggregate.

When using the traditional reallocation method against a SnapMirror source. reallocate has a minimal impact on system performance. Physical Reallocate The physical reallocation method does not have the same impact on Snapshot copies as the traditional reallocation method. schedule reallocate runs during off-peak hours. This low-level movement does not cause the copies to occupy any additional capacity. Unlike the traditional reallocation method. Data ONTAP Snapshot technology does not notice any changes. If a system is heavily loaded. Because reallocate rewrites data to improve layout. regardless of whether the contents have actually changed.4 System and Feature Interactions This section describes how the reallocate tools interact with other features in Data ONTAP. After a block is read from a Snapshot copy. Physical reallocate does not cause the active file system to diverge from existing Snapshot copies. Therefore. a nonoptimal file system layout requires many data blocks to be rewritten. 4. Using physical reallocate can introduce a read performance impact to existing Snapshot copies. a heavily loaded system also causes reallocate to take longer to complete.1 Overall System Performance By design. However.2 Volume Snapshot and SnapMirror Both the traditional and physical reallocation methods affect existing Snapshot copies in some way. Best Practice When possible. reallocate is treated as a low-priority operation and should not significantly slow users’ access to data. subsequent accesses to that block do not require additional lookups. but the original data block remains in any existing Snapshot copies. WAFL improves the layout by moving the data blocks. if a large amount of data was reallocated. SnapMirror does not identify any data that was reallocated as data that must be sent during the next sync. but rather moves data blocks at a lower level without altering the logical location. Administrators must plan for this additional capacity utilization when running reallocate. 4. 8 Reallocate Best Practices . Snapshot copies do not require any additional space unless a block in the active file system is altered or removed. the traditional reallocation method rewrites the data in the active file system to improve layout on disk. This performance impact is due to additional operations that must be completed to locate Snapshot data after a reallocation completes. SnapMirror flags any reallocated data as data that needs to be sent during the next update. Traditional Reallocate As previously described. The additional utilization lasts only as long as the life of the Snapshot copies. SnapMirror hinges off of Snapshot technology to create point-in-time mirroring. Because physical reallocation does not follow the complete rewrite process. SnapMirror will require a large update on the next sync. Physical reallocation does not affect the performance of copies created after reallocate completes. The following two subsections describe how Snapshot copies and SnapMirror are affected by both methods. thus increasing the required Snapshot space.

1 release. The parent remains in the same. 9 Reallocate Best Practices . Running reallocate against a parent causes any data that is reallocated to be maintained in the Snapshot copy for the clones. or free space reallocate. using additional Snapshot reserve space. Compressed data will not be reallocated by reallocate or read reallocate. Version History Version Version 1. Updated for 8. review the System Administration Guide on the NetApp Support site for your version of Data ONTAP. the clones might exhibit some read performance degradation during the first access of data due to additional location lookups. If physical reallocate is used against a parent of FlexClone volumes. the parent is reallocated as expected.0 release. so executing reallocate has similar implications.4. Updated for 8.1 Version 1. unoptimized state.0 Version 1. 4. including deduplication changes. however. including free_space_reallocate. If the clone is reallocated. Although data may be shared by multiple files when deduplicated.1 deduplicated data can be reallocated using physical reallocation or read_realloc space_optimized. and it is not recommended to run reallocate on compressed volumes. read reallocate.4 Deduplication and Compression Starting in Data ONTAP 8.1.2 Date June 2011 October 2011 June 2012 Document Version History Initial publication.1.3 FlexClone Volumes FlexClone volumes also hinge off of Snapshot technology. Prior versions of Data ONTAP do not support reallocation of deduplicated data and will skip any deduplicated data encountered. 5 More Information To get more information about using reallocate. Physical reallocate and space_optimized read reallocation against a clone are not allowed. it diverges from the parent and uses additional space in the aggregate. reallocate uses an intelligent algorithm to only reallocate the data the first time a shared block is encountered.

Inc. NetApp provides no representations or warranties regarding the accuracy. Specific results depend on each customer's installation in accordance with published specifications. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document. All rights reserved. Inc. faster. FlexVol. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as Reallocate Best Practices such. or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. FlexClone. SnapMirror. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Go further. reliability. 10 © 2012 NetApp. or serviceability of any information or recommendations provided in this publication. the NetApp logo. and the use of this information or the implementation of any recommendations or techniques herein is a customer’s responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. Inc. No portions of this document may be reproduced without prior written consent of NetApp. NetApp. The information in this document is distributed AS IS.Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. TR-3929-0512 . and WAFL are trademarks or registered trademarks of NetApp. Specifications are subject to change without notice. Snapshot. in the United States and/or other countries. Data ONTAP.