AIX Tip of the Week: Minimizing Downtime to MoveLarge Data Sets Between Storage Devices
This tip offers several AIX techniques to minimize the downtime required to move largedata sets between storage devices. This tip originated from a customer situation where ittook an extended outage (over 24 hours) to move a TB of data from one storage device toanother using a single cp ‘R command.
Here are a few of my suggestions to improve their copy time in the future. In order of preference:
•
Use SAN/Application tools when available
•
Multiple copies in parallel
•
Migrate
•
Split mirror copy
•
Point in time copy, followed by an incremental copy
Discussion
SAN/Application Tools
: My first choice for copying bulk data is to use SAN or application specific tools when available. In general, these tools do a better job of preserving data integrity, have better performance and can reduce the load on the server.However, this is an AIX tip, so here are some AIX specific techniques.
Multiple Copies in Parallel
: This involves running multiple copies in parallel. For example, running two copies in parallel might reduce the copy time by half. Runningthree copies in parallel might reduce the time to one third, and so on. For example,instead of running a single copy # cp ‘pR /source/* /destinationrun multiple copies# cp ‘pR /source/dir_a/* /destination/dir_a# cp ‘pR /source/dir_b/* /destination/dir_bThere are a couple limitations with this technique. First, it does not work well if the datais unevenly distributed through a file system. Second, it also requires the application to be stopped (or run in read-only mode) for the duration of the copy. Finally, multiple
copies will degrade performance if the data resides on the same physical disk.
Migrate
: the AIX migratepv or replacepv commands can move physical
partitions from one device to another in the background while running production. This
has the additional benefit of not moving the location of the data, as the copy commanddoes. For example, the following command will move the contents of hdisk1 to hdisk5and hdisk5.
migratepv hdisk1 hdisk5 hdisk6
Alternatively, migratepv can move individual logical volumes. The following
command moves datalv to hdisk4.
migratepv ‘l datalv hdisk4
The technique requires source and destination hdisks to reside in the same volume group.
Split mirror copy
: This involves mirroring the source LV on the destination LV.Once the mirror synchronization is completed, the application is temporarily suspendedwhile the source LV copy is removed. The mirror synchronization can run in the background during production.The following example creates a mirror copy of datalv on hdisk1 to hdisk2,
synchronizes the mirror, then removes the original copy.# mklvcopy datalv 2 hdisk2# syncvg ‘l ‘P6 datalv# lslv datalv # verify sync is completeno stale partitions
Although not required, I recommend stopping the application to be safe.
# rmlvcopy datalv 1 hdisk1The technique requires source and destination hdisks to reside in the same volume group.Also this technique will take longer to synchronize than a straight copy, as all PPs are
copied (including empty PPs).. For more information, see
Point in time copy
: This technique copies files in the background during production.Then later, the application is shutdown and only the changed files need to be copied. For example# touch timestamp (create a file for a timestamp)
# find . ‘print | cpio ‘pld /source /destination..shutdown application to copy changed files..
# find . ‘newer timestamp ‘print | cpio ‘pld /source /destinationA better command is GNUs rsync command, which can be found on the Linux
Toolbox CD. The syntax is:
of 00
Copy Lg Data1
231 Reads
Info and Rating
| Category: | Uncategorized. |
| Rating: | |
| Upload Date: | 11/18/2009 |
| Copyright: | Attribution Non-commercial |
| Tags: |
Leave a Comment