AIX - Logical Volume Manager

---------------------------------------------------------------------Table of Contents What is LVM Really? High Level Commands Intermediate Commands Library Calls - API LVM Device Driver Disk Device Driver SCSI Device Driver Terms Physical Volumes PVIDs VGDA VGSA Volume Groups VGDA expansion Case A, VGDA expansion Case B, VGDA expansion Case C, VGDA Expansion Logical Volumes Mirrors What Does LVM Give You? Flexibility Data Integrity Write Verify Speed exportvg importvg reducevg varyonvg Problem Determination Problem Recovery Failure/Recovery Examples Example1: Resynchronize Good PV Gone Offline Example2: Replace Failing Physical Volume Example3: Bad Magic Number in Superblock Example4: Fix LVM/ODM Inconsistency on non-rootvg VG Example5: Fix LVM/ODM Inconsistency on rootvg Example6: Phantom ODM entries Summary APPENDIX: LVM Data Recovery Example ---------------------------------------------------------------------What is LVM Really? High Level Commands

 Intermediate Commands Library Commands LVM Device Driver Disk Device Driver SCSI Device Driver Actual Disk High Level Commands High Level Commands Commands used by SMIT Commands used by users in command line or shell scripts High levels of error checking Notes: High Level Commands Most of the LVM commands reside in /usr/sbin. There are exceptions such as the library commands which reside in /usr/lib/liblvm.a and /usr/lib/libsm.a and /etc/cfgvg. The High Level commands are a combination of shell scripts and C binaries: chlv , chpv, chvg, cplv, exporting, extendlv**, extendvg, importvg, lslv*, lsvg*, lsvgfs*, migratepv, mklv, mkvg, reducevg, reorgvg, rmlv, rmlvcopy, syncvg, synclvodm, varyonvg, varyonvg* indicates that this command is a binary ** this command also is used for the mklvcopy command These commands are those that SMIT and experienced users are meant to execute directly. Intermediate Commands Commands used by High Level commands Commands provide usage statements - unfortunately Some level of error checking still provided, no documentation on output Notes: Intermediate Commands The Intermediate commands are all C binaries: getlvcb, getlvname, getlvodm, getvgname, lvgenmajor, lvgenminor, lvrelmajor, lvrelminor, putlvcb, putlvodm, lchangelv, lcreatelv, ldeletelv, lextendlv, lquerylv, lreducelv, lresynclv, lchangepv, ldeletepv, linstallpv, lquerypv, lresyncpv, lcreatevg, lqueryvg, lqueryvgs, lvaryonvg, lvaryoffvg, lresynclp, lmigratepp. The commands are called by the High Level command shell scripts. These commands unfortunately give usage statements when queried from the command line, which has given users the incorrect impression that these commands are for general use in LVM. This is not true. These commands have been "tuned" for use by the High Level shell scripts. Because it is known that only high level shell scripts will be calling these intermediate commands, certain error checking is not performed by the intermediate commands. The High Level commands are responsible for screening potentially dangerous errors or conditions out. These binaries also contain some functionality that are not used and even some bugs that have not been fixed because the function isn't used! Library Calls - API Library calls to be used as Application Program Interface Error checking provided Calls documented in Infoexplorer or books Very rarely used

Notes: Library Calls - API The LVM library calls are part of the LVM Applications Programmer Interface (API) which lets programmers access the LVM layer directly through the library layer provided by /usr/lib/liblvm.a. In our experience, very few people use this API because of the complexity of issues within LVM. Users mostly rely on shell scripts that call the High Level commands. The API library functions begin with lvm_ and are documented within Infoexplorer. These commands are only called in situations where the user is writing C code and does not want to access the High Level shell scripts from within a C program. Or, they require a faster response from LVM than they can achieve with shell scripts. Writing an LVM specific program using the API certainly does give a performance improvement to some applications. Note, the LVM library API is not a thread-safe library. Thus, user programs written for threads that reference the LVM API may not work correctly. LVM Device Driver Access same as those for typical IBM device driver Checking provided, but assumes user has basic device driver knowledge Rarely used. If used wrong, disastrous results! Notes: LVM Device Driver The LVM device driver comes in two portions, the pinned portion and the non-pinned portion. The pinned portion is called /usr/lib/drivers/hd_pin and /usr/lib/drivers/hd_pin_bot. Before AIX 4.1, the driver was just called hd_pin and the entire driver was pinned into memory (not pageable). With AIX 4.1, the driver's true non-pageable portion is in hd_pin_bot and the hd_pin is now pageable. The LVM device driver is either called by the jfs filesystem or the lvm library routines. When a request is received by the LVM device driver, it calls the disk device driver. Disk Device Driver Access same as those for typical IBM device driver Sometimes other vendors use the disk device driver as pseudo driver for their products Notes: Disk Device Driver There are various disk device drivers in AIX. The most common disk device driver is the one for scsi device drivers, /usr/lib/drivers/scdisk and /usr/lib/drivers/scdiskpin. The second most common disk device driver is probably the one for the 9333 serial dasd device. This is found in the binaries /usr/lib/drivers/sd and /usr/lib/drivers/sdpin. In both device drivers, the portion called "pin" is that which is pinned into memory and cannot be paged out of memory. When a request from the LVM device driver is received by the disk device driver, the request is packaged and then transferred down to the scsi device driver. SCSI Device Driver Access same as those for typical IBM device driver Very useful to access devices directly through this interface, although people prefer not to use this access point Notes: SCSI Device Driver There are several scsi device drivers for AIX and they have the common name string of "scsi" residing as part of their name. The most common of them is the original SCSI-1 device driver /usr/lib/drivers/hscsidd. The scsi device driver takes a command from a scsi device such as tape, disk, scanner, etc...and processes it to be sent to the scsi device connected onto the scsi bus. The scsi device is device neutral, it does not know or care which device sent it a command or even what that command is. It treats all requests the same and puts them into the same type of scsi command packaging

required by all scsi devices. [Image] Terms Physical Volumes Volume Groups Logical Volumes Mirror Notes: Terms [Image] On one system, many disks can make up a volume group. Disks cannot be shared between volume groups. The entire disk is dedicated to being part of one volume group. [Image] Logical volumes reside only on volume groups. A logical volume can span multiple disks within a volume group (but cannot span multiple volume groups): lv00. Or, a logical volume can just reside on one disk: biglv. Finally, a logical volume can have multiple copies of itself: mirrlv. Physical Volumes * Disk Independence o Physical Type of disk is transparent to the LVM. * Portability of volume groups o Volume groups can be easily imported into another system, the disks under AIX are portable * Limit of 1016 physical partitions per disk maximum o tracked by VGSA * Verify the true ID of a disk o each disk has a PVID (Physical Volume ID) - or soft serial number o can be accidentally destroyed with a dd command, or intentionally with disk formatting Notes: Physical Volumes The PVID of the disk is stored in ODM, use "odmget -q "name=hdisk<#>" CuAt" to get it. Also can check the PVID on teh disk with "lquerypv -h /dev/hdisk<#>", this command will print out in HEX format the first 128 byest from the beginning of the disk platter. The PVID is held in the first two columns of hex row 0x80. PVIDs PVID's and how they configure o New disk doesn't have a PVID, until used in a Volume Group o PVID is a combination of the machine's serial number and the date when the PVID is generated. It is not recommended to use the "dd" command to copy the entire contents of one disk to another o this will also copy the PVID, and each disk should have a unique PVID or the system could become confused. VGDA Volume Group Descriptor Area - VGDA o at the front of each disk o contains info about the volume group, the logical volumes that reside on the volume group, and the disks that make up the volume group.

o VGDA also used in quorum voting. Notes: VGDA Maximum of 32 disks per volume group. The VGDA can be thought of like a suitcase of information, it can be packed up and unloaded at another AIX system, and all the information about that volume group is available to the new system. VGSA Volume Group Status Area - VGSA o comprised of 127 bytes, where each bit represents one of the1016 Physical Partitions (PP) that reside on each disk. o Used in Mirroring as a bit-mask to determine which PPs have become stale. Volume Groups LVM Requires Quorum of disks in the volume group o Sanity check of VGDA to prevent data corruption. o 51% or more votes must be available to a volume group before LVM can continue. Volume Group Identifier - VGID o VGID is a soft serial number for the volume group, similar to PVID for the physical volume. Notes: Volume Groups For a single disk, there are 2 VGDAs on disk, with a quorum of 2. For a second disk, disk 1 retains 2 VGDAs and disk 2 gets 1 VGDA, for a quorum of 3. Adding a third disk results in the extra VGDA from the first disk moving to the third disk, for a quorum of 3 with each disk having one vote. Adding each additional disk adds one new VGDA per disk. VGDA expansion * The following pictures are a graphical view of VGDA expansion. * Shows how the VGDA grows from one disk to another o shows how the information is carried in the VGDA which cross-references information among the disks in a volume group. Case A, VGDA expansion [Image] Notes: Case A In these examples, the large box in the upper left is the VGDA. You can see how the VGDA is used to monitor disks and logical volumes that belong to a volume group. In Case A, the volume group, samplevg, contains only hdisk0. The example also shows how the VGDA holds the map for the logical volume lv00. There is no VGDA represented on hdisk1 since it is not part of the samplevg. However, it may be the case that hdisk1 contains some long-forgotten VGDA from a previous volume group. But since the VGDA for samplevg has no record of hdisk1 belonging to the volume group, hdisk1 is ignored in samplevg's eyes. Case B, VGDA expansion [Image] Notes: Case B Case B is an example of the aftermath of an extendvg of samplevg into hdisk1. A copy of the VGDA from hdisk0 is placed onto hdisk1. But, notice that the VGDA has also been updated on hdisk0 to reflect the inclusion of a new disk into the volume group. Although lv00 does not exist on hdisk1, the VGDA on

hdisk1 will have a record of the lv00 mapping sitting on another disk since it's part of the same volume group. Case C, VGDA Expansion [Image] Notes: Case C, VGDA Case C shows the result of the creation of two logical volumes, lv01 and lv02, onto hdisk1. This is to show how the VGDA sitting on hdisk0 is updated with the activity that is being conducted on another disk in the volume group. Logical Volumes Members of specific volume groups o cannot be spanned from one volume group to another. Can control location of logical volumes being created o control which disk or partition where logical volume is placed Can view location of logical volumes o lspv -p physical_volume Non-typical logical volumes o log logical volume used by JFS o dump device logical volume o boot logical volume o paging device logical volume Logical Volume Control Block - lvcb o consists of the first 512 bytes of a logical volume o holds important information + creation date of logical volume + information about mirrored copies + possible mount points into a JFS Logical Volume Identifier - LVID o Soft serial number used to represent the logical volume to the LVM libraries and low level commands. o Made up of VGID.<num>, where num is the order in which it was created in the volume group. Mirrors Sequential vs. Parallel Sequential Mirroring + Sequential mirroring based on concept of an order within mirrors. + All read-write requests go through a PRIMARY copy, and if the request was a write, it is propagated to the secondary copies. + Write request is not complete until the secondary copies have finished the write request. Parallel Mirroring + A read request will be serviced by the first available copy. + A write request will be broadcast to the copies and finishes when all are done. What good is Sequential Mirroring? o There is a definite PRIMARY copy which will be the source mirror copy for mirror resync. Parallel Mirroring is the most commonly used type.

Staleness of Mirrors o If a copy has become unavailable, it is referred to as "stale". o Tracked in the VGSA in terms of physical partitions. Notes: Mirrors Sequential Mirroring [Image] Parallel Mirroring [Image] What Does LVM Give You? * Flexibility * Data Integrity * Speed Flexibility * Real-time Volume Group and Logical Volume expansion/deletion * Use of Logical Volume under filesystem o LVM provides a physical to logical mapping layer, filesystem is defined on top of LVM. * Use of Logical Volume as raw data storage o It is not required to have a JFS on a logical volume. * User customized logical volumes * Ability to customize data integrity check Notes: Flexibility User can change mirror write consistency check, create mirroring and change the requirements for a quorum in a volume group. User can create logical volumes using a map file which allows them to specify the exact disks the logical volume will inhabit, and the order within that disk of the logical volume. Data Integrity * Mirrors o AIX allows up to three copies of a logical volume in sequential or parallel organization. * Mirror Write Consistency Check - MWCC o Method of tracking the last 62 writes to a mirrored LV. o If crash occurs, upon reboot the last 62 writes to mirrors are examined and one of the mirrors is used as a source to synchronize the mirrors. + For sequential mirroring, the latest write that occurred is guaranteed to be completed, since there is a definite PRIMARY disk which is used as the source for synchronization. + For parallel mirroring, all mirrors are guaranteed to be consistent, but the latest write is not guaranteed to be completed after the crash. * Ability to detect staleness and to correct o Stale PPs are tracked in VGSA. o No further writes are sent to stale PP, saving time. o When problem is corrected, partitions can be updated from other mirrors. * Quorum checking o LVM requires a quorum before working in a Volume Group. * Data relocation - due to physical problems with disk o Type 1: Internal to disk o Type 2: hardware relocate ordered by LVM o Type 3: Software relocate

Notes: Data Integrity There is a command used to back up mirrored logical volumes which essentially allows you to back up a mirrored logical volume and freeze the copy so that changes are not occuring while the backup is going on. This command is called splitlvcopy There are three types of data relocation: internal to the disk, hardware relocate ordered by LVM, and software relocation. Relocation typically occurs when the system fails to perform a read or a write due to physical problems with the disk platter. In some cases, the data i/o request completes, but with warnings. Depending on the type of recovered error, we may be so wary of the success of the next request to that physical location that LVM orders a relocation to be on the safe side. The lowest logical layer of relocation is the one that is internal to the disk. These types of relocations are typical private to the disk and there is no notification to the user that a relocation occurred. The next level up in terms of relocation complexity is a hardware relocation called for by the LVM device driver. This type of relocation will instruct the disk to relocate the data on one physical partition to another portion (reserved) of the disk. The disk takes the data in physical location A and copies it to a reserved portion of the disk, location B. However, after this is complete the LVM device driver will continue to reference physical location A, with the understanding that the disk itself will handle the true i/o to the real location B. The top layer of data relocation is the "soft" relocation handled by the LVM device driver. In this case, the LVM device driver maintains a bad block directory and whenever it receives a request to access a logical location A, the LVM device driver will look up the bad block table and translate it to actually send the request to the disk drive at physical location B. Write Verify * Every write to a physical portion of a disk that is part of a logical volume causes the disk device driver to issue the Write and Verify scsi command. * After each write, the disk will re-read the data to check that what was written was correct. * This obviously can have a performance impact. Speed * Mirrors o Main objective of Mirroring is Reliability of Data. o Parallel disk mirroring can improve the read performance of the system, but at a cost to the write performance. * Striping o Main objective of Striping is speed improvements for sequential access. o Striping is the concept of spreading sequential data across more than one disk to improve disk I/O. Notes: Speed [Image] We present an example to show the benefit of striping: A piece of data that is stored of the disk is 100 bytes. The physical cache of the system is only 25 bytes. Thus, it takes 4 read requests to the same disk to complete the reading of 100 bytes: As you can see, since the data is on the same disk, four sequential reads must be required. If this logical volume were created with a stripe width of 4 (how many disks) and a stripe size of 25 (how many consecutive bytes before going to the next disk), then you would see: [Image] As you can see, each disk only requires one read request and the time to gather all 100 bytes has been reduced 4-fold. However, there is still a bottleneck of having the four independent data disks channel

through one adapater card. But, this can be remedied with the expensive option of having each disk on an independent adapter card. Note the effect of using striping: the user has now lost the usage of 3 disks that could have been used for other volume groups. exportvg * The exportvg command is used to remove all knowledge of a volume group from a system. * exportvg deletes information about a volume group from ODM. * exportvg also cleans up some special files related to the volume group from the system. * Exportvg command does not affect the structures on disk, such as VGDA. * Before the disks associated with a volume group are removed, exportvg should be performed. Notes: exportvg Exportvg first makes sure the volume group is varied off. Then, it deletes /etc/filesystems entries associating logical volumes and filesystems. The /dev/special files are removed, and then ODM entries associated with the volume group are deleted from ODM. No actoin is taken on the physical disk platter data structures, such as VGDA. importvg * used to bring a set of disks composing a volume group into a new system. * importvg reads information from the on-disk VGDA to understand the new volume group configuration. Notes: importvg AIX makes moving disks composing volume groups from one system to another easy. importvg is used to bring a set of disks composing a volume group into a new system. importvg allows the system to learn which disks constitute the data disks, what logical volumes exist on the data disks, what mount points exist for logical volumes, PVIDs for the disks, LVIDs for the logical volumes, and preferred logical volume names. The system will prevent name collisions between LVM entries in the new and existing system, and LVM may rename the logical volume without user approval to avoid name collisions. reducevg * reducevg is used to remove a disk from a volume group. * reducevg is totally different from exportvg. * reducevg modifies both ODM and the on disk VGDA. Notes: reducevg Reducevg removes information about the disk and any logical volumes residing on that disk. This includes logical volumes that may straddle that disk and other disks. If any part of the logical volume is on the disk to be removed from the volume group, the logical volume is removed. reducevg removes information about the disk from the VGDA on the remaining disks in the volume group, as well as the target disk of the reducevg. If you wish to later import a volume group into a new system, use exportvg to remove information from the original system. Using reducevg on all the disks in the volume group is not equivalent to using exportvg. If you use reducevg on all the disks in the volume group, and then try to import that volume group into a new system, all the VGDA information needed by the new system will have been removed by the reducevg. varyonvg * varyonvg activates a volume group & logical volumes. * checks for a quorum of physical volumes. * checks for discrepancy between on-disk structures and ODM. Notes: varyonvg varyonvg can be run on an already varied-on volume group.

Use with the -b flag (4.2 and later) to break a lock on disks locked as a result of some other command. There exists an ODM object known as the vg-lock. Whenever an LVM modification command is started, the LVM command will lock the vg-lock for the volume group. If for some reason the lock is inadvertently left behind, the volume group can be unlocked by running the varyonvg -b command. Problem Determination * Questions to ask... * What is the root cause of the error? * Can this problem be distilled to the simplest case? o Try to find a simple test case to cause the problem. * What has been lost and what is left behind? * Is this situation repairable? o Sometimes may not be. Problem Recovery * Warn the user of the consequences o Sometimes failed recovery attempts can leave the system worse off than before. * Gather all possible data o get info from the VGDA, ODM and errpt. * Save off what can be saved o Save any files that have not been corrupted, such as ODM database files in /etc/objrepos. Save any other important files. * Each case is different, so must be the solution Failure/Recovery Examples * Example1: Resynchronize failing physical volume * Example2: Replace failing physical volume * Example3: Bad magic number in superblock * Example4: Fix LVM/ODM inconsistency on non-rootvg volume group * Example5: Fix LVM/ODM inconsistency on rootvg volume group * Example6: Fix Phantom ODM entries Example1: Resynchronize Good PV Gone Offline * Use these steps to reintroduce a physical volume o went offline for some reason without loss of quorum (if mirrored) or data integrity. The cause of problem is a bad cable, etc. * The data on the physical volume must be intact and the disk itself must not be damaged. o If data on the physical volume is not intact, use the next recovery procedure. Example 1 Steps to Repair * Remove the device from the system configuration in the ODM Device Configuration Database. rmdev -l PVname * Repair the problem. (loose or bad cable, unseated adapter etc.) * Add the disk back into the ODM Device Configuration Database. mkdev -l PVname * Synchronize the stale physical partitions. This synchronizes logical volume copies that are not current. syncvg -p PVname Example 1 assumes that the logical volumes in the volume group were mirrored. Example2: Replace Failing Physical Volume

* Use the following steps to remove a failing physical volume from the system and replace the physical volume. Example 2 Steps * Back up all single-copy logical volumes on the failing device, if possible. * Unmount all single copy filesystems from the disk. Mirrored filesystems do not have to be unmounted. unmount /DIRECTORY1 * Remove all single copy filesystems from the failed physical volume. rmfs /DIRECTORY1 * Remove remaining physical page copies from the physical volume. rmlvcopy PVname #REMAINING_PPs * Remove the physical volume from the volume group reducevg -df VGname PVname * Remove the device from the Device Configuration Database in ODM. rmdev -l PVname * Remove the device and replace it with a working device. Configure the new device into the system using smit or cfgmgr. * Make the device available. chdev -l PVname -a pv=yes * Add the new physical volume to the volume group. extendvg VGname PVname * Remake the logical volumes on the physical volume. mklv -OPTIONS VGname #PPs PVname * Recreate the filesystems on the logical volume. crfs -v jfs -d LVname -m /DIRECTORY1 * Extend multiple-copy logical volumes onto the disk again. mklvcopy LVname #LPCOPIES PVname * Resync copied physical partitions to the new physical volume. syncvg -p PVname * Restore single copy filesystem data from backup media to logical volumes using utilities such as tar. Notes: Example 2 How do you tell if it is a single copy filesystem (i.e. not mirrored)? Single copy filesystems have the same amount of logical partitions as physical partitions on the output from the lspv command. Pages that remain on the physical volume after rmfs are mirrored on other physical volumes. Example3: Bad Magic Number in Superblock * The primary superblock of a filesystem can be corrupted. * Most damage to the superblock is catastrophic. * One simple and correctable problem is a bad magic number. o Generally this problem shows up when using fsck on a filesystem. The fsck command returns fsck: Not an AIXV4 filesystem. Example 3 Steps to Repair 1. Determine the logical volume associated with the failing filesystem. 2. Display the primary superblock using the od utility. 3. Determine what data is corrupted in the primary superblock by comparing the data with the superblock structure found in /usr/include/jfs/filsys.h. If the magic number is corrupted, the value will be something other than the contents of kernel variable fsv3magic. Currently, this value is 0x43218765. 4. Display secondary superblock using the od utility (see next page for example).

5. Determine what data is corrupted in the secondary superblock by comparing the data with the superblock structure found in /usr/include/jfs/filsys.h. 6. If the magic number is correct (0x43218765 for an AIX V3 filesystem or 0x65872143 for an AIX V4 filesystem), the secondary superblock is probably correct. If the secondary superblock is intact, use the dd command to copy the secondary superblock to the primary superblock. * Step 2: Display primary superblock using od. Note the bad magic value at byte 0x1000. $ od -x -N 64 /dev/lv00 +0x1000 0001000 1234 0234 0000 0000 0000 4000 0000 000a 0001010 0001 8000 1000 0000 2f6c 7633 0000 6c76 0001020 3300 0000 000a 0003 0100 0000 2f28 0383 0001030 0000 0001 0000 0200 0000 2000 0000 0000 0001040 * Step 4: Display secondary superblock. Note correct magic value 0x65872143 at 0x1f000. $ od -x -N 64 /dev/lv00 +0x1f000 001f000 6587 2143 0000 0000 0000 4000 0000 000a 001f010 0001 8000 1000 0000 2f6c 7633 0000 6c76 001f020 3300 0000 000a 0003 0100 0000 2f28 0383 001f030 0000 0001 0000 0200 0000 2000 0000 0000 001f040 You must be superuser to do this recovery. The example output assumes that lv00 is the logical volume containing the filesystem with the bad magic number and that it is in rootvg. If the filesystem is an AIX Version 4 filesystem (i.e., created with nondefault values of fragment size and nbpi or compressed), the magic number will be the value of fsv3pmagic in /usr/include/jfs/filsys.h and that is 0x65872143. The s_time field of the secondary superblock can be checked to determine the last time the secondary superblock was updated. This time is given in seconds since epoch. Note: The hdf utility has been removed in AIX V4. However, the same functionality can be got by using the -h flag with lquerypv as - "lquerypv -h /dev/lv00 0x1000 60" or the hdf executable from an AIX V3 system can be run on an AIX V4 system as "hdf /dev/lv00 0x1000 60". These are alternatives to using the od command as shown on the foil. * Step 6: Copy the secondary superblock to the primary superblock. dd count=1 bs=4k skip=31 seek=1 if=/dev/lv00 of=/dev/lv00 dd: 1+0 records in. dd: 1+0 records out. * Step 7: Use logform to clear out the log for the filesystem. /usr/sbin/logform /dev/hd8 * Step 8: Unmount the filesystem. unmount /dev/lv00 * Step 9: Use fsck to clean up inconsistent files caused by using secondary superblock. fsck /dev/lv00 2>&1 | tee /tmp/fsck.errs Example4: Fix LVM/ODM Inconsistency on non-rootvg VG * If data in the ODM becomes inconsistent with data on the physical disk in the VGDA, VGSA or LVCB, the volume group or portions thereof may be

made unusable. * This example damages the ODM data base, shows the symptoms of the damage, then fixes the damage. The system does not have to be in maintenance mode to perform these recovery steps. * This procedure does not work with the rootvg volume group. Notes: Example 4 LVM/ODM Inconsistency in nonrootvg Damage to the ODM was done by typing: $ odmdelete -q "name=testvg" -o CuAt ( Do not actually try to damage the ODM (especially rootvg)). Then tried: $ mklv -a ie -y lra3 -x 30 testvg 20 0516-328 putlvodm: Identifier not entered. Usage: putlvodm [-a IntraPolicy] [-B label] [-c Copies] [-e InterPolicy] [-L LVid] [-l LVname] [-n NewLVName] [-r Relocate] [-s Strict] [-t Type] [-u UpperBound] [-y Copyflag] [-z Size] LVid putlvodm [-o Auto-on] [-k] [-K] [-q VGstate] [-v VGname -m majornum] [-V] VGid putlvodm [-p VGid] [-P] PVid 0516-822 mklv: Unable to create logical volume. Example 4 Recovery Steps * Export the broken volume group to remove entries from the Device Configuration Database of the ODM exportvg testvg * Reassociate the volume group with the physical volumes. In this case the volume group testvg was located on physical volume hdisk1. Any disk that was previously in the volume group can be specified. importvg -y testvg hdisk1 * Vary online the volume group . If data on the disk is intact, the volume group is restored. varyonvg -f testvg Example5: Fix LVM/ODM Inconsistency on rootvg * These steps recover the rootvg volume group. * First try using redefinevg while system is running. o If that doesn't fix the problem must go into maintenance mode. Notes: Example 5 In order to perform the following repair, the system must be in the RAM Disk Maintenance Mode . The steps can also be used to recover other volume groups by substituting the appropriate physical volume and volume group information. It is suggested that this example be made a script. Service boot menu: ---------------------------------------------------------------------------------------Select One Option: 1. Boot to Diagnostics mode

2. Boot to Single User mode 3. RAM disk maintenance mode Choice [1-3] : ----------------------------------------------------------------------------------------* Delete all of the ODM information about logical volumes. o Get the list of logical volumes from the VGDA of the physical volume. * Delete the volume group information from ODM. Assume that hdisk0 is part of rootvg. In CuDvDr, value1 = major number, value2=minor number, value3= object name for major/minor number. rootvg typically has value1 = 10, due to the order of it's creation. 1. lqueryvg -p hdisk0 -L | awk '{print $2 }' | while read LVname; do odmdelete -q "name=$LVname" -o CuAt odmdelete -q "name=$LVname" -o CuDv odmdelete -q "value3=$LVname" -o CuDvDr done 2. odmdelete -q "name=rootvg" -o CuAt odmdelete -q "parent=rootvg" -o CuDv odmdelete -q "name=rootvg" -o CuDv odmdelete -q "name=rootvg" -o CuDep odmdelete -q "dependency=rootvg" -o CuDep odmdelete -q "value1=10" -o CuDvDr odmdelete -q "value3=rootvg" -o CuDvDr * Add the volume group associated with the physical volume back to the ODM. * Recreate the Device Configuration Database in the ODM from information on the physical volume. 3. importvg -y rootvg hdisk0 4. varyonvg -f rootvg Example6: Phantom ODM entries * Problem: a disk is removed from a system, leaving behind phantom ODM entries. How to clean up? * Try to do rmlv of the phantom lvs, reducevg on the volume group, and rmdev of the disk. * If this doesn't work you must delete all the information about the volume group from ODM by hand. Notes: Example 6 The most common problem seen at customer locations is that of an external disk being yanked off without doing the necessary pre-yanking operations such as reducevg, rmdev etc. How best can you clean up the system once the damage has been done? In other words, what operations will purge the ODM of the entries pertaining to the removed disk, it's volume group, it's logical volumes and filesystems? You will see that the lsvg and lspv commands still show you that a volume group (ex. newvg) that used to exist on the disk is present. However, it is not really there, because the disk has been removed.

If the first commands don't succeed, the only way to completely clean up the ODM is by doing the following 1. First, backup the Cu* files just in case you end up deleting more than you should mkdir /tmp/Cu.bak cp /etc/objrepos/Cu* /tmp/Cu.bak 2. Search for all instances of the volume group, hdisk#, lv#, loglv# in each of the customized classes. For example, odmget CuAt | grep newvg odmget CuAt | grep hdisk# will show you the attribute of these objects in this class. Search in all the customized classes - CuAt, CuDv, CuDep, CuDvDr and CuVPD. Each of them needs to be cleaned up. Generally CuAt will handle the names, lvids, pvids and vgids of the volume groups and logical volumes. CuDvDr controls major/minor number generation. In CuDvDr, value1 = major number, value2=minor number, value3= object name for major/minor number. 3. Next, delete what you find in 2. as follows If the output of 2. says name = "newvg" You can clean this by using odmdelete -o CuAt -q "name=newvg" If the output of "odmget CuDvDr | grep hdisk3" says value3 = hdisk3 type odmdelete -o CuDvDr -q "value3=hdisk3"