Logical Volume Management

Michal Sedlak
michal6103@gmail.com
 Common problems
● Resize partition?
● Use space on separated disks?
● Use speed of multiple devices?
● Write same data on more devices?
● Backup used filesystem?

2/29
 LVM
● Flexible capacity - multiple disks/partitions
● Resizeable storage pools - extend/reduce
● Online data relocation
● Convenient device naming - names/tags
● Disk striping
● Mirroring volumes
● Volume Snapshots

3/29
Concept

4/29
 PV
● partition, disk, LUN
● LVM label - 2nd sector
● 0, 1 or 2 copies of ASCII metadata
● More PV on one disk?

# pvcreate /dev/sd[a-c]1

5/29
 VG
● Pool of PVs
● extent
● fixed for whole VG
● smallest allocable space
● Size of logical extent = Size of physical extent
● Map logical extents to physical extents

# vgcreate -s 4M vg_test /dev/sd[a-c]1

6/29
 LV
● Linear
● Striped
● Mirrored
● Snapshot
● Virtual

7/29
 Linear
● concatenate physical
extents to logical
volume
● flexible space
allocation

● Example:
● 2x99 physical extents
● 198 logical extents
# lvcreate -L 50G -n lv_linear vg_test
# lvcreate -l 100%FREE -n lv_linear vg_test
8/29
 Striped
● round robin
● effective for large I/O

# lvcreate -L 50G -i 3 -I 128 -n lv_linear vg_test

9/29
Mirrored
● Identical copies
● Mirror log on another
device or in memory
● Mirror != backup
● mirror_image_fault_p
olicy
● allocate
● remove

10/29
 Mirror creation
# lvcreate -m1 -L 1G -n lv_mirror vg_test
# lvcreate -m1 -L 2T -R 2 -n lv_mirror vg_test
# lvcreate -m1 -L 1G --mirrorlog core -n lv_mirror vg_test
# lvcreate -m1 -L 1G -n lv_mirror -alloc anywhere vg_test
# lvcreate -m1 -L 1G --mirrorlog mirrored -n lv_mirror vg_test
# lvcreate -m1 -L 1G -n lv_mirror vg_test /dev/sda1 /dev/sdb1
/dev/sdc1

# lvcreate -m1 -l 5 --mirrorlog mirrored -n lv_mirror vg_test /

> /dev/sda1:0-4 /dev/sdb1:0-4 /dev/sda1:5 /dev/sdb1:5

# lvcreate -m1 -i2 --mirrorlog core -n lv_lvm vg_four

11/29
 Snapshot
● stores only differences
● origin and snapshot, both are RW
● snapshot != backup
● 100% full is lost completely
● Use
● backup
● fsck
● test/rollback on production data “--merge”
● virtual machines
lvcreate --size 100M --snapshot --name lv_snap /dev/vg0/lv0
12/29
 Virtual
● --virtualsize VirtualSize
● sparse device

$ lvcreate --virtualsize 1T --size 100M --snapshot --name sparse vg1

13/29
 Device mapper
● linear
● striped
● mirror
● snapshot and snapshot-origin
● error
● zero
● multipath
● crypt

14/29
$ sudo dmsetup ls --tree
vg0-lv0 (253:5)
├─vg0-lv0_mimage_1 (253:2)
│ ├─ (7:2)
│ ├─ (7:1)
│ └─ (7:0)
├─vg0-lv0_mimage_0 (253:1)
│ ├─ (7:6)
│ ├─ (7:5)
│ └─ (7:4)
└─vg0-lv0_mlog (253:0)
└─ (7:3)

$ sudo dmsetup status

vg0-lv0_mlog: 0 8192 linear
vg0-lv0_mimage_1: 0 2850816 striped 3 7:0 7:1 7:2 1 AAA
vg0-lv0_mimage_0: 0 2850816 striped 3 7:4 7:5 7:6 1 AAA
vg0-lv0: 0 2850816 mirror 2 253:1 253:2 2784/2784 1 AA 3 disk 253:0 A

15/29
$ sudo dmsetup info
Name: vg0-lv0_mlog
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: LVM-ggZ5NqMTOZx43akuLZxxn03JdQGNta6gWlHnr0WT1Ua8Ek56ke6X9CdH8m3OVsAg
Name: vg0-lv0_mimage_1
State: ACTIVE
Read Ahead: 768
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 2
Number of targets: 1
UUID: LVM-ggZ5NqMTOZx43akuLZxxn03JdQGNta6gKIufDqbRgOaURN83IeA2AbYfH0zwTI0d
Name: vg0-lv0_mimage_0
State: ACTIVE
Read Ahead: 768
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 1
Number of targets: 1
UUID: LVM-ggZ5NqMTOZx43akuLZxxn03JdQGNta6g23mlbEbEk5r5l9Hy6YgRxuu2hd4DFg0Q
Name: vg0-lv0
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 1 16/29
Example: Striped Mirror

17/29
$ cd ~/tmp
$ for i in {0..6}; do dd if=/dev/zero of=lvm$i.img bs=1 count=0 /
seek=505M; done
$ ls -lh lvm*
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm0.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm1.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm2.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm3.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm4.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm5.img
-rw-rw-r-- 1 michal michal 505M 2011-10-20 22:36 lvm6.img
$ for i in {0..6}; do sudo losetup /dev/loop$i lvm$i.img; done

$ sudo losetup -a
/dev/loop0: [0802]:1966260
/dev/loop1: [0802]:1966347
/dev/loop2: [0802]:1966350
/dev/loop3: [0802]:1966351
/dev/loop4: [0802]:1966354
/dev/loop5: [0802]:1966355
/dev/loop6: [0802]:1966356
(/home/michal/tmp/lvm0.img)
(/home/michal/tmp/lvm1.img)
(/home/michal/tmp/lvm2.img)
(/home/michal/tmp/lvm3.img)
(/home/michal/tmp/lvm4.img)
(/home/michal/tmp/lvm5.img)
(/home/michal/tmp/lvm6.img)

18/29
$ sudo pvcreate /dev/loop[0-6]
Physical volume "/dev/loop0" successfully created
Physical volume "/dev/loop1" successfully created
Physical volume "/dev/loop2" successfully created
Physical volume "/dev/loop3" successfully created
Physical volume "/dev/loop4" successfully created
Physical volume "/dev/loop5" successfully created
Physical volume "/dev/loop6" successfully created

$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/loop0 lvm2 a- 500,00m 500,00m
/dev/loop1 lvm2 a- 500,00m 500,00m
/dev/loop2 lvm2 a- 500,00m 500,00m
/dev/loop3 lvm2 a- 500,00m 500,00m
/dev/loop4 lvm2 a- 500,00m 500,00m
/dev/loop5 lvm2 a- 500,00m 500,00m
/dev/loop6 lvm2 a- 500,00m 500,00m

$ sudo vgcreate vg0 /dev/loop[0-6]

Volume group "vg0" successfully created

$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg0 7 0 0 wz--n- 3,39g 3,39g
19/29
$ sudo vgdisplay
--- Volume group ---
VG Name vg0
System ID
Format lvm2
Metadata Areas 7
Metadata Sequence No 22
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 7
Act PV 7
VG Size 3,39 GiB
PE Size 4,00 MiB
Total PE 868
Alloc PE / Size 0 / 0
Free PE / Size 868 / 3,39 GiB
VG UUID ggZ5Nq-MTOZ-x43a-kuLZ-xxn0-3JdQ-GNta6g

$ sudo lvcreate -i2 -m1 -L1200m -n lv0 vg0

Using default stripesize 64,00 KiB
Logical volume "lv0" created

20/29
$ sudo lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
lv0 vg0 mwi-a- 1,17g lv0_mlog 10,29 lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop0(0),/dev/loop1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop2(0),/dev/loop3(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop4(0),/dev/loop5(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop2(26),/dev/loop3(27)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(26)

$ sudo mkfs.ext4 /dev/vg0/lv0

$ sudo lvconvert -m0 --stripes 3 vg0/lv0

Logical volume lv0 converted.

$ sudo mount /dev/vg0/lv0 /mnt

$ sudo lvs -a -o +devices

LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
lv0 vg0 -wi-a- 1,17g /dev/loop0(0),/dev/loop1(0)
lv0 vg0 -wi-a- 1,17g /dev/loop2(0),/dev/loop3(0)

$ sudo lvconvert -m1 --stripes 3 vg0/lv0

Using default stripesize 64,00 KiB
vg0/lv0: Converted: 0,3%
vg0/lv0: Converted: 4,7%
.
.
vg0/lv0: Converted: 100%

21/29
$ sudo lvs -a -o +devices
LV VG Attr LSize Log Devices
lv0 vg0 mwi-a- 1,17g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop0(0),/dev/loop1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop2(0),/dev/loop3(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(26)

$ sudo lvconvert --splitmirrors 1 --name lv_split vg0/lv0

Logical volume lv0 converted.

$ sudo lvs -a -o +devices

LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
lv0 vg0 -wi-a- 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
lv_split vg0 -wi-a- 1,17g /dev/loop0(0),/dev/loop1(0)
lv_split vg0 -wi-a- 1,17g /dev/loop2(0),/dev/loop3(0)

$ sudo lvremove vg0/lv_split

Do you really want to remove active logical volume lv_split? [y/n]: y
Logical volume "lv_split" successfully removed

$ sudo lvs -a -o +devices

LV VG Attr LSize Origin Snap% Move Devices
lv0 vg0 -wi-a- 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)

$ sudo lvconvert -m1 --stripes 3 vg0/lv0

Using default stripesize 64,00 KiB

22/29
$ sudo lvs -a -o +devices
LV VG Attr LSize Log Devices
lv0 vg0 mwi-a- 1,17g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(0)

$ fsck /dev/vg0/lv0
fsck from util-linux 2.19.1
e2fsck 1.41.14 (22-Dec-2010)
/dev/mapper/vg0-lv0 is mounted.
WARNING!!! The filesystem is mounted. If you continue you ***WILL***
cause ***SEVERE*** filesystem damage.
Do you really want to continue (y/n)? no
check aborted.

$ sudo lvcreate --size 100M --snapshot --name lv_snap /dev/vg0/lv0

Logical volume "lv_snap" created
$ sudo fsck /dev/vg0/lv_snap
fsck from util-linux 2.19.1
e2fsck 1.41.14 (22-Dec-2010)
/dev/mapper/vg0-lv_snap: clean, 12/76800 files, 13475/307200 blocks

23/29
$ sudo lvs -a -o +devices
LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,36g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 Iwi-ao 1,36g /dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(0)

$ df -h /mnt
Súborový systém Size Used Avail Use% Pripojený na
/dev/mapper/vg0-lv0 1,2G 34M 1,1G 4% /mnt

$ sudo resize2fs /dev/vg0/lv0

resize2fs 1.41.14 (22-Dec-2010)
Filesystem at /dev/vg0/lv0 is mounted on /mnt; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/vg0/lv0 to 356352 (4k) blocks.
The filesystem on /dev/vg0/lv0 is now 356352 blocks long.

$ df -h /mnt
Súborový systém Size Used Avail Use% Pripojený na
/dev/mapper/vg0-lv0 1,4G 34M 1,3G 3% /mnt

24/29
$ sudo dd if=/dev/zero of=/dev/loop1 count=10
10+0 záznamov prečítaných
10+0 záznamov zapísaných
5120 skopírovaných bajtov (5,1 kB), 0,00188767 s, 2,7 MB/s
$ sudo lvs -a -o +devices
Couldn't find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J.
LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,36g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 iwi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 iwi-ao 1,36g /dev/loop0(0),unknown device(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(0)

$ sudo vgreduce --removemissing vg0

Couldn't find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J.
WARNING: Partial LV lv0 needs to be repaired or removed.
WARNING: Partial LV lv0_mimage_1 needs to be repaired or removed.
WARNING: There are still partial LVs in VG vg0.
To remove them unconditionally use: vgreduce --removemissing --force.
Proceeding to remove empty missing PVs.

$ sudo vgreduce --removemissing vg0 --force

Couldn't find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J.
WARNING: Bad device removed from mirror volume, vg0/lv0
WARNING: Mirror volume, vg0/lv0 converted to linear due to device failure.
Wrote out consistent volume group vg0

25/29
 Corrupted metadata
$ sudo lvs -a -o +devices
LV VG Attr LSize Devices
lv0 vg0 -wi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)

$ sudo pvcreate --uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J /dev/loop1

Physical volume "/dev/loop1" successfully created

$ sudo vgcfgrestore -f /etc/lvm/archive/vg0_00170.vg vg0

Restored volume group vg0

$ sudo lvs -a -o +devices

LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,27g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 iwi--- 1,27g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 iwi--- 1,27g /dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi--- 4,00m /dev/loop3(0)

26/29
$ sudo lvs -a -o +devices
New disk
LV VG Attr LSize Devices
lv0 vg0 -wi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)

$ sudo pvcreate --restorefile /etc/lvm/archive/vg0_00170.vg --uuid "hmPWAT-kPoM-yHZ8-

Yh5M-m11G-yDjI-WLA28J"
Physical volume "/dev/loop1" successfully created

$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/loop0 vg0 lvm2 a- 496,00m 496,00m
/dev/loop1 lvm2 a- 505,00m 505,00m
/dev/loop2 vg0 lvm2 a- 496,00m 496,00m
/dev/loop3 vg0 lvm2 a- 496,00m 496,00m
/dev/loop4 vg0 lvm2 a- 496,00m 64,00m
/dev/loop5 vg0 lvm2 a- 496,00m 64,00m
/dev/loop6 vg0 lvm2 a- 496,00m 64,00m

$ sudo vgextend vg0 /dev/loop1

Volume group "vg0" successfully extended

$ sudo lvconvert -m1 --stripes 3 vg0/lv0

$ sudo lvs -a -o +devices

LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,27g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,27g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 Iwi-ao 1,27g /dev/loop0(0),/dev/loop2(0),/dev/loop3(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop1(0) 27/29
 Caveats
● Until Linux kernel 2.6.31[1], write barriers were
not supported (fully supported in 2.6.33). This
means that the guarantee against filesystem
corruption offered by journaled file systems like
ext3 and XFS was negated under some
circumstances.[2]
● Device mapper does not honor barriers
● Barriers are disabled by default
● mount -t ext3 -o barrier=1 <device> <mount
point>

28/29
Thank you

Questions

29/29
Thank you

Questions

30/29
 RHEL 6
● mirror_image_fault_policy=allocate/remove
● data_alignment_detection
● udev integration
● lvconvert --repair
● lvconvert --merge
● lvconvert --splitmirrors
● lvcreate --mirrorlog mirrored
● lvextend --alloc cling
● multiple --addtag and --deltag

31/29