Comment 5 for bug 1197648

Revision history for this message
Rongze Zhu (zrzhit) wrote : Re: LVM Unable to clone a volume

After we created the clone volume, we need to zero the temp_snapshot, cinder will write snapshot full and an error occurred in device-mapper.

We can reproduce it .

OS: Ubuntu 12.04.2
Kernel: 3.2.0-40-virtual
LVM:
  LVM version: 2.02.66(2) (2010-05-20)
  Library version: 1.02.48 (2010-05-20)
  Driver version: 4.22.0

### 1. create vg

ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo truncate -s 10G stack-volumes
ubuntu@galaxy2:/mnt$ ll
total 24
drwxr-xr-x 3 root root 4096 Jul 5 07:09 ./
drwxr-xr-x 23 root root 4096 Jul 5 04:36 ../
drwx------ 2 root root 16384 Jun 11 08:29 lost+found/
-rw-r--r-- 1 root root 10737418240 Jul 5 07:09 stack-volumes
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo losetup -f --show stack-volumes
/dev/loop0
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo pvcreate /dev/loop0
  Physical volume "/dev/loop0" successfully created
ubuntu@galaxy2:/mnt$ sudo vgcreate vg0 /dev/loop0
  Volume group "vg0" successfully created

### 2. create the origin volume and snapshots of the origin volume

ubuntu@galaxy2:/mnt$ sudo lvcreate -L 1G -n lv0 vg0
  Logical volume "lv0" created
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvs
  LV VG Attr LSize Origin Snap% Move Log Copy% Convert
  lv0 vg0 -wi-a- 1.00g
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvcreate -L 2G -n lv0_snapshot_0 --snapshot vg0/lv0
  Logical volume "lv0_snapshot_0" created
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvcreate -L 1G -n lv0_snapshot_1 --snapshot vg0/lv0
  Logical volume "lv0_snapshot_1" created
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvs
  LV VG Attr LSize Origin Snap% Move Log Copy% Convert
  lv0 vg0 owi-a- 1.00g
  lv0_snapshot_0 vg0 swi-a- 2.00g lv0 0.00
  lv0_snapshot_1 vg0 swi-a- 1.00g lv0 0.00
ubuntu@galaxy2:/mnt$

### 3. zero the first snapshot

ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo dd if=/dev/zero of=/dev/mapper/vg0-lv0_snapshot_0 bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 124.915 s, 8.6 MB/s
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvs
  LV VG Attr LSize Origin Snap% Move Log Copy% Convert
  lv0 vg0 owi-a- 1.00g
  lv0_snapshot_0 vg0 swi-a- 2.00g lv0 50.20
  lv0_snapshot_1 vg0 swi-a- 1.00g lv0 0.00
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvdisplay vg0/lv0_snapshot_0
  --- Logical volume ---
  LV Name /dev/vg0/lv0_snapshot_0
  VG Name vg0
  LV UUID KE7i1E-ZNCR-mc2Z-XNtQ-wiO3-tNjE-a8p2pT
  LV Write Access read/write
  LV snapshot status active destination for /dev/vg0/lv0
  LV Status available
  # open 0
  LV Size 1.00 GiB
  Current LE 256
  COW-table size 2.00 GiB
  COW-table LE 512
  Allocated to snapshot 50.20%
  Snapshot chunk size 4.00 KiB
  Segments 1
  Allocation inherit
  Read ahead sectors auto
  - currently set to 256
  Block device 252:1

ubuntu@galaxy2:/mnt$

### 4. zero the second snapshot

ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo dd if=/dev/zero of=/dev/mapper/vg0-lv0_snapshot_1 bs=1M count=1024 conv=fdatasync
dd: fdatasync failed for `/dev/mapper/vg0-lv0_snapshot_1': Input/output error
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 48.987 s, 21.9 MB/s
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvs
  /dev/dm-4: read failed after 0 of 4096 at 0: Input/output error
  LV VG Attr LSize Origin Snap% Move Log Copy% Convert
  lv0 vg0 owi-a- 1.00g
  lv0_snapshot_0 vg0 swi-a- 2.00g lv0 50.20
  lv0_snapshot_1 vg0 Swi-I- 1.00g lv0 100.00
ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ sudo lvdisplay vg0/lv0_snapshot_1
  /dev/dm-4: read failed after 0 of 4096 at 0: Input/output error
  --- Logical volume ---
  LV Name /dev/vg0/lv0_snapshot_1
  VG Name vg0
  LV UUID fO6gFN-c2b5-8bQO-TsKU-or9Q-SYEy-zlxY38
  LV Write Access read/write
  LV snapshot status INACTIVE destination for /dev/vg0/lv0
  LV Status available
  # open 0
  LV Size 1.00 GiB
  Current LE 256
  COW-table size 1.00 GiB
  COW-table LE 256
  Snapshot chunk size 4.00 KiB
  Segments 1
  Allocation inherit
  Read ahead sectors auto
  - currently set to 256
  Block device 252:4

ubuntu@galaxy2:/mnt$
ubuntu@galaxy2:/mnt$ dmesg
[2069983.216951] device-mapper: snapshots: Invalidating snapshot: Unable to allocate exception.
[2070028.707217] quiet_error: 136 callbacks suppressed
[2070028.707223] Buffer I/O error on device dm-4, logical block 261122
[2070028.707236] Buffer I/O error on device dm-4, logical block 261123
[2070028.707240] lost page write due to I/O error on dm-4
......

###########

we notice that it is 50.20% in Snap% after we wrote 1024MB data to the first snapshot.

2048 * 50.2% = 1028.096
1028.096 - 1024 = 4.096

I think 4.096MB is additional metadata of snapshot that need to be written to snapshot space along with 1024 MB data.
So, I think when we wrote 1024MB data to the second snapshot , data and additional data is more than snapshot size, an error occurred in device-mapper.