LVM in 16.04 cannot mount LVM Cached Volumes created in 15.10

Bug #1556602 reported by Ben McCann on 2016-03-13
This bug affects 3 people
Affects Status Importance Assigned to Milestone
lvm2 (Ubuntu)

Bug Description

I'm test driving Ubuntu Mate Beta 1 (from USB) on a machine that uses LVM caching. (I use some SSD to cache old hard drives). My LVM setup was created with Linux Mint Debian Edition - Betsy release.

The system cannot mount these cached volumes due to the following error reported by 'pvs':

ubuntu-mate@ubuntu-mate:~$ sudo pvs
  LV CacheArchive has invalid cache's feature flag.
  LV CacheArchive is missing cache policy name.
  Internal error: LV segments corrupted in CacheArchive.
  LV CacheArchive has invalid cache's feature flag.
  LV CacheArchive is missing cache policy name.
  Internal error: LV segments corrupted in CacheArchive.
  PV VG Fmt Attr PSize PFree
  /dev/sda5 vga lvm2 a-- 80.00g 70.00g
  /dev/sda6 vga lvm2 a-- 160.00g 50.00g
  /dev/sdb1 vga lvm2 a-- 200.00g 50.00g

I did a bit of googling and found a similar report for upgrades from Fedora 21 to Fedora 23. See

I'll annotate this bug with the LVM version from my existing Debian based configuration in a few minutes.

This is a show-stopper for me because I use LVM caching on both of my PCs. It's a great way to marry a 256 GB SSD with a 2 TB hard drive. I keep the OS and some key partitions on the SSD and then use the rest of the SSD to cache the hard drive.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: lvm2 2.02.133-1ubuntu7
ProcVersionSignature: Ubuntu 4.4.0-7.22-generic 4.4.2
Uname: Linux 4.4.0-7-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CasperVersion: 1.367
CurrentDesktop: MATE
Date: Sun Mar 13 12:48:41 2016
LiveMediaBuild: Ubuntu-MATE 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160224)
 PATH=(custom, no user)
SourcePackage: lvm2
UpgradeStatus: No upgrade log present (probably fresh install)

Ben McCann (ben-mccann) wrote :
Ben McCann (ben-mccann) wrote :

Here is the package info for LVM2 on LDME Betsy:

dpkg-query -l lvm2
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
ii lvm2 2.02.111-2.2 amd64 Linux Logical Volume Manager

aptitude show lvm2
Package: lvm2
State: installed
Automatically installed: no
Multi-Arch: foreign
Version: 2.02.111-2.2
Priority: optional
Section: admin
Maintainer: Debian LVM Team <email address hidden>
Architecture: amd64
Uncompressed Size: 1,663 k
Depends: libc6 (>= 2.15), libdevmapper-event1.02.1 (>= 2:1.02.74),
         libdevmapper1.02.1 (>= 2:1.02.90), libreadline5 (>= 5.2), libudev1 (>=
         183), init-system-helpers (>= 1.18~), lsb-base, dmsetup (> 2:1.02.47),
         dmeventd (> 2:1.02.90), initscripts (>= 2.88dsf-13.3)
Suggests: thin-provisioning-tools
Conflicts: lvm2
Breaks: initramfs-tools (< 0.99), initramfs-tools (< 0.99),
        thin-provisioning-tools (< 0.3.2), thin-provisioning-tools (< 0.3.2)
Provides: lvm2
Provided by: lvm2
Description: Linux Logical Volume Manager
 This is LVM2, the rewrite of The Linux Logical Volume Manager. LVM supports
 enterprise level volume management of disk and disk subsystems by grouping
 arbitrary disks into volume groups. The total capacity of volume groups can be
 allocated to logical volumes, which are accessed as regular block devices.

Tags: admin::filesystem, interface::commandline, role::program, scope::utility,

Hmmm... I wonder whether thin-provisioning-tools was missing on the Ubuntu Xenial install ISO. Here's that package:

aptitude show thin-provisioning-tools
Package: thin-provisioning-tools
State: installed
Automatically installed: no
Version: 0.3.2-1
Priority: optional
Section: admin
Maintainer: Debian LVM Team <email address hidden>
Architecture: amd64
Uncompressed Size: 17.1 M
Depends: libc6 (>= 2.14), libexpat1 (>= 2.0.1), libgcc1 (>= 1:4.1.1), libstdc++6
         (>= 4.6)
Conflicts: thin-provisioning-tools
Description: Tools for handling thinly provisioned device-mapper meta-data
 This package contains tools to handle meta-data from the device-mapper thin
 target. This target allows the use of a single backing store for multple
 thinly provisioned volumes. Numerous snapshots can be taken from such volumes.
 The tools can check the meta-data for consistency, repair damaged information
 and dump or restore the meta-data in textual form.

I vaguely recall having to manually install that when I set up these LVM cache's.

Ben McCann (ben-mccann) wrote :

Installing the thin-provisioning-tools after booting the Live USB did not help. I still get the same error when checking the LVM partition status with 'pvs'.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in lvm2 (Ubuntu):
status: New → Confirmed
Guy Thouret (guy-thouret) wrote :

As I understand it, this bug affects any cached volumes created using the lvm2 package from 15.10 and then attempting to mount that volume under 16.04.

I have this issue on two machines. As a workaroundon one of the machines I was able to mount the volume after booting into a 15.10 live environment, converting the lvs back to uncached volumes, booting back to 16.04 and converting to cached volumes again.

Looking at the Fedora bug report this is caused by new cache metadata attributes missing in the metadata of volumes created using previous lvm2 version. A patch exists that adds default attributes to the cache metadata if they don't already exists and this will resolve the issue.

Link to patch:

I'll check upstream to see if a bugreport exists for this lvm2 package.

Guy Thouret (guy-thouret) wrote :

At the very least, there should be a caveat in the 16.04 upgrade notes to state that any cached lvm volumes should have the caches removed before the upgrade.

Guy Thouret (guy-thouret) wrote :

The patch referenced above does not have a related bug report in the upstream project.

The earliest release version the patch is included in is lvm2 2.02.135

summary: - LVM cannot mount LVM Cached Volumes from LMDE Betsy
+ LVM in 16.04 cannot mount LVM Cached Volumes created in 15.10
Guy Thouret (guy-thouret) wrote :

I've restored my system to a working state by building lvm2 from upstream source that includes the referenced patch and using the compiled lvm to do a config backup which added the missing metadata attributes then using native lvm (lvm2 2.02.133-1ubuntu7) to restore the config backup.

All is working now using lvm2 2.02.133-1ubuntu7 package.

guy@mediaserver:~/tools$ sudo lvs
  LV lv_cache has invalid cache's feature flag.
  LV lv_cache is missing cache policy name.
  Internal error: LV segments corrupted in lv_cache.

guy@mediaserver:~/tools$ sudo ./lvm vgcfgbackup -v
    Using volume group(s) on command line.
    Cache is missing cache policy, using mq.
    Cache is missing cache mode, using writethrough.
    Creating volume group backup "/etc/lvm/backup/vg1" (seqno 13).
  Volume group "vg1" successfully backed up.

guy@mediaserver:~/tools$ sudo vgcfgrestore vg1
  Restored volume group vg1

guy@mediaserver:~/tools$ sudo lvs
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  media vg1 Cwi---C--- 500.00g [lv_cache] [media_corig]

knz (knz) wrote :

The suggested workaround in did not work for me. What did work however was to build a modified lvm2 package using the aforementioned patch, then boot from external media, install the modified lvm2 on the externally booted ubuntu, then mount the system's main fs using the modified lvm, then install the modified lvm2 again this time on the system's fs using chroot.

This process is cumbersome and I wish ubuntu would include this patch by default.

Changed in lvm2 (Ubuntu):
importance: Undecided → High
Ben McCann (ben-mccann) wrote :

LVM caching enabled in Ubuntu 16.04 (or, to be completely honest, Linux Mint 18) is also incompatible with the older caching implementation in Linux Mint Debian Edition 2 (i.e. Debian Jessie). I hoped the following procedure would create an LVM cache that works for both Jessie and Xenial:

1) Under Jessie, remove the cache from the LVM volume.
2) Reboot into Xenial and add an LVM cache to the volume. (This works fine).
3) Reboot back to Jessie to use the LVM volume with caching created under Xenial.

My goal is to support dual boot between jessie and xenial with LVM caching enabled on my data volumes.

This doesn't work. The LVM volume comes up a disabled or offline under Jessie and there's no apparent way to bring it up without removing the cache. (I can easily recreate this on my scratch/test machine in case there's useful data to be gathered).

So, LVM caching between debian jessie and ubuntu xenial is neither forward or backwards compatible.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers