Comment 12 for bug 1965587

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

Root-cause: Focal's mkfs.xfs enables the reflink feature, not supported by CentOS
---

The default for 'mkfs.xfs -m reflink=0|1' changed from 0 in Bionic to 1 in Focal.
Per mkfs.xfs(8) manpages:

 - Bionic: [1]
 By default, mkfs.xfs __will not__ create reference count btrees
 and therefore __will not__ enable the reflink feature.

 - Focal: [2]
 By default, mkfs.xfs __will__ create reference count btrees
 and therefore __will__ enable the reflink feature.

 [1] http://manpages.ubuntu.com/manpages/bionic/en/man8/mkfs.xfs.8.html
 [2] http://manpages.ubuntu.com/manpages/focal/en/man8/mkfs.xfs.8.html

When the MAAS config includes a XFS partition, the curtin config runs 'mkfs.xfs'.
(the curtin config schema has 'storage/format/extra_options' for 'mkfs', however,
the MAAS API doesn't expose it; just 'mount'-time options [3,4]).

 [ 68.680243] cloud-init[1652]: Running command ['mkfs.xfs', '-f', '-L', '', '-m', 'uuid=6841d6ce-0658-4c5c-b293-ce1c2f69f4d5', '/dev/vda3'] with allowed return codes [0] (capture=True)

 [3] https://github.com/canonical/curtin/blob/master/doc/topics/storage.rst#format-command
 [4] https://maas.io/docs/api
     POST /MAAS/api/2.0/nodes/{system_id}/blockdevices/{device_id}/partition/{id}?op=format

So, the reflink feature bit (0x4) is set, and XFS in CentOS 7 doesn't know of it,
and the filesystem mount unit fails, bringing systemd to emergency/recovery mode.

 [ 0.000000] Linux version 3.10.0-1160.45.1.el7.x86_64 ...
 ...
 [ 3.704656] XFS (vda3): Superblock has unknown read-only compatible features (0x4) enabled.
 [ 3.705751] XFS (vda3): Attempted to mount read-only compatible filesystem read-write.
 [ 3.706749] XFS (vda3): Filesystem can only be safely mounted read only.
 [ 3.707921] XFS (vda3): SB validate failed with error -22.

From ubuntu-focal.git kernel source:

 451 #define XFS_SB_FEAT_RO_COMPAT_REFLINK (1 << 2) /* reflinked files */

The solution is to disable the reflink feature at mkfs.xfs time, with 'mkfs.xfs -m reflink=0'.

In order to do this in a os/release-dependent, not machine-dependent way, we can use a
curtin_userdata file that is specific to CentOS 7.