On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10 Buster), ds-identify fails to detect when a disk of a NoCloud datasource has a label written to the boot sector of the disk. Before util-linux-2.33-rc1, blkid showed "LABEL=cidata". With the change, blkid shows "LABEL_FATBOOT=cidata" (newly introduced / additional label).
Longer description:
I ran into this when using cloud-init together with Xen Orchestra v5.48 (Xen Orchestra is a management interface for xen; in my case xcp-ng v8.0.0). I created a VM template based on the recently released Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds-identify fails to detect the NoCloud datasource / virtual disk which Xen Orchestra generated (the disk is created with this code from https://github.com/natevw/fatfs. With an older Debian 8 (util-linux-2.25.0) based template, ds-identify detects the NoCloud datasource disk fine.
Likely explanation:
Xen Orchestra creates the NoCloud as a partition-less disk with a FAT16 filesystem which has the NoCloud user-data and meta-data files. The label "cidata" is written into the boot sector of the virtual disk. With the same disk, oder versions of blkid report "LABEL=cidata" whereas newer versions detect "LABEL_FATBOOT=cidata". The ds-identify shell script checks only for the presence of the field called "LABEL" and not for "LABEL_FATBOOT".
Relevant commit message from the util-linux-2.33-rc1 changelog (commit f0ca7e80d7a171701d0d04a3eae22d97f15d0683):
libblkid: vfat: Change parsing label in special cases
* Use only label from the root directory and do not fallback to the label
stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, Windows 98,
Windows XP and also Windows 10 behave. Moreover Windows XP and Windows 10
do not touch label in boot sector anymore, so removing FAT label on those
Windowses leads to having old label still stored in boot sector (which
MS-DOS and Windows fully ignore).
* Label entry "NO NAME" in root directory is treated as label "NO NAME"
instead of empty label. In root directory it has no special meaning.
String "NO NAME" has a special meaning (empty label) only for label
stored in boot sector.
* Label from the boot sector is now stored into LABEL_FATBOOT field. So if
there are applications which depends or needs to read this label, they
have ability.
After this change LABEL always correspondent to the label from the root
directory and LABEL_FATBOOT to the label stored in the boot sector. If some
of those labels is missing or is not present (e.g. "NO LABEL" in boot
sector) then particular field is not set.
Possible fix:
I did a trivial change of 2 lines to ds-identify to check for LABEL_FATBOOT after the check for LABEL. For me this solves the problem, as in: the cloud-init enabled VM boots up, ds-identify finds "LABEL_FATBOOT=cidata" and cloud-init correctly executes. In cases where both labels are written, the latter over-writes the former, which could be a theoretical problem if the values differ, but I am not sure how likely this case is.
Further debug information as requested by @rharper on IRC:
Original bug report with detailed description was created for Xen Orchestra here: https:/ /github. com/vatesfr/ xen-orchestra/ issues/ 4449
Brief description:
On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10 Buster), ds-identify fails to detect when a disk of a NoCloud datasource has a label written to the boot sector of the disk. Before util-linux- 2.33-rc1, blkid showed "LABEL=cidata". With the change, blkid shows "LABEL_ FATBOOT= cidata" (newly introduced / additional label).
Longer description:
I ran into this when using cloud-init together with Xen Orchestra v5.48 (Xen Orchestra is a management interface for xen; in my case xcp-ng v8.0.0). I created a VM template based on the recently released Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds-identify fails to detect the NoCloud datasource / virtual disk which Xen Orchestra generated (the disk is created with this code from https:/ /github. com/natevw/ fatfs. With an older Debian 8 (util-linux-2.25.0) based template, ds-identify detects the NoCloud datasource disk fine.
Likely explanation:
Xen Orchestra creates the NoCloud as a partition-less disk with a FAT16 filesystem which has the NoCloud user-data and meta-data files. The label "cidata" is written into the boot sector of the virtual disk. With the same disk, oder versions of blkid report "LABEL=cidata" whereas newer versions detect "LABEL_ FATBOOT= cidata" . The ds-identify shell script checks only for the presence of the field called "LABEL" and not for "LABEL_FATBOOT".
Relevant commit message from the util-linux-2.33-rc1 changelog (commit f0ca7e80d7a1717 01d0d04a3eae22d 97f15d0683) :
libblkid: vfat: Change parsing label in special cases
* Use only label from the root directory and do not fallback to the label
stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, Windows 98,
Windows XP and also Windows 10 behave. Moreover Windows XP and Windows 10
do not touch label in boot sector anymore, so removing FAT label on those
Windowses leads to having old label still stored in boot sector (which
MS-DOS and Windows fully ignore).
* Label entry "NO NAME" in root directory is treated as label "NO NAME"
instead of empty label. In root directory it has no special meaning.
String "NO NAME" has a special meaning (empty label) only for label
stored in boot sector.
* Label from the boot sector is now stored into LABEL_FATBOOT field. So if
there are applications which depends or needs to read this label, they
have ability.
After this change LABEL always correspondent to the label from the root
directory and LABEL_FATBOOT to the label stored in the boot sector. If some
of those labels is missing or is not present (e.g. "NO LABEL" in boot
sector) then particular field is not set.
Possible fix:
I did a trivial change of 2 lines to ds-identify to check for LABEL_FATBOOT after the check for LABEL. For me this solves the problem, as in: the cloud-init enabled VM boots up, ds-identify finds "LABEL_ FATBOOT= cidata" and cloud-init correctly executes. In cases where both labels are written, the latter over-writes the former, which could be a theoretical problem if the values differ, but I am not sure how likely this case is.
Further debug information as requested by @rharper on IRC:
- cloud-init.tar.gz (Debian 10 / ds-identify fail)
- Debian version:
debian@ cloudbuster: ~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
- util-linux version:
debian@ cloudbuster: ~$ sudo blkid -V
blkid from util-linux 2.33.1 (libblkid 2.33.1, 09-Jan-2019)
- blkid output:
debian@ cloudbuster: ~$ sudo blkid /dev/xvdb "cidata" UUID="355A-4FC2" TYPE="vfat"
/dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT=
- udevadm outout: cloudbuster: ~$ udevadm info --query=all /sys/class/ block/xvdb vbd-832/ block/xvdb uuid/355A- 4FC2 /devices/ vbd-832/ block/xvdb D=4239917 355A-4FC2 ENC=355A- 4FC2 filesystem /dev/disk/ by-uuid/ 355A-4FC2
debian@
P: /devices/
N: xvdb
L: 0
S: disk/by-
E: DEVPATH=
E: DEVNAME=/dev/xvdb
E: DEVTYPE=disk
E: MAJOR=202
E: MINOR=16
E: SUBSYSTEM=block
E: USEC_INITIALIZE
E: ID_FS_UUID=
E: ID_FS_UUID_
E: ID_FS_VERSION=FAT16
E: ID_FS_TYPE=vfat
E: ID_FS_USAGE=
E: DEVLINKS=
E: TAGS=:systemd:
# Some experiments:
- This is interesting - dosfslabel incorrectly reports the label, while blkid (above) clearly shows the field is empty / not set:
debian@ cloudbuster: ~$ sudo dosfslabel /dev/xvdb
cidata
- Here I am first setting the label with dosfslabel to see what happens and then check blkid again:
debian@ cloudbuster: ~$ sudo dosfslabel /dev/xvdb cidata
fatlabel: warning - lowercase labels might not work properly with DOS or Windows
debian@ cloudbuster: ~$ sudo blkid /dev/xvdb "cidata" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat"
/dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT=
# Now blkid reports both labels