Azure provider: storage device names changed and wrong device mounted post-reboot

[Haw Loeung]

Hi,

Deploying an application to the Azure public cloud using Juju storage:

| $ jsft ubuntu-repository-cache | grep '^ubuntu-repository-cache'
| ubuntu-repository-cache unknown 2 ubuntu-repository-cache local 1 ubuntu exposed
| ubuntu-repository-cache/0 unknown idle 0 20.193.47.26 80/tcp
| ubuntu-repository-cache/1* unknown idle 1 20.53.72.175 80/tcp

| $ juju list-storage
| Unit Storage id Type Pool Size Status Message
| ubuntu-repository-cache/0 ubuntu-repository-cache/0 filesystem azure 100GiB attached
| ubuntu-repository-cache/1 ubuntu-repository-cache/1 filesystem azure 100GiB attached

Sadly, on rebooting one of the units, the device names changed where root/sdb was previously the root device, sda the ephemeral, and sdc the newly attached persistent volume:

| $ juju run --application ubuntu-repository-cache "df -h -t ext4"
| - Stdout: |
| Filesystem Size Used Avail Use% Mounted on
| /dev/root 31G 2.4G 29G 8% /
| /dev/sda1 98G 61M 93G 1% /mnt
| /dev/sdc1 98G 26G 68G 28% /srv/ubuntu-repository-cache
| UnitId: ubuntu-repository-cache/0
| - Stdout: |
| Filesystem Size Used Avail Use% Mounted on
| /dev/root 31G 2.4G 29G 8% /
| /dev/sda1 98G 61M 93G 1% /mnt
| /dev/sdc1 98G 26G 68G 28% /srv/ubuntu-repository-cache
| UnitId: ubuntu-repository-cache/1

It is now:

| $ sudo fdisk -l
| Disk /dev/sda: 100 GiB, 107374182400 bytes, 209715200 sectors
| Disk model: Virtual Disk
| Units: sectors of 1 * 512 = 512 bytes
| Sector size (logical/physical): 512 bytes / 4096 bytes
| I/O size (minimum/optimal): 4096 bytes / 4096 bytes
| Disklabel type: gpt
| Disk identifier: C986DF66-8602-400D-A26F-9C51B553A82A
|
| Device Start End Sectors Size Type
| /dev/sda1 2048 209715165 209713118 100G Linux filesystem
|
|
| Disk /dev/sdb: 100 GiB, 107374182400 bytes, 209715200 sectors
| Disk model: Virtual Disk
| Units: sectors of 1 * 512 = 512 bytes
| Sector size (logical/physical): 512 bytes / 4096 bytes
| I/O size (minimum/optimal): 4096 bytes / 4096 bytes
| Disklabel type: dos
| Disk identifier: 0xcb40b58b
|
| Device Boot Start End Sectors Size Id Type
| /dev/sdb1 128 209713151 209713024 100G 7 HPFS/NTFS/exFAT
|
|
| Disk /dev/sdc: 32 GiB, 34359738368 bytes, 67108864 sectors
| Disk model: Virtual Disk
| Units: sectors of 1 * 512 = 512 bytes
| Sector size (logical/physical): 512 bytes / 4096 bytes
| I/O size (minimum/optimal): 4096 bytes / 4096 bytes
| Disklabel type: gpt
| Disk identifier: 0BC6C56D-1FD3-4FD5-A89E-8840BADAC20D
|
| Device Start End Sectors Size Type
| /dev/sdc1 227328 67108830 66881503 31.9G Linux filesystem
| /dev/sdc14 2048 10239 8192 4M BIOS boot
| /dev/sdc15 10240 227327 217088 106M EFI System
|
| Partition table entries are not in disk order.

With /etc/fstab:

| ubuntu@machine-0:/srv/ubuntu-repository-cache$ cat /etc/fstab
| # CLOUD_IMG: This file was created/modified by the Cloud Image build process
| UUID=b193e0d2-0ae2-4a0f-9bf5-1e29bce9b644 / ext4 defaults,discard 0 0
| UUID=93D5-EE75 /boot/efi vfat defaults,discard 0 0
| /dev/disk/cloud/azure_resource-part1 /mnt auto defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 2
|
| /dev/sdc1 /srv/ubuntu-repository-cache ext4 rw,relatime 0 0

Perhaps the created /etc/fstab entry should use UUID?

This is with Juju 2.8.1.

[Haw Loeung (hloeung)]

FYI, nothing for the persistent volume under /dev/disk/cloud:

| ubuntu@machine-0:~$ ls /dev/disk/cloud/ -la
| total 0
| drwxr-xr-x 2 root root 160 Jul 14 22:12 .
| drwxr-xr-x 9 root root 180 Jul 14 22:12 ..
| lrwxrwxrwx 1 root root 9 Jul 14 22:12 azure_resource -> ../../sdb
| lrwxrwxrwx 1 root root 10 Jul 14 22:12 azure_resource-part1 -> ../../sdb1
| lrwxrwxrwx 1 root root 9 Jul 14 22:12 azure_root -> ../../sdc
| lrwxrwxrwx 1 root root 10 Jul 14 22:12 azure_root-part1 -> ../../sdc1
| lrwxrwxrwx 1 root root 11 Jul 14 22:12 azure_root-part14 -> ../../sdc14
| lrwxrwxrwx 1 root root 11 Jul 14 22:12 azure_root-part15 -> ../../sdc15
| ubuntu@machine-0:~$

[Haw Loeung (hloeung)]

As Ian pointed out on IRC, this has already been suggested in LP:1877380 so likely we can mark this as a duplicate or resolve the two together.