Comment 4 for bug 1780332

With the error in vault, we initialized vault and unsealed. vault showed it is in ready status. The error from juju log show uuid is d045f2c8-b705-4113-9291-2ef203600fb0 but this uuid does not exist. Where does the charm gets d045f2c8-b705-4113-9291-2ef203600fb0 from?

It seems that with thedac's proposed fix, it did have the correct symlink for uuid 4c151bc2-8ad8-4c18-b4e8-58b22a66a6b2 but when the charm ran, it used uuid d045f2c8-b705-4113-9291-2ef203600fb0.

/etc/udev/rules.d# cat bcache1.rules
SUBSYSTEM=="block", ACTION=="add|change", ENV{CACHED_UUID}=="4c151bc2-8ad8-4c18-b4e8-58b22a66a6b2", SYMLINK+="disk/by-dname/bcache1"

hook failed: "secrets-storage-relation-changed"

2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed vaultlocker: Command '['cryptsetup', '--batch-mode', '--key-file', '-', 'open', 'UUID=d045f2c8-b705-4113-9291-2ef203600fb0', 'crypt-d045f2c8-b705-4113-9291-2ef203600fb0', '--type', 'luks']' returned non-zero exit status 4
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed Traceback (most recent call last):
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/var/lib/juju/agents/unit-nova-compute-kvm-5/charm/hooks/secrets-storage-relation-changed", line 579, in <module>
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed main()
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/var/lib/juju/agents/unit-nova-compute-kvm-5/charm/hooks/secrets-storage-relation-changed", line 572, in main
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed hooks.execute(sys.argv)
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/var/lib/juju/agents/unit-nova-compute-kvm-5/charm/hooks/charmhelpers/core/hookenv.py", line 823, in execute
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed self._hooks[hook_name]()
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/var/lib/juju/agents/unit-nova-compute-kvm-5/charm/hooks/secrets-storage-relation-changed", line 556, in secrets_storage_changed
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed configure_local_ephemeral_storage()
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/var/lib/juju/agents/unit-nova-compute-kvm-5/charm/hooks/nova_compute_utils.py", line 883, in configure_local_ephemeral_storage
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed dev])
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed raise CalledProcessError(retcode, cmd)
2018-07-06 07:54:37 DEBUG secrets-storage-relation-changed subprocess.CalledProcessError: Command '['vaultlocker', 'encrypt', '--uuid', 'd045f2c8-b705-4113-9291-2ef203600fb0', '/dev/disk/by-dname/bcache1']' returned non-zero exit status 1

===================

# vaultlocker encrypt --uuid d045f2c8-b705-4113-9291-2ef203600fb0 /dev/disk/by-dname/bcache1
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.216.2.23
DEBUG:urllib3.connectionpool:http://10.216.2.23:8200 "POST /v1/auth/approle/login HTTP/1.1" 200 437
INFO:vaultlocker.dmcrypt:LUKS formatting /dev/disk/by-dname/bcache1 using UUID:d045f2c8-b705-4113-9291-2ef203600fb0
INFO:vaultlocker.dmcrypt:udevadm settle /dev/disk/by-uuid/d045f2c8-b705-4113-9291-2ef203600fb0
DEBUG:urllib3.connectionpool:http://10.216.2.23:8200 "PUT /v1/charm-vaultlocker/DCS1-CLP-NOD17/d045f2c8-b705-4113-9291-2ef203600fb0 HTTP/1.1" 204 0
DEBUG:urllib3.connectionpool:http://10.216.2.23:8200 "GET /v1/charm-vaultlocker/DCS1-CLP-NOD17/d045f2c8-b705-4113-9291-2ef203600fb0 HTTP/1.1" 200 866
INFO:vaultlocker.dmcrypt:LUKS opening d045f2c8-b705-4113-9291-2ef203600fb0
Device /dev/disk/by-uuid/d045f2c8-b705-4113-9291-2ef203600fb0 doesn't exist or access denied.
vaultlocker: Command '['cryptsetup', '--batch-mode', '--key-file', '-', 'open', 'UUID=d045f2c8-b705-4113-9291-2ef203600fb0', 'crypt-d045f2c8-b705-4113-9291-2ef203600fb0', '--type', 'luks']' returned non-zero exit status 4

# cryptsetup --batch-mode --key-file - open UUID=d045f2c8-b705-4113-9291-2ef203600fb0 crypt-d045f2c8-b705-4113-9291-2ef203600fb0 --type luks
Device /dev/disk/by-uuid/d045f2c8-b705-4113-9291-2ef203600fb0 doesn't exist or access denied.

/dev/bcache# ls
by-label by-uuid
root@DCS1-CLP-NOD17:/dev/bcache# ls -al /dev/disk/by-uuid/*
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/05EC-7CF7 -> ../../sda1
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/2d8a4580-df8a-4e79-9189-8931f66dc978 -> ../../sdf
lrwxrwxrwx 1 root root 15 Jul 5 22:52 /dev/disk/by-uuid/38be3b4b-90dd-4bc3-8030-668042b4d572 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/4c151bc2-8ad8-4c18-b4e8-58b22a66a6b2 -> ../../sda4
lrwxrwxrwx 1 root root 15 Jul 5 22:52 /dev/disk/by-uuid/576c447b-1e76-438a-8158-0b316e4f5c12 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/58ca75fe-0bf2-4543-9a7e-151414193814 -> ../../sdd
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/8c7cb4cb-0e29-44e3-87ab-339c64400b9c -> ../../sdc
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/b1940fc7-80aa-49cc-b528-d46dc05fe7b5 -> ../../sda2
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/b454f3b4-b17d-4ba8-b13a-001b9aedfbbe -> ../../sdg
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/cd955f04-0cbe-4f66-8ba9-198d3058f098 -> ../../sde
lrwxrwxrwx 1 root root 13 Jul 5 22:52 /dev/disk/by-uuid/d0fbbf40-5a04-4152-9937-a2164c95bb8e -> ../../bcache7
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/fac76349-eab1-4574-9fda-455126075b7f -> ../../sdb
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/fd6432a5-e436-450c-a240-3098024351d5 -> ../../sda3

==================

# cryptsetup luksDump /dev/disk/by-dname/bcache1
LUKS header information for /dev/disk/by-dname/bcache1

Version: 1
Cipher name: aes
Cipher mode: xts-plain64
Hash spec: sha1
Payload offset: 4096
MK bits: 256
MK digest: a8 2c 89 2f 32 a3 30 99 ea 10 15 89 af 4b 24 e7 cd 07 4d a6
MK salt: 07 5c 0f 31 e2 e8 17 3b ca 95 10 33 b6 aa 17 c5
                cc df 46 eb 99 1b ad b5 72 18 ff b5 79 b1 b4 64
MK iterations: 142125
UUID: e466562f-50f8-4b70-8842-9723f41be266

Key Slot 0: ENABLED
 Iterations: 575280
 Salt: 21 b4 37 78 d3 47 21 d4 b2 3e 96 33 e5 30 fe 67
                        af 7f 65 d6 57 da b6 69 bb d2 71 77 a2 92 09 58
 Key material offset: 8
 AF stripes: 4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

=====================

# ls -al /dev/disk/by-uuid/*
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/05EC-7CF7 -> ../../sda1
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/2d8a4580-df8a-4e79-9189-8931f66dc978 -> ../../sdf
lrwxrwxrwx 1 root root 15 Jul 5 22:52 /dev/disk/by-uuid/38be3b4b-90dd-4bc3-8030-668042b4d572 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/4c151bc2-8ad8-4c18-b4e8-58b22a66a6b2 -> ../../sda4
lrwxrwxrwx 1 root root 15 Jul 5 22:52 /dev/disk/by-uuid/576c447b-1e76-438a-8158-0b316e4f5c12 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/58ca75fe-0bf2-4543-9a7e-151414193814 -> ../../sdd
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/8c7cb4cb-0e29-44e3-87ab-339c64400b9c -> ../../sdc
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/b1940fc7-80aa-49cc-b528-d46dc05fe7b5 -> ../../sda2
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/b454f3b4-b17d-4ba8-b13a-001b9aedfbbe -> ../../sdg
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/cd955f04-0cbe-4f66-8ba9-198d3058f098 -> ../../sde
lrwxrwxrwx 1 root root 13 Jul 5 22:52 /dev/disk/by-uuid/d0fbbf40-5a04-4152-9937-a2164c95bb8e -> ../../bcache7
lrwxrwxrwx 1 root root 9 Jul 5 22:52 /dev/disk/by-uuid/fac76349-eab1-4574-9fda-455126075b7f -> ../../sdb
lrwxrwxrwx 1 root root 10 Jul 5 22:52 /dev/disk/by-uuid/fd6432a5-e436-450c-a240-3098024351d5 -> ../../sda3

===========

/etc/udev/rules.d# cat bcache1.rules
SUBSYSTEM=="block", ACTION=="add|change", ENV{CACHED_UUID}=="4c151bc2-8ad8-4c18-b4e8-58b22a66a6b2", SYMLINK+="disk/by-dname/bcache1"