I think the kernel side of it needs consideration but inherently persistent /dev/bcache<i> cannot be provided due to the fact that there is no deterministic enumeration and /dev/bcache<i> names cannot be changed afterwards (only symlinks can be created).
man 7 udev is clear about that:
"udev supplies the system software with device events, manages permissions of device nodes and may create additional symlinks in the /dev directory, or renames network interfaces. The kernel usually just assigns unpredictable device names based on the order of discovery."
bcache code uses alloc_disk from gendisk to get a struct gendisk which means no custom uevent variables on coldplug - it just gets whatever any block device gets (with exception of partitions).
hotplug uevents with necessary UUIDs only happen when init script runs in initramfs and are not handled as there is no udev to listen for those (they are just ignored) so no symlinks are ever set up.
Fixing the kernel side is not trivial as this would imply changing the DEVTYPE (userspace breakage) and adding custom uevents to that file - essentially rewriting alloc_disk.
So, given that this will realistically not land for 4.15 as there is no patch and the idea of changing uevent file is not perfect I think we should handle that from userspace.
We can read a bcache superblock of a backing device during cold-plug as we know that a device name will contain the word "bcache" and we can set up those symlinks with a small overhead of running several processes.
We just need a mapping from superblock UUID -> dname.
/dev/by-dname/<link> symlinks are respected by our user-space (MAAS API returns by-dname names and Juju storage uses that instead of /dev/<name>).
bcache-tools provide a way to see the "version" of a superblock (cache device or backing device) and are present in our cloud images.
A more robust way with superblock version parsing (we can use a python script fwiw).
(for i in `ls -1 /sys/block/bcache0/slaves` ; do sudo bcache-super-show /dev/$i ; done) | grep -Pzo '(?s)(?<=sb.version\t\t1).*?dev.uuid.*?\n' | grep -a uuid | cut -f 3
fb7c070c-5f96-4add-8565-1398bd831b1f
We could also use bcache label to hold a dname and use that instead (just need to set it on creation but UUID is way more reliable and we should use that).
That way we can solve this in a relatively cheap way by only modifying curtin to create UUID -> dname mappings and 69-bcache.rules from bcache-tools to respect those mappings on coldplug.
I think the kernel side of it needs consideration but inherently persistent /dev/bcache<i> cannot be provided due to the fact that there is no deterministic enumeration and /dev/bcache<i> names cannot be changed afterwards (only symlinks can be created).
man 7 udev is clear about that:
"udev supplies the system software with device events, manages permissions of device nodes and may create additional symlinks in the /dev directory, or renames network interfaces. The kernel usually just assigns unpredictable device names based on the order of discovery."
bcache code uses alloc_disk from gendisk to get a struct gendisk which means no custom uevent variables on coldplug - it just gets whatever any block device gets (with exception of partitions).
http:// elixir. free-electrons. com/linux/ v4.14/source/ drivers/ base/core. c#L865 (other settings set from bcache code and genhd.c added to uevent) elixir. free-electrons. com/linux/ v4.14/source/ block/genhd. c#L1234 (DEVTYPE = disk)
http://
http:// elixir. free-electrons. com/linux/ v4.14/source/ block/genhd. c#L1357 (alloc_disk -> alloc_disk_node -> device_initialize)
http:// elixir. free-electrons. com/linux/ v4.14/source/ drivers/ md/bcache/ super.c# L788 BCACHE_ MINORS) )) { // <--- alloc_disk d->disk- >disk_name, DISK_NAME_LEN, "bcache%i",
!(d->disk = alloc_disk(
snprintf(
minor / BCACHE_MINORS);
As a result, we have this on coldplug:
cat /sys/class/ block/bcache0/ uevent
MAJOR=252
MINOR=0
DEVNAME=bcache0
DEVTYPE=disk
hotplug uevents with necessary UUIDs only happen when init script runs in initramfs and are not handled as there is no udev to listen for those (they are just ignored) so no symlinks are ever set up.
Fixing the kernel side is not trivial as this would imply changing the DEVTYPE (userspace breakage) and adding custom uevents to that file - essentially rewriting alloc_disk.
So, given that this will realistically not land for 4.15 as there is no patch and the idea of changing uevent file is not perfect I think we should handle that from userspace.
We can read a bcache superblock of a backing device during cold-plug as we know that a device name will contain the word "bcache" and we can set up those symlinks with a small overhead of running several processes.
We just need a mapping from superblock UUID -> dname.
/dev/by- dname/< link> symlinks are respected by our user-space (MAAS API returns by-dname names and Juju storage uses that instead of /dev/<name>).
bcache-tools provide a way to see the "version" of a superblock (cache device or backing device) and are present in our cloud images.
The simplest way to do it is by using timestamp:
sudo bcache-super-show /dev/`ls -c -1t /sys/block/ bcache0/ slaves/ | tail -n1` | grep dev.uuid | cut -f 3 5f96-4add- 8565-1398bd831b 1f
fb7c070c-
A more robust way with superblock version parsing (we can use a python script fwiw).
(for i in `ls -1 /sys/block/ bcache0/ slaves` ; do sudo bcache-super-show /dev/$i ; done) | grep -Pzo '(?s)(? <=sb.version\ t\t1).* ?dev.uuid. *?\n' | grep -a uuid | cut -f 3 5f96-4add- 8565-1398bd831b 1f
fb7c070c-
We could also use bcache label to hold a dname and use that instead (just need to set it on creation but UUID is way more reliable and we should use that).
This is based on the following:
ubuntu@ maas-xenial6: ~$ sudo bcache-super-show /dev/`ls -c -1t /sys/block/ bcache0/ slaves/ | tail -n1`
sb.magic ok
sb.first_sector 8 [match]
sb.csum A61100F0EECCAB33 [match]
sb.version 1 [backing device]
dev.label (empty) 5f96-4add- 8565-1398bd831b 1f per_block 1 per_bucket 1024 first_sector 16 cache_state 1 [clean]
dev.uuid fb7c070c-
dev.sectors_
dev.sectors_
dev.data.
dev.data.cache_mode 1 [writeback]
dev.data.
cset.uuid 26bb5252- 2f4d-4a55- b4f9-8666cc1cd4 d6 maas-xenial6: ~$ sudo bcache-super-show /dev/sdb
ubuntu@
sb.magic ok
sb.first_sector 8 [match]
sb.csum 50DFF194DEF15F95 [match]
sb.version 3 [cache device]
dev.label (empty) 652b-4cc3- be28-d2b7b30689 ce per_block 1 per_bucket 1024 first_sector 1024 cache_sectors 134216704 total_sectors 134217728 replacement 0 [lru]
dev.uuid 00851121-
dev.sectors_
dev.sectors_
dev.cache.
dev.cache.
dev.cache.
dev.cache.ordered yes
dev.cache.discard no
dev.cache.pos 0
dev.cache.
cset.uuid 26bb5252- 2f4d-4a55- b4f9-8666cc1cd4 d6
That way we can solve this in a relatively cheap way by only modifying curtin to create UUID -> dname mappings and 69-bcache.rules from bcache-tools to respect those mappings on coldplug.
https:/ /git.launchpad. net/~usd- import- team/ubuntu/ +source/ bcache- tools/tree/ 69-bcache. rules