Description
===========
As per the runtime updates for the 2025.1 release [1], we're attempting to switch all test jobs to use Ubuntu Noble Numbat (24.04 LTS) instead of Ubuntu Jammy Jellyfish (22.04 LTS) as devstack test jobs currently do [2]. The installation and configuration of ZFSOnLinux is performed by manila's devstack plugin [3]. This isn't working on Ubuntu 24.04
Failure Log excerpt:
024-10-25 20:28:38.371953 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:752 : '[' manila.share.drivers.zfsonlinux.driver.ZFSonLinuxShareDriver == manila.share.drivers.zfsonlinux.driver.ZFSonLinuxShareDriver ']'
2024-10-25 20:28:38.375998 | controller | +++ /opt/stack/manila/devstack/plugin.sh:install_manila:753 : sudo zfs list
2024-10-25 20:28:38.387862 | controller | +++ /opt/stack/manila/devstack/plugin.sh:install_manila:753 : echo absent
2024-10-25 20:28:38.391438 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:753 : [[ absent == \a\b\s\e\n\t ]]
2024-10-25 20:28:38.394545 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:755 : is_ubuntu
2024-10-25 20:28:38.397697 | controller | ++ functions-common:is_ubuntu:556 : [[ -z deb ]]
2024-10-25 20:28:38.400588 | controller | ++ functions-common:is_ubuntu:559 : '[' deb = deb ']'
2024-10-25 20:28:38.403589 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:756 : sudo apt-get install -y zfsutils-linux
2024-10-25 20:28:38.435184 | controller | Reading package lists...
2024-10-25 20:28:38.688428 | controller | Building dependency tree...
2024-10-25 20:28:38.689311 | controller | Reading state information...
2024-10-25 20:28:39.011704 | controller | The following additional packages will be installed:
2024-10-25 20:28:39.012411 | controller | libnvpair3linux libuutil3linux libzfs4linux libzpool5linux
2024-10-25 20:28:39.013657 | controller | Suggested packages:
2024-10-25 20:28:39.013687 | controller | zfs-initramfs | zfs-dracut
2024-10-25 20:28:39.013706 | controller | Recommended packages:
2024-10-25 20:28:39.043123 | controller | zfs-zed
2024-10-25 20:28:39.043187 | controller | The following NEW packages will be installed:
2024-10-25 20:28:39.044138 | controller | libnvpair3linux libuutil3linux libzfs4linux libzpool5linux zfsutils-linux
2024-10-25 20:28:39.450946 | controller | 0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
2024-10-25 20:28:39.450993 | controller | Need to get 2,289 kB of archives.
2024-10-25 20:28:39.451015 | controller | After this operation, 7,116 kB of additional disk space will be used.
2024-10-25 20:28:39.703304 | controller | Get:1 https://mirror-int.iad.rax.opendev.org/ubuntu noble-updates/main amd64 libnvpair3linux amd64 2.2.2-0ubuntu9.1 [61.6 kB]
2024-10-25 20:28:39.703411 | controller | Get:2 https://mirror-int.iad.rax.opendev.org/ubuntu noble-updates/main amd64 libuutil3linux amd64 2.2.2-0ubuntu9.1 [52.7 kB]
2024-10-25 20:28:40.084431 | controller | Get:3 https://mirror-int.iad.rax.opendev.org/ubuntu noble-updates/main amd64 libzfs4linux amd64 2.2.2-0ubuntu9.1 [226 kB]
2024-10-25 20:28:41.439384 | controller | Get:4 https://mirror-int.iad.rax.opendev.org/ubuntu noble-updates/main amd64 libzpool5linux amd64 2.2.2-0ubuntu9.1 [1,397 kB]
2024-10-25 20:28:42.038676 | controller | Get:5 https://mirror-int.iad.rax.opendev.org/ubuntu noble-updates/main amd64 zfsutils-linux amd64 2.2.2-0ubuntu9.1 [551 kB]
2024-10-25 20:28:42.345098 | controller | debconf: delaying package configuration, since apt-utils is not installed
2024-10-25 20:28:42.382948 | controller | Fetched 2,289 kB in 3s (765 kB/s)
2024-10-25 20:28:42.413114 | controller | Selecting previously unselected package libnvpair3linux.
2024-10-25 20:28:42.450089 | controller | (Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 60270 files and directories currently installed.)
2024-10-25 20:28:42.452808 | controller | Preparing to unpack .../libnvpair3linux_2.2.2-0ubuntu9.1_amd64.deb ...
2024-10-25 20:28:42.456937 | controller | Unpacking libnvpair3linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.487962 | controller | Selecting previously unselected package libuutil3linux.
2024-10-25 20:28:42.495525 | controller | Preparing to unpack .../libuutil3linux_2.2.2-0ubuntu9.1_amd64.deb ...
2024-10-25 20:28:42.497795 | controller | Unpacking libuutil3linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.530025 | controller | Selecting previously unselected package libzfs4linux.
2024-10-25 20:28:42.534969 | controller | Preparing to unpack .../libzfs4linux_2.2.2-0ubuntu9.1_amd64.deb ...
2024-10-25 20:28:42.536695 | controller | Unpacking libzfs4linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.570178 | controller | Selecting previously unselected package libzpool5linux.
2024-10-25 20:28:42.577925 | controller | Preparing to unpack .../libzpool5linux_2.2.2-0ubuntu9.1_amd64.deb ...
2024-10-25 20:28:42.579690 | controller | Unpacking libzpool5linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.650030 | controller | Selecting previously unselected package zfsutils-linux.
2024-10-25 20:28:42.658846 | controller | Preparing to unpack .../zfsutils-linux_2.2.2-0ubuntu9.1_amd64.deb ...
2024-10-25 20:28:42.662682 | controller | Unpacking zfsutils-linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.793113 | controller | Setting up libnvpair3linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.798626 | controller | Setting up libuutil3linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.804070 | controller | Setting up libzpool5linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.809411 | controller | Setting up libzfs4linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.814624 | controller | Setting up zfsutils-linux (2.2.2-0ubuntu9.1) ...
2024-10-25 20:28:42.838913 | controller | insmod /lib/modules/6.8.0-47-generic/kernel/zfs/spl.ko.zst
2024-10-25 20:28:42.851186 | controller | insmod /lib/modules/6.8.0-47-generic/kernel/zfs/zfs.ko.zst
2024-10-25 20:28:47.599118 | controller | Created symlink /etc/systemd/system/zfs-import.target.wants/zfs-import-cache.service → /usr/lib/systemd/system/zfs-import-cache.service.
2024-10-25 20:28:48.044225 | controller | Created symlink /etc/systemd/system/zfs.target.wants/zfs-import.target → /usr/lib/systemd/system/zfs-import.target.
2024-10-25 20:28:48.271055 | controller | Created symlink /etc/systemd/system/zfs-mount.service.wants/zfs-load-module.service → /usr/lib/systemd/system/zfs-load-module.service.
2024-10-25 20:28:48.490545 | controller | Created symlink /etc/systemd/system/zfs.target.wants/zfs-load-module.service → /usr/lib/systemd/system/zfs-load-module.service.
2024-10-25 20:28:48.490608 | controller | Created symlink /etc/systemd/system/zfs.target.wants/zfs-mount.service → /usr/lib/systemd/system/zfs-mount.service.
2024-10-25 20:28:48.712164 | controller | Created symlink /etc/systemd/system/zfs.target.wants/zfs-share.service → /usr/lib/systemd/system/zfs-share.service.
2024-10-25 20:28:48.934145 | controller | Created symlink /etc/systemd/system/zfs-volumes.target.wants/zfs-volume-wait.service → /usr/lib/systemd/system/zfs-volume-wait.service.
2024-10-25 20:28:49.159146 | controller | Created symlink /etc/systemd/system/zfs.target.wants/zfs-volumes.target → /usr/lib/systemd/system/zfs-volumes.target.
2024-10-25 20:28:49.383025 | controller | Created symlink /etc/systemd/system/multi-user.target.wants/zfs.target → /usr/lib/systemd/system/zfs.target.
2024-10-25 20:28:49.852595 | controller | zfs-import-scan.service is a disabled or a static unit, not starting it.
2024-10-25 20:28:50.011733 | controller | Processing triggers for man-db (2.12.0-4build2) ...
2024-10-25 20:28:51.375322 | controller | Processing triggers for libc-bin (2.39-0ubuntu8.3) ...
2024-10-25 20:28:51.569511 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:764 : sudo modprobe zfs
2024-10-25 20:28:51.587652 | controller | ++ /opt/stack/manila/devstack/plugin.sh:install_manila:765 : sudo modprobe zpool
2024-10-25 20:28:51.598989 | controller | modprobe: FATAL: Module zpool not found in directory /lib/modules/6.8.0-47-generic
2024-10-25 20:28:51.604914 | controller | + /opt/stack/manila/devstack/plugin.sh:install_manila:1 : exit_trap
2024-10-25 20:28:51.608048 | controller | + ./stack.sh:exit_trap:540 : local r=1
2024-10-25 20:28:51.611650 | controller | ++ ./stack.sh:exit_trap:541 : jobs -p
2024-10-25 20:28:51.615828 | controller | + ./stack.sh:exit_trap:541 : jobs=40178
2024-10-25 20:28:51.618928 | controller | + ./stack.sh:exit_trap:544 : [[ -n 40178 ]]
2024-10-25 20:28:51.621939 | controller | + ./stack.sh:exit_trap:544 : [[ -n /opt/stack/logs/devstacklog.txt.2024-10-25-202121 ]]
2024-10-25 20:28:51.624974 | controller | + ./stack.sh:exit_trap:544 : [[ True == \T\r\u\e ]]
2024-10-25 20:28:51.627928 | controller | + ./stack.sh:exit_trap:545 : echo 'exit_trap: cleaning up child processes'
2024-10-25 20:28:51.627960 | controller | exit_trap: cleaning up child processes
2024-10-25 20:28:51.631525 | controller | + ./stack.sh:exit_trap:546 : kill 40178
2024-10-25 20:28:51.634163 | controller | + ./stack.sh:exit_trap:550 : '[' -f '' ']'
2024-10-25 20:28:51.638038 | controller | + ./stack.sh:exit_trap:555 : kill_spinner
2024-10-25 20:28:51.641342 | controller | + ./stack.sh:kill_spinner:450 : '[' '!' -z '' ']'
2024-10-25 20:28:51.645097 | controller | + ./stack.sh:exit_trap:557 : [[ 1 -ne 0 ]]
2024-10-25 20:28:51.648613 | controller | + ./stack.sh:exit_trap:558 : echo 'Error on exit'
2024-10-25 20:28:51.649279 | controller | Error on exit
2024-10-25 20:28:51.652942 | controller | + ./stack.sh:exit_trap:560 : type -p generate-subunit
2024-10-25 20:28:51.656934 | controller | + ./stack.sh:exit_trap:561 : generate-subunit 1729887677 454 fail
2024-10-25 20:28:51.903478 | controller | + ./stack.sh:exit_trap:563 : [[ -z /opt/stack/logs ]]
2024-10-25 20:28:51.906408 | controller | + ./stack.sh:exit_trap:566 : /opt/stack/data/venv/bin/python3 /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs
2024-10-25 20:28:53.017217 | controller | + ./stack.sh:exit_trap:575 : exit 1
2024-10-25 20:28:53.018346 | controller | *** FINISHED ***
2024-10-25 20:29:03.380628 | controller | ERROR
2024-10-25 20:29:03.380905 | controller | {
2024-10-25 20:29:03.380975 | controller | "delta": "0:07:45.440085",
2024-10-25 20:29:03.381053 | controller | "end": "2024-10-25 20:29:03.021014",
2024-10-25 20:29:03.381153 | controller | "msg": "non-zero return code",
2024-10-25 20:29:03.381239 | controller | "rc": 1,
2024-10-25 20:29:03.381299 | controller | "start": "2024-10-25 20:21:17.580929"
2024-10-25 20:29:03.381383 | controller | }
failure
[1] https://governance.openstack.org/tc/reference/runtimes/2025.1.html
[2] https://review.opendev.org/c/openstack/manila-tempest-plugin/+/932956
[3] https://github.com/openstack/manila/blob/8f69ccbb2f9339c5d3ae9e609961456f3d4a5153/devstack/plugin.sh#L752-L767
manila-