cloud-init

LXD module installs the wrong ZFS package if it's missing

Bug #1799779 reported by Michael Skalka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Unassigned
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Medium
Unassigned
Bionic
Fix Committed
Medium
Unassigned
Cosmic
Fix Committed
Medium
Unassigned
Disco
Fix Released
Medium
Unassigned

Bug Description

When using the LXD module cloud-init will attempt to install ZFS if it does not exist on the target system. However instead of installing the `zfsutils-linux` package it attempts to install `zfs` resulting in an error.

This was captured from a MAAS deployed server however the bug is platform agnostic.

###
ubuntu@node10ob68:~$ cloud-init --version
/usr/bin/cloud-init 18.3-9-g2e62cb8a-0ubuntu1~18.04.2

###
less /var/log/cloud-init.log
...
2018-10-24 19:23:54,255 - util.py[DEBUG]: apt-install [eatmydata apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install zfs] took 0.302 seconds
2018-10-24 19:23:54,255 - cc_lxd.py[WARNING]: failed to install packages ['zfs']: Unexpected error while running command.
Command: ['eatmydata', 'apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'install', 'zfs']
Exit code: 100
...

Related branches

~chad.smith/cloud-init:ubuntu/bionic
Server Team CI bot: Approve (continuous-integration)
cloud-init Commiters: Pending requested
Diff: 7789 lines (+3838/-768)
98 files modified
ChangeLog (+54/-0)
HACKING.rst (+2/-2)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/cloud_id.py (+90/-0)
cloudinit/cmd/devel/logs.py (+23/-8)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/devel/render.py (+24/-11)
cloudinit/cmd/devel/tests/test_logs.py (+37/-6)
cloudinit/cmd/devel/tests/test_render.py (+44/-1)
cloudinit/cmd/main.py (+4/-16)
cloudinit/cmd/query.py (+24/-12)
cloudinit/cmd/tests/test_cloud_id.py (+127/-0)
cloudinit/cmd/tests/test_query.py (+71/-5)
cloudinit/config/cc_disk_setup.py (+1/-1)
cloudinit/config/cc_lxd.py (+1/-1)
cloudinit/config/cc_resizefs.py (+7/-0)
cloudinit/config/cc_set_passwords.py (+1/-1)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/config/tests/test_set_passwords.py (+40/-0)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/handlers/jinja_template.py (+9/-1)
cloudinit/net/__init__.py (+38/-4)
cloudinit/net/dhcp.py (+76/-25)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+61/-5)
cloudinit/net/tests/test_dhcp.py (+47/-4)
cloudinit/net/tests/test_init.py (+51/-1)
cloudinit/sources/DataSourceAliYun.py (+5/-15)
cloudinit/sources/DataSourceAltCloud.py (+22/-11)
cloudinit/sources/DataSourceAzure.py (+82/-31)
cloudinit/sources/DataSourceBigstep.py (+4/-0)
cloudinit/sources/DataSourceCloudSigma.py (+5/-1)
cloudinit/sources/DataSourceConfigDrive.py (+12/-0)
cloudinit/sources/DataSourceEc2.py (+59/-56)
cloudinit/sources/DataSourceIBMCloud.py (+4/-0)
cloudinit/sources/DataSourceMAAS.py (+4/-0)
cloudinit/sources/DataSourceNoCloud.py (+52/-1)
cloudinit/sources/DataSourceNone.py (+4/-0)
cloudinit/sources/DataSourceOVF.py (+36/-26)
cloudinit/sources/DataSourceOpenNebula.py (+9/-1)
cloudinit/sources/DataSourceOracle.py (+4/-0)
cloudinit/sources/DataSourceScaleway.py (+10/-1)
cloudinit/sources/DataSourceSmartOS.py (+3/-0)
cloudinit/sources/__init__.py (+104/-21)
cloudinit/sources/helpers/netlink.py (+250/-0)
cloudinit/sources/helpers/tests/test_netlink.py (+373/-0)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/sources/tests/test_init.py (+83/-3)
cloudinit/sources/tests/test_oracle.py (+8/-0)
cloudinit/temp_utils.py (+2/-2)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/tests/test_temp_utils.py (+17/-1)
cloudinit/tests/test_url_helper.py (+24/-1)
cloudinit/tests/test_util.py (+82/-17)
cloudinit/url_helper.py (+25/-6)
cloudinit/util.py (+25/-3)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+75/-0)
doc/rtd/topics/datasources.rst (+60/-1)
doc/rtd/topics/datasources/azure.rst (+65/-38)
doc/rtd/topics/instancedata.rst (+137/-46)
doc/rtd/topics/network-config-format-v1.rst (+1/-1)
packages/redhat/cloud-init.spec.in (+1/-0)
packages/suse/cloud-init.spec.in (+1/-0)
setup.py (+2/-1)
systemd/cloud-init.service.tmpl (+1/-2)
templates/sources.list.ubuntu.tmpl (+17/-17)
tests/cloud_tests/releases.yaml (+16/-0)
tests/cloud_tests/testcases/base.py (+15/-3)
tests/cloud_tests/testcases/modules/apt_configure_primary.py (+9/-5)
tests/cloud_tests/testcases/modules/apt_configure_primary.yaml (+0/-7)
tests/unittests/test_builtin_handlers.py (+25/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_aliyun.py (+4/-0)
tests/unittests/test_datasource/test_altcloud.py (+67/-51)
tests/unittests/test_datasource/test_azure.py (+262/-79)
tests/unittests/test_datasource/test_cloudsigma.py (+6/-0)
tests/unittests/test_datasource/test_configdrive.py (+3/-0)
tests/unittests/test_datasource/test_ec2.py (+37/-23)
tests/unittests/test_datasource/test_ibmcloud.py (+39/-1)
tests/unittests/test_datasource/test_nocloud.py (+98/-41)
tests/unittests/test_datasource/test_opennebula.py (+4/-0)
tests/unittests/test_datasource/test_ovf.py (+119/-39)
tests/unittests/test_datasource/test_scaleway.py (+72/-4)
tests/unittests/test_datasource/test_smartos.py (+7/-0)
tests/unittests/test_ds_identify.py (+16/-1)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_resizefs.py (+42/-10)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+137/-6)
tests/unittests/test_util.py (+6/-0)
tests/unittests/test_vmware_config_file.py (+52/-6)
tools/ds-identify (+32/-6)
tools/run-container (+1/-0)
tox.ini (+2/-2)
udev/66-azure-ephemeral.rules (+17/-1)
~chad.smith/cloud-init:ubuntu/cosmic
Server Team CI bot: Approve (continuous-integration)
cloud-init Commiters: Pending requested
Diff: 5357 lines (+2757/-473)
69 files modified
ChangeLog (+54/-0)
HACKING.rst (+2/-2)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/devel/logs.py (+23/-8)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/devel/render.py (+24/-11)
cloudinit/cmd/devel/tests/test_logs.py (+37/-6)
cloudinit/cmd/devel/tests/test_render.py (+44/-1)
cloudinit/cmd/main.py (+4/-16)
cloudinit/cmd/query.py (+24/-12)
cloudinit/cmd/tests/test_query.py (+71/-5)
cloudinit/config/cc_disk_setup.py (+1/-1)
cloudinit/config/cc_lxd.py (+1/-1)
cloudinit/config/cc_resizefs.py (+7/-0)
cloudinit/config/cc_set_passwords.py (+1/-1)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/config/tests/test_set_passwords.py (+40/-0)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/handlers/jinja_template.py (+9/-1)
cloudinit/net/__init__.py (+34/-2)
cloudinit/net/dhcp.py (+76/-25)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+61/-5)
cloudinit/net/tests/test_dhcp.py (+47/-4)
cloudinit/net/tests/test_init.py (+51/-1)
cloudinit/sources/DataSourceAzure.py (+74/-31)
cloudinit/sources/DataSourceNoCloud.py (+31/-1)
cloudinit/sources/DataSourceOVF.py (+30/-26)
cloudinit/sources/DataSourceOpenNebula.py (+1/-1)
cloudinit/sources/DataSourceScaleway.py (+10/-1)
cloudinit/sources/helpers/netlink.py (+250/-0)
cloudinit/sources/helpers/tests/test_netlink.py (+373/-0)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/temp_utils.py (+2/-2)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/tests/test_temp_utils.py (+17/-1)
cloudinit/tests/test_url_helper.py (+24/-1)
cloudinit/tests/test_util.py (+66/-17)
cloudinit/url_helper.py (+25/-6)
cloudinit/util.py (+20/-3)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+70/-0)
doc/rtd/topics/datasources.rst (+60/-1)
doc/rtd/topics/datasources/azure.rst (+65/-38)
doc/rtd/topics/network-config-format-v1.rst (+1/-1)
packages/redhat/cloud-init.spec.in (+1/-0)
packages/suse/cloud-init.spec.in (+1/-0)
systemd/cloud-init.service.tmpl (+1/-2)
tests/cloud_tests/releases.yaml (+16/-0)
tests/unittests/test_builtin_handlers.py (+25/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_azure.py (+201/-34)
tests/unittests/test_datasource/test_ec2.py (+24/-16)
tests/unittests/test_datasource/test_nocloud.py (+66/-34)
tests/unittests/test_datasource/test_ovf.py (+79/-43)
tests/unittests/test_datasource/test_scaleway.py (+72/-4)
tests/unittests/test_ds_identify.py (+16/-1)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_resizefs.py (+42/-10)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+123/-6)
tests/unittests/test_util.py (+6/-0)
tests/unittests/test_vmware_config_file.py (+52/-6)
tools/ds-identify (+32/-6)
tools/run-container (+1/-0)
tox.ini (+2/-2)
udev/66-azure-ephemeral.rules (+17/-1)
~johnsonshi/cloud-init:zfs-package-fix
Server Team CI bot: Approve (continuous-integration)
Chad Smith: Approve
Ryan Harper: Pending requested
Scott Moser: Pending requested
Diff: 26 lines (+2/-2)
2 files modified
cloudinit/config/cc_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
~mskalka/cloud-init:zfs-package-fix
Rejected for merging into cloud-init:master
cloud-init Commiters: Pending requested
Diff: 13 lines (+1/-1)
1 file modified
cloudinit/config/cc_lxd.py (+1/-1)
Revision history for this message
Scott Moser (smoser) wrote :

marking this as 'fix-released' on xenial, it is not a problem there as there is a 'zfs' (virtual) package

Changed in cloud-init:
importance: Undecided → Medium
status: New → Confirmed
Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Medium
status: New → Fix Released
Changed in cloud-init (Ubuntu Bionic):
status: New → Confirmed
importance: Undecided → Medium
Johnson Shi (johnsonshi)
Changed in cloud-init:
assignee: nobody → Johnson Shi (johnsonshi)
status: Confirmed → In Progress
Johnson Shi (johnsonshi)
Changed in cloud-init (Ubuntu Bionic):
assignee: nobody → Johnson Shi (johnsonshi)
Revision history for this message
Ryan Harper (raharper) wrote :
Download full text (4.6 KiB)

Looking closer at Ubuntu releases:

------
Xenial
------

'zfs' meta package is present, and wants to install:

Note, selecting 'zfsutils-linux' instead of 'zfs'
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  binutils cpp cpp-5 dkms fakeroot gcc gcc-5 libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libfakeroot libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0
  libmpc3 libmpx0 libnvpair1linux libquadmath0 libtsan0 libubsan0 libuutil1linux libzfs2linux libzpool2linux linux-libc-dev make manpages-dev spl spl-dkms zfs-dkms zfs-doc
  zfs-zed
Suggested packages:
  binutils-doc cpp-doc gcc-5-locales gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg
  libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg glibc-doc make-doc default-mta | mail-transport-agent
  samba-common-bin nfs-kernel-server zfs-initramfs
The following NEW packages will be installed:
  binutils cpp cpp-5 dkms fakeroot gcc gcc-5 libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libfakeroot libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0
  libmpc3 libmpx0 libnvpair1linux libquadmath0 libtsan0 libubsan0 libuutil1linux libzfs2linux libzpool2linux linux-libc-dev make manpages-dev spl spl-dkms zfs-dkms zfs-doc
  zfs-zed zfsutils-linux

------
bionic
------

No 'zfs' meta package present, 'zfsutils-linux' offers to install

The following additional packages will be installed:
  libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-zed
Suggested packages:
  zfs-modules | zfs-dkms nfs-kernel-server samba-common-bin zfs-initramfs | zfs-dracut
The following NEW packages will be installed:
  libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-zed zfsutils-linux

------
cosmic
------

No 'zfs' meta package present, but has a reference to it in the archive.

$ sudo apt install zfs
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package zfs is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'zfs' has no installation candidate

'zfsutils-linux' offers:

$ sudo apt install zfsutils-linux
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  grub-common grub-gfxpayload-lists grub-pc grub-pc-bin grub2-common libnvpair1linux libuutil1linux libzfs2linux libzpool2linux linux-image-unsigned-4.18.0-13-lowlatency
  linux-modules-4.18.0-13-lowlatency os-prober zfs-zed
Suggested packages:
  multiboot-doc grub-emu xorriso desktop-base fdutils linux-doc-4.18.0 | linux-source-4.18.0 linux-tools linux-headers-4.18.0-13-lowlatency nfs-kernel-server
  samba-common-bin zfs-initramfs | zfs-dracut
The following NEW packages will be installed:
  grub-common grub-gfxpayload-lists grub-pc grub-pc-bin grub2-common libnvpair1linux libuutil...

Read more...

Revision history for this message
Johnson Shi (johnsonshi) wrote :

Fix is awaiting approval in my branch.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

This bug is fixed with commit c283321b to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=c283321b

Changed in cloud-init:
status: In Progress → Fix Committed
Johnson Shi (johnsonshi)
Changed in cloud-init (Ubuntu Bionic):
status: Confirmed → Fix Committed
Johnson Shi (johnsonshi)
Changed in cloud-init (Ubuntu Cosmic):
status: Confirmed → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 18.5-17-gd1a2fe73-0ubuntu1

---------------
cloud-init (18.5-17-gd1a2fe73-0ubuntu1) disco; urgency=medium

  * New upstream snapshot.
    - opennebula: exclude EPOCHREALTIME as known bash env variable with a
      delta (LP: #1813383)
    - tox: fix disco httpretty dependencies for py37 (LP: #1813361)
    - run-container: uncomment baseurl in yum.repos.d/*.repo when using a
      proxy [Paride Legovini]
    - lxd: install zfs-linux instead of zfs meta package
      [Johnson Shi] (LP: #1799779)
    - net/sysconfig: do not write a resolv.conf file with only the header.
      [Robert Schweikert]
    - net: Make sysconfig renderer compatible with Network Manager.
      [Eduardo Otubo]
    - cc_set_passwords: Fix regex when parsing hashed passwords
      [Marlin Cremers] (LP: #1811446)
    - net: Wait for dhclient to daemonize before reading lease file
      [Jason Zions] (LP: #1794399)
    - [Azure] Increase retries when talking to Wireserver during metadata walk
      [Jason Zions]
    - Add documentation on adding a datasource.
    - doc: clean up some datasource documentation.
    - ds-identify: fix wrong variable name in ovf_vmware_transport_guestinfo.
    - Scaleway: Support ssh keys provided inside an instance tag. [PORTE Loïc]
    - OVF: simplify expected return values of transport functions.
    - Vmware: Add support for the com.vmware.guestInfo OVF transport.
      (LP: #1807466)
    - HACKING.rst: change contact info to Josh Powers

 -- Chad Smith <email address hidden> Sat, 26 Jan 2019 08:27:35 -0700

Changed in cloud-init (Ubuntu Disco):
status: Confirmed → Fix Released
Revision history for this message
Chad Smith (chad.smith) wrote : Fixed in cloud-init version 19.1.

This bug is believed to be fixed in cloud-init in version 19.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Johnson Shi (johnsonshi)
Changed in cloud-init (Ubuntu Bionic):
assignee: Johnson Shi (johnsonshi) → nobody
Changed in cloud-init:
assignee: Johnson Shi (johnsonshi) → nobody
Revision history for this message
James Falcon (falcojr) wrote :

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3264

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.