[SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs

Bug #1754015 reported by Junien Fridrick on 2018-03-07
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
High
Unassigned
Pike
High
Unassigned
Queens
High
Unassigned
libvirt (Ubuntu)
Undecided
Unassigned
Artful
Undecided
Unassigned
Bionic
Undecided
Unassigned
nova (Ubuntu)
High
James Page
Artful
High
James Page
Bionic
High
James Page

Bug Description

[Impact]
On Ubuntu installs where --no-recommends is enabled as the default, users of nova are unable to boot instances on non-x86 based architecures

[Test Case]
Deploy openstack using charms on amr64
Boot instance
Fails with error message from original bug report

[Regression Potential]
Minimal; ipxe-qemu is a Depends for x86, but this appears to be a legacy feature; nova will use libvirt in such a way that ipxe-qemu roms are needed on all architectures.

[Original Bug Report]
If you deploy a nova-compute unit on arm64 with openstack-origin='cloud:xenial-pike', ipxe-qemu does not get pulled, so nova fails to create instances because :

2018-03-06 14:58:59.608 9770 ERROR nova.compute.manager [instance: 50a8ca28-d43c-4c36-8354-edea0138e7f1] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-03-06 14:58:59.608 9770 ERROR nova.compute.manager [instance: 50a8ca28-d43c-4c36-8354-edea0138e7f1] libvirtError: internal error: process exited while connecting to monitor: 2018-03-06T14:58:58.689865Z qemu-system-aarch64: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:12:80:f6,bus=pci.1,addr=0x0: failed to find romfile "efi-virtio.rom"

(and efi-virtio.rom comes from ipxe-qemu)

This is because of the Depends change on nova-compute-kvm :

cloud-archive nova-compute-kvm has :
Depends: nova-compute-libvirt (= 2:16.0.3-0ubuntu1~cloud0), qemu-kvm | qemu-system (>= 1.3.0) | kvm

regular nova-compute-kvm has :
Depends: nova-compute-libvirt (= 2:13.1.4-0ubuntu4.2), qemu-system (>= 1.3.0) | kvm

and qemu-kvm only pulls qemu-system-arm, whereas qemu-system brings in all architectures, including qemu-system-x86, which pulls in ipxe-qemu.

I guess one fix is to make qemu-system-arm64 depend on ipxe-qemu

Thanks

Corey Bryant (corey.bryant) wrote :

Hi Junien,

Thanks for the detailed bug report. I'm wondering if ipxe-qemu installed is not getting installed because you're not installing recommended packages. ie. 'apt install --no-install-recommends'. They should get installed by default. Note that qemu-system-arm specifies ipxe-qemu as a 'Recommends' package whereas qemu-system-x86 specifies ipxe-qemu as a 'Depends' package.

For qemu 1:2.10+dfsg-0ubuntu3.5~cloud0 from xenial-pike, the debian/control file has:

Package: qemu-system
Architecture: amd64 arm arm64 armel armhf hppa i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc powerpcspe ppc64 ppc64el s390x sparc sparc64
Multi-Arch: foreign
Depends: ${misc:Depends},
 qemu-system-arm,
 qemu-system-mips,
 qemu-system-ppc,
 qemu-system-sparc,
 qemu-system-x86,
 qemu-system-s390x,
 qemu-system-misc

Package: qemu-kvm
Architecture: i386 amd64 x32 armhf armel arm64 powerpc ppc64 ppc64el s390x
Multi-Arch: foreign
Pre-Depends: ${misc:Pre-Depends}
Depends:
 ${misc:Depends},
 qemu-system-x86 (= ${binary:Version}) [i386 amd64 x32],
 qemu-system-arm (= ${binary:Version}) [armhf armel arm64],
 qemu-system-ppc (= ${binary:Version}) [powerpc ppc64 ppc64el],
 qemu-system-s390x (= ${binary:Version}) [s390x],
 powerpc-utils [ppc64el],

Package: qemu-system-arm
Architecture: amd64 arm arm64 armel armhf hppa i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc powerpcspe ppc64 ppc64el s390x sparc sparc64
Multi-Arch: foreign
Depends: ${shlibs:Depends}, ${misc:Depends}, qemu-system-common (>> 2.0.0+dfsg-7~)
Recommends: qemu-utils,
# aarch64 arm uses bootroms
 ipxe-qemu (>= 1.0.0+git-20131111.c3d1e78-1~),

Please let me know what you find. For now I'm going to mark this
Thanks,
Corey

Corey Bryant (corey.bryant) wrote :

... mark this incomplete until we have further details.

Changed in cloud-archive:
status: New → Incomplete
Junien Fridrick (axino) wrote :

Hi,

Indeed we're not installing "recommended" packages.

Changed in cloud-archive:
status: Incomplete → New
Junien Fridrick (axino) wrote :

Note that ipxe-qemu is also needed on ppc64el

Corey Bryant (corey.bryant) wrote :

Hi Junien,

I hope you're up and running now. Can we close this bug?

Thanks,
Corey

Junien Fridrick (axino) wrote :

I don't think that we can't close this bug because it's not resolved. I shouldn't have to manually install packages to get a working cloud, even if I have APT::Install-Recommends set to False.

Either this is a package bug, or a charm bug, but my opinion is that it is a bug and it needs to be resolved before getting closed.

Hi,
first I agree to Coreys first checks.

In terms of packages, arm (as well as ppc) clearly are recommend dependencies.
They have several use cases that work without ipxe-qemu - just not the one that nova-compute-kvm is driving.

If we would follow the depend/recommend definition hard also x86 would become a recommend.
But for historical compatibility x86 stays a depend.
OTOH on arm/ppc there are more (likely) cases were one doesn't need ipxe-qemu.

That said making it a depends from ther qemu- packages is wrong.
The solution is:
1. install with recommends or fix up what you lack (general recommends case)
OR
2. nova-compute-kvm might depend on it since the use case "from nova-compute-kvm" seems to need it always
OR
3. pull it in from the charm or whatever is driving your install.

Changed in libvirt (Ubuntu):
status: New → Incomplete
James Page (james-page) on 2018-03-12
affects: charm-nova-compute → nova (Ubuntu)
Changed in nova (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in cloud-archive:
status: New → Triaged
importance: Undecided → High
Changed in nova (Ubuntu):
milestone: none → ubuntu-18.03
James Page (james-page) on 2018-03-12
Changed in nova (Ubuntu):
assignee: nobody → James Page (james-page)
status: Triaged → In Progress
Changed in nova (Ubuntu Artful):
status: New → Triaged
importance: Undecided → High
assignee: nobody → James Page (james-page)
milestone: none → artful-updates
description: updated
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nova - 2:17.0.1-0ubuntu1

---------------
nova (2:17.0.1-0ubuntu1) bionic; urgency=medium

  * New upstream point release for OpenStack Queens.

 -- Corey Bryant <email address hidden> Mon, 12 Mar 2018 12:43:39 -0400

Changed in nova (Ubuntu Bionic):
status: In Progress → Fix Released
summary: - nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs
+ [SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs
Corey Bryant (corey.bryant) wrote :

I've uploaded a new version of nova to the Artful review queue where it is awaiting SRU team review [1]. Once it is accepted into artful-proposed we can backport this to pike-proposed for testing.

[1] https://launchpad.net/ubuntu/artful/+queue?queue_state=1&queue_text=

Hello Junien, or anyone else affected,

Accepted nova into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:16.1.0-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Artful):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-artful
Junien Fridrick (axino) wrote :

I don't have an artful around to test, FWIW.

Corey Bryant (corey.bryant) wrote :

Hello Junien, or anyone else affected,

Accepted nova into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:pike-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-pike-needed
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in libvirt (Ubuntu Artful):
status: New → Confirmed
Junien Fridrick (axino) wrote :

Testing done :
1. create new xenial LXD
2. sudo add-apt-repository cloud-archive:pike-proposed ; sudo apt-get update
3. echo 'APT::Install-Recommends "false";' | sudo tee /etc/apt/apt.conf.d/99no-recommends
4. sudo apt-get install nova-compute-kvm
5. verify ipxe-qemu is installed

Testing successful !

tags: added: verification-pike-done
removed: verification-pike-needed
Corey Bryant (corey.bryant) wrote :

Testing completed successfully for artful-proposed:
1. create new artful LXD container
2. sudo add-apt-repository cloud-archive:pike-proposed ; sudo apt update
3. echo 'APT::Install-Recommends "false";' | sudo tee /etc/apt/apt.conf.d/99no-recommends
4. sudo apt-get install nova-compute-kvm
5. version is 2:16.1.0-0ubuntu1
6. verify ipxe-qemu is installed

tags: added: verification-done-artful
removed: verification-needed-artful
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nova - 2:16.1.0-0ubuntu1

---------------
nova (2:16.1.0-0ubuntu1) artful; urgency=medium

  [ James Page ]
  * Add Depends nova-compute-kvm -> ipxe-qemu to ensure that required
    ROM's are installed for all architectures (LP: #1754015).

  [ Corey Bryant ]
  * New stable point release for OpenStack Pike (LP: #1758046).
  * d/p/aarch64-libvirt-compat.patch: Rebased and updated to mock
    os.path.exists return value to true in order to simulate
    _has_uefi_support returning true (LP: #1758060).

 -- Corey Bryant <email address hidden> Thu, 22 Mar 2018 08:44:37 -0400

Changed in nova (Ubuntu Artful):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package nova - 2:16.1.0-0ubuntu1~cloud0
---------------

 nova (2:16.1.0-0ubuntu1~cloud0) xenial-pike; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 nova (2:16.1.0-0ubuntu1) artful; urgency=medium
 .
   [ James Page ]
   * Add Depends nova-compute-kvm -> ipxe-qemu to ensure that required
     ROM's are installed for all architectures (LP: #1754015).
 .
   [ Corey Bryant ]
   * New stable point release for OpenStack Pike (LP: #1758046).
   * d/p/aarch64-libvirt-compat.patch: Rebased and updated to mock
     os.path.exists return value to true in order to simulate
     _has_uefi_support returning true (LP: #1758060).

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers