virt-install fails on s390x with options --location and --extra-args

Bug #1713990 reported by bugproxy on 2017-08-30
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Undecided
Unassigned
virt-manager (Debian)
Fix Released
Unknown
virt-manager (Ubuntu)
Undecided
Skipper Bug Screeners
Xenial
Undecided
Unassigned
Zesty
Undecided
Unassigned

Bug Description

[Impact]

 * virtinst not usable on s390x lacking proper image parsing
 * virtinst extra-args missed (actually cross arch)

 * Especially on s390x people are even more conservative, so getting back
   to LTS is the only way to make it available were it is needed

 * Fix consists of 2 backports from Upstream

[Test Case]

 * All on s390x
 * Create a test image file
   $ sudo qemu-img create -f qcow2 /var/lib/libvirt/images/ubuntu.qcow2 6G
 * Use virt inst to install into that
   $ sudo virt-install --name ubuntu --ram 512 --disk /var/lib/libvirt/images/ubuntu.qcow2,bus=virtio --location http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-s390x --extra-args="s390_iommu=strict" --extra-args="hvc_iucv=2"

  * Without the fix it will
    1. fail on:
    ERROR Error validating install location: Could not find an installable distribution at 'http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-s390x'
    2. clobber one of the extra args

  * With the fix applied it will
    1. work to get the image and start it
    2. Further on in the install have all extra args from the example passed to the kernel like:
    [ 0.386098] Kernel command line: s390_iommu=strict hvc_iucv=2 [...]

[Regression Potential]

 * Change #1 is only having effect on s390x (so lower risk). There it
   fixes the formerly broken location (and not more than that). I don't
   see an issue in the change, but there might be some due to e.g. older
   python - but testing should confirm that this is no issue.

 * Change #2 is making a python arg that is a string being paased later to
   become an append (a list that is concatenated and then passed as one
   string like before. I think the change is fine as all former usage was
   only with one (1) arg which is behaving the same as it always did. Just
   any use case with more than one arg was broken and therefore fixed now.
   Again older python might be an unknown here, so testing required.

[Other Info]

 * n/a

----

---Problem Description---
virt-install fails on s390x with options --location and --extra-args

---uname output---
Linux s3560022 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:06:48 UTC 2017 s390x s390x s390x GNU/Linux

Machine Type = IBM z13

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 Run the following command:
sudo virt-install --name ubuntu --ram 1024 --disk /var/lib/libvirt/images/ubuntu.qcow2,bus=virtio --location http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-s390x --extra-args="preseed/url=http://192.168.122.1/tmp/preseed"
Expected result: A KVM virtual machine "ubuntu" is installed using a local preseed file.
Observed result: Error message
sudo virt-install --name ubuntu --ram 1024 --disk /var/lib/libvirt/images/ubuntu.qcow2,bus=virtio --location http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-s390x

Userspace tool common name: virt-install

The userspace tool has the following bit modes: 64

Userspace rpm: virt-manager

Userspace tool obtained from project website: na

The following upstream commit solves the location validation problem:
f248088 virtinst: adjust media paths for s390x

However, the extra-args are ignored. For this it is necessary to pick the following commit
4c3e796 virt-install: concatenate all extra-args argument

Without these fixes it is not possible to perform a Ubuntu installation of a KVM guest using virt-install.

Minor correction: the binary package (not rpm) is virtinst, built from the virt-manager source package.

bugproxy (bugproxy) on 2017-08-30
tags: added: architecture-s39064 bugnameltc-158140 severity-high targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → virtinst (Ubuntu)
affects: virtinst (Ubuntu) → virt-manager (Ubuntu)
bugproxy (bugproxy) on 2017-08-30
tags: added: targetmilestone-inin16042
removed: targetmilestone-inin---
Viktor Mihajlovski (mihajlov) wrote :

The virt-manager version in Debian stretch is not affected by the --extra-args issue, but is subject to the --location failure.

Hiho,

4c3e796 virt-install: concatenate all extra-args argument
=> this is in since 1.4.0 and therefore good since the latest merge to 1:1.4.0-5ubuntu1

f248088 virtinst: adjust media paths for s390x
=> this OTOH is just 1 day old from:
Author: Viktor Mihajlovski <email address hidden>
Date: Tue Aug 29 15:51:43 2017 +0200

Hi Victor btw o/ !

There is a follow on fix to the self tests that "belongs" to that which is:
commit 427592bc7b9cab676456dbee11922f2bc61380a1 (HEAD -> master, origin/master, origin/HEAD)
Author: Cole Robinson <email address hidden>
Date: Tue Aug 29 11:57:45 2017 -0400

While I'm not a virt-inst fan or user I see the use case and this is bug-fix only.
Not sure how important that is for an SRU since this didn't hurt anybody all the time, but lets start at Artful and consider more from there.

I'll take a look, but that should backport rather straight forward I'd think.

Changed in virt-manager (Ubuntu):
status: New → Triaged

I also want a review for the final upload, but you could already test the ppa build at [1].

If you could confirm (artful) that this fixes the issue as you expect, that would be great.
I only uploaded now, so it might need a few minutes to build and publish virt-manager_1.4.0-5ubuntu2 there.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/2935

------- Comment From <email address hidden> 2017-08-30 09:04 EDT-------
(In reply to comment #7)
> Hiho,
>
> 4c3e796 virt-install: concatenate all extra-args argument
> => this is in since 1.4.0 and therefore good since the latest merge to
> 1:1.4.0-5ubuntu1
Hi Christian, I know, but the issue is specifically with Xenial and we have still more than three years to go...
>
> f248088 virtinst: adjust media paths for s390x
> => this OTOH is just 1 day old from:
> Author: Viktor Mihajlovski <email address hidden>
> Date: Tue Aug 29 15:51:43 2017 +0200
>
> Hi Victor btw o/ !
>
> There is a follow on fix to the self tests that "belongs" to that which is:
> commit 427592bc7b9cab676456dbee11922f2bc61380a1 (HEAD -> master,
> origin/master, origin/HEAD)
> Author: Cole Robinson <email address hidden>
> Date: Tue Aug 29 11:57:45 2017 -0400
I didn't need this patch to build the binary packages.
>
> While I'm not a virt-inst fan or user I see the use case and this is bug-fix
> only.
> Not sure how important that is for an SRU since this didn't hurt anybody all
> the time, but lets start at Artful and consider more from there.
>
Well, people take some time before the start to try out stuff on Z. The issue came up because a colleague needs to perform unattended installation of Xenial guests, and he's stuck now. So we really need that for Xenial (but not for the other releases preceding Artful).
> I'll take a look, but that should backport rather straight forward I'd think.

Dimitri John Ledkov (xnox) wrote :

We do not recommend to use virt-install to create Ubuntu guest instances.

Instead, mirror and launch Ubuntu Cloud Image, with a cloud config drive to customize the instances.

You can find ready made Ubuntu Cloud Images for s390x at https://cloud-images.ubuntu.com/xenial/current/ note they are regularly refreshed to boot with latest kernel, SRU and security fixes.

You can customize the instances as needed using cloud-config stanzas http://cloudinit.readthedocs.io/en/latest/topics/examples.html

And one can trivially provide config-drive to the virtual machine using e.g. NoCloud http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html

Have you considered using cloud-images to simply boot and initialize customized Xenial guests? It will complete provision much faster than d-i based methods.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-08-30 09:23 EDT-------
(In reply to comment #8)
> I also want a review for the final upload, but you could already test the
> ppa build at [1].
>
> If you could confirm (artful) that this fixes the issue as you expect, that
> would be great.
> I only uploaded now, so it might need a few minutes to build and publish
> virt-manager_1.4.0-5ubuntu2 there.
>
> [1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/2935

To my surprise and pleasure I could install the new virtinst deb on Xenial and could successfully install a KVM guest, with location, preseed and all. But, as I said ... it's broken in Xenial and the fix is needed there.

@Victor - three things:

1. ads Dimitri mentioned virtinst isn' recommended.
   I use cloud images all day via as Dimitri outlined.
   I do so via uvtool, but that also has a s390x issue (bug 1452016), but you can provide
   the template I provide there.
   From there you can use all you want and cloud-init your way as you need - Dimitri
   already added a lot of great links, no need to requote them.

2. that does not mean I won't consider fixing virt-manager if the changes clearly apply
   that is a fix worth to help you and others.
   Just expect if you hit other things in virtinst that it might not be the most
   used/tested/supported solution to achieve what you need.

3. Even if you are only interested in Xenial the SRU process defines to fix in currrent
   dev release and from there SRU back.
   Among other things that ensures that nobody encounters a regression on an update from
   Xenial to something newer.

I'll get back once I got a review on my change, that is blocked on someting unrelated on virt-manager for now.

bugproxy (bugproxy) on 2017-08-30
tags: added: targetmilestone-inin16043
removed: targetmilestone-inin16042

I was able to reproduce on Artful and test the fix on my own as well.
I think it is a valid improvement and since all changes are upstream it is no permanent Delta burden.

That said pushing fix for artful.
And starting to consider SRU'ability

I added an SRU Template already, but need to
1. backport for SRUs
2. test backports

If one of them fails or gets more complex than expected we have to reconsider the SRU, but let me try first.

description: updated
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-08-31 05:05 EDT-------
(In reply to comment #11)
[snip]
> Have you considered using cloud-images to simply boot and initialize
> customized Xenial guests? It will complete provision much faster than d-i
> based methods.
FWIW: I was told that the cloud images were not usable because of special partitioning requirements for the guests (hence the need to use preseed).

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 1:1.4.0-5ubuntu2

---------------
virt-manager (1:1.4.0-5ubuntu2) artful; urgency=medium

  * - d/p/virtinst-adjust-media-paths-for-s390x.patch: fix virtinst on
      s390x (LP: #1713990)

 -- Christian Ehrhardt <email address hidden> Wed, 30 Aug 2017 14:48:17 +0200

Changed in virt-manager (Ubuntu):
status: Triaged → Fix Released
Changed in virt-manager (Debian):
status: Unknown → Fix Released

Got an ack on the MP.
Note: the MPs are redone and deleted since we (unrelated to this SRU) wrestle with the reimport of this package in USDI.

But:
- review complete
- Tests complete
- SRU Template complete
=> Uploading for SRU consideration

There is an old version 1.3.2-3ubuntu5 [1] that never appeared anywhere.
It was handled in Artful on the last merge to be correct.
In Zesty this change is only superseded/deleted and also now there is no reason to change this in Zesty.
Never the less it is part of the publishing history, so the upload for Zesty needs to be bumped to match (drop that change, mention in changelog and bump version).

Preparing ...

[1]: https://launchpad.net/ubuntu/+source/virt-manager/1:1.3.2-3ubuntu5 is

Fixed up the non-released zesty upload to be correctly included. It is now correctly dropped (in the changelog only as it never released) and the version bumped as needed.

That being complete Xenial and Zesty are ready for review by the SRU Team.

Crap it is really monday morning, of course ...
I beg your pardon - 1:1.3.2-3ubuntu5 was (first) to artful when while 1:1.3.2-3ubuntu4 was last Zesty.

The upload just accidentally mis-bumped the version and should have been 1:1.3.2-3ubuntu4.1 form the beginning.

The correct upload is ready, as soon as an SRU Team member rejected the wrong 1:1.3.2-3ubuntu6 from Zesty unapproved I can upload the right version.

Fixed the mistake and now I hope really ready for SRU team :-)

Hello bugproxy, or anyone else affected,

Accepted virt-manager into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/virt-manager/1:1.3.2-3ubuntu4.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in virt-manager (Ubuntu Zesty):
status: New → Fix Committed
tags: added: verification-needed verification-needed-zesty
Changed in virt-manager (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed-xenial
Łukasz Zemczak (sil2100) wrote :

Hello bugproxy, or anyone else affected,

Accepted virt-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/virt-manager/1:1.3.2-3ubuntu1.16.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

------- Comment From <email address hidden> 2017-09-08 04:32 EDT-------
This is to let you know that I've successfully tested the virtinst package (1:1.3.2-3ubuntu1.16.04.4) from xenial-proposed.

tags: added: verification-done-xenial
removed: verification-needed-xenial

Tested the same on zesty, now able to install instead of being blocked by the images being not found.
Thanks Victor for doing the Xenial test.

tags: added: verification-done verification-done-zesty
removed: verification-needed verification-needed-zesty
description: updated
Changed in ubuntu-z-systems:
status: New → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 1:1.3.2-3ubuntu1.16.04.4

---------------
virt-manager (1:1.3.2-3ubuntu1.16.04.4) xenial; urgency=medium

  * Fix virtinst location on s390x and clobbering extra args (LP: #1713990)
    - d/p/bug-1713990-virt-install-concatenate-all-extra-args-argument.patch
    - d/p/bug-1713990-virtinst-adjust-media-paths-for-s390x.patch

 -- Christian Ehrhardt <email address hidden> Thu, 31 Aug 2017 09:49:42 +0200

Changed in virt-manager (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for virt-manager has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 1:1.3.2-3ubuntu4.1

---------------
virt-manager (1:1.3.2-3ubuntu4.1) zesty; urgency=medium

  * Fix virtinst location on s390x and clobbering extra args (LP: #1713990)
    - d/p/bug-1713990-virt-install-concatenate-all-extra-args-argument.patch
    - d/p/bug-1713990-virtinst-adjust-media-paths-for-s390x.patch

 -- Christian Ehrhardt <email address hidden> Thu, 31 Aug 2017 09:42:32 +0200

Changed in virt-manager (Ubuntu Zesty):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.