virt-install does not work with --os-variant ubuntu18.04

[James Fehrmann]

[Impact]

 * An explanation of the effects of the bug on users and

 * justification for backporting the fix to the stable release.

 * In addition, it is helpful, but not required, to include an
   explanation of how the upload fixes this bug.

[Test Case]

 * B) install a Ubuntu 18.04 ISO in virt-manager
   While doing so try to select the OS version as 18.04 (or newer)

[Regression Potential]

 * People will see and might not expect some changes. For example it was
   found and fixed that the minimum size of Ubuntu Images was too small
   (bug 1796037). Now if somebody did not have these issues his 16.04
   guest default size will still change. But since it fixes issues it is a
   change we want. Most people specify RAM explicitly anyway as in non API
   use cases it is mandatory.
   In a similar way, this contains many data updates and if any of them
   might be wrong there might be issues. I read through the updates and
   they all looked sane to me, but in terms of regression potential this
   is the biggest potential threat to me.

[Other Info]

 * This package is mostly a data-carrier, it has not much active code
   Therefore it was safe and much less error prone to suggest pushing the
   cosmic/disco version to Bionic as-is instead of trying to remove the
   few packaging changes that happened to "only push the new data".
   @SRU-team: Let me know if this is a problem in your opinion.

Works: # virt-install --os-variant ubuntu16.04 ...other params...

Fails: # virt-install --os-variant ubuntu18.04 ...other params...

Error Message:

ERROR Error validating install location: Distro 'ubuntu18.04' does not exist in our dictionary

Verion info (new/clean install):

libosinfo-bin/bionic,now 1.1.0-1 amd64 [installed]

virtinst/bionic,bionic,now 1:1.5.1-0ubuntu1 all [installed]

Linux 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

18.04 is not listed in osinfo-query either:

#osinfo-query os |grep ubuntu

 ubuntu10.04 | Ubuntu 10.04 LTS | 10.04 | http://ubuntu.com/ubuntu/10.04
 ubuntu10.10 | Ubuntu 10.10 | 10.10 | http://ubuntu.com/ubuntu/10.10
 ubuntu11.04 | Ubuntu 11.04 | 11.04 | http://ubuntu.com/ubuntu/11.04
 ubuntu11.10 | Ubuntu 11.10 | 11.10 | http://ubuntu.com/ubuntu/11.10
 ubuntu12.04 | Ubuntu 12.04 LTS | 12.04 | http://ubuntu.com/ubuntu/12.04
 ubuntu12.10 | Ubuntu 12.10 | 12.10 | http://ubuntu.com/ubuntu/12.10
 ubuntu13.04 | Ubuntu 13.04 | 13.04 | http://ubuntu.com/ubuntu/13.04
 ubuntu13.10 | Ubuntu 13.10 | 13.10 | http://ubuntu.com/ubuntu/13.10
 ubuntu14.04 | Ubuntu 14.04 LTS | 14.04 | http://ubuntu.com/ubuntu/14.04
 ubuntu14.10 | Ubuntu 14.10 | 14.10 | http://ubuntu.com/ubuntu/14.10
 ubuntu15.04 | Ubuntu 15.04 | 15.04 | http://ubuntu.com/ubuntu/15.04
 ubuntu15.10 | Ubuntu 15.10 | 15.10 | http://ubuntu.com/ubuntu/15.10
 ubuntu16.04 | Ubuntu 16.04 | 16.04 | http://ubuntu.com/ubuntu/16.04
 ubuntu16.10 | Ubuntu 16.10 | 16.10 | http://ubuntu.com/ubuntu/16.10
 ubuntu17.04 | Ubuntu 17.04 | 17.04 | http://ubuntu.com/ubuntu/17.04
 ubuntu17.10 | Ubuntu 17.10 | 17.10 | http://ubuntu.com/ubuntu/17.10
 ubuntu4.10 | Ubuntu 4.10 | 4.10 | http://ubuntu.com/ubuntu/4.10
 ubuntu5.04 | Ubuntu 5.04 | 5.04 | http://ubuntu.com/ubuntu/5.04
 ubuntu5.10 | Ubuntu 5.10 | 5.10 | http://ubuntu.com/ubuntu/5.10
 ubuntu6.06 | Ubuntu 6.06 LTS | 6.06 | http://ubuntu.com/ubuntu/6.06
 ubuntu6.10 | Ubuntu 6.10 | 6.10 | http://ubuntu.com/ubuntu/6.10
 ubuntu7.04 | Ubuntu 7.04 | 7.04 | http://ubuntu.com/ubuntu/7.04
 ubuntu7.10 | Ubuntu 7.10 | 7.10 | http://ubuntu.com/ubuntu/7.10
 ubuntu8.04 | Ubuntu 8.04 LTS | 8.04 | http://ubuntu.com/ubuntu/8.04
 ubuntu8.10 | Ubuntu 8.10 | 8.10 | http://ubuntu.com/ubuntu/8.10
 ubuntu9.04 | Ubuntu 9.04 | 9.04 | http://ubuntu.com/ubuntu/9.04
 ubuntu9.10 | Ubuntu 9.10 | 9.10 | http://ubuntu.com/ubuntu/9.10

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Ehrhardt  (paelzer)]

This is actually due to an outdated osinfo-db

But honestly this isn't too critical for virt-manager or others, all the aux data like min memory and such are not changing so you could just use 17.10 or 16.04 and be good.
The only thing this breaks IMHO is the auto-fetching of ISOs from that URL.

Checking reverse deps - hmm, yeah cockpit-machines, gnome-boxes, tracker-extract, virt-manager, virtinst would be good to fix that.

[Christian Ehrhardt  (paelzer)]

We might want to update this pkg from https://releases.pagure.org/libosinfo/osinfo-db-20180514.tar.xz into Cosmic and then SRU to Bionic at least.

Not sure thou since this is universe :-/
I assume nobody else is volunteering to do this?

[Christian Ehrhardt  (paelzer)]

Since this is outdated in Debian as well (and agx is usually doing the updates, so he knows which tarball/git to merge from which repo) I wanted to file this as a bug in Debian as well.
But that would not really help even when synced, as even the latest upstream release being 20180514 does not container 18.04 :-/

[Christian Ehrhardt  (paelzer)]

This was not touched for 180 days (a full dev cycle) the reason IMHO is that this just isn't important.
We are always as up-to-date in that DB as upstream is at the time, but we don't want to do too much and update this all the time.

I think if someone volunteers and steps up doing the re-packaging of the new data we will help, but this is not that important that I could do it. It is not important because ubuntu usually doe snot change so using 17.10 instead of 18.04 in your example command works just fine.

Setting this to wishlist and subscribing myself if one steps up to volunteer.

[Eli (eliterdaboss)]

Not really a programmatic fix, but anyone else who needs to get Ubuntu 18.04 on a KVM, there is a workaround that I have found: https://www.server-world.info/en/note?os=Ubuntu_18.04&p=kvm&f=2

[Christian Ehrhardt  (paelzer)]

If we are trying to document alternatives here:
1. the type is not needed to get 18.04. I never understood why people use virt-install so much.
If one insist on installing ISO just install any from [1] or similar, you don't need the type for that at all - any Ubuntu (or even other Linux) type will work.
2. If "needs to get Ubuntu 18.04 on a KVM" is the only target here, there are many way better ways than virt-install anyway - I wonder why these days this is still used so much.
2.1 uvtool
# only needed once
$ uvt-simplestreams-libvirt sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily
# get a new guest
$ uvt-kvm create bionic1 arch=amd64 release=bionic label=daily
2.2 multipass
$ sudo snap install --beta multipass --classic
$ multipass launch daily:18.04

There are probably much more, but those are much more comfortable getting an image of the day running within a guest.

P.S. even thou I don't see a reason to fix this with priority I get from the reports that others want/need it - and I'm certainly not alone, so lets bump the prio a bit for all of you that like the way based on osinfo-db entries.

[1]: https://www.ubuntu.com/download/server

[Christian Ehrhardt  (paelzer)]

Hey,
sinsce you all made me realize that this is important to you (thanks!) I prepped a new version in a PPA. Could you give [1] a try and let me know if that would solve the issues you were facing?

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3521

[Christian Ehrhardt  (paelzer)]

@SRU Team - I pinged in #ubuntu-release already, but for safety let me write it here as well.
By accident I uplaoded this to B-unapproved already - But I want people (and myself) to test this from the PPA first. So please cancel the current upload in B-unapproved.
I'll state here on the bug once I do an intended upload, sorry for the hazzle.

[Stephen Croll (stephen-d-croll)]

It works for me:

scroll@ivy:~$ sudo add-apt-repository ppa:ci-train-ppa-service/3521
[sudo] password for scroll:
 https://bileto.ubuntu.com/#/ticket/3521

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_39a8dbb93caf4ec889f8a1b7f69885db/bileto-3521/2018-11-16_06:34:48/bionic_osinfo-db_content.diff (52531 lines)
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_39a8dbb93caf4ec889f8a1b7f69885db/bileto-3521/2018-11-16_06:34:48/bionic_osinfo-db_packaging_changes.diff (240 lines)
 More info: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3521
Press [ENTER] to continue or Ctrl-c to cancel adding it.

Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:3 http://ppa.launchpad.net/ci-train-ppa-service/3521/ubuntu bionic InRelease [20.8 kB]
Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Get:6 http://ppa.launchpad.net/ci-train-ppa-service/3521/ubuntu bionic/main amd64 Packages [508 B]
Get:7 http://ppa.launchpad.net/ci-train-ppa-service/3521/ubuntu bionic/main Translation-en [364 B]
Fetched 21.6 kB in 1s (15.2 kB/s)
Reading package lists... Done

scroll@ivy:~$ sudo apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit:3 http://ppa.launchpad.net/ci-train-ppa-service/3521/ubuntu bionic InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Reading package lists... Done

scroll@ivy:~$ sudo apt install osinfo-db
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  osinfo-db
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 76.7 kB of archives.
After this operation, 220 kB of additional disk space will be used.
Get:1 http://ppa.launchpad.net/ci-train-ppa-service/3521/ubuntu bionic/main amd64 osinfo-db all 0.20180929-1ubuntu0.1 [76.7 kB]
Fetched 76.7 kB in 1s (129 kB/s)
(Reading database ... 225419 files and directories currently installed.)
Preparing to unpack .../osinfo-db_0.20180929-1ubuntu0.1_all.deb ...
Unpacking osinfo-db (0.20180929-1ubuntu0.1) over (0.20180226-1) ...
Setting up osinfo-db (0.20180929-1ubuntu0.1) ...

scroll@ivy:~$ sudo osinfo-query os | grep ubuntu18.04
 ubuntu18.04 | Ubuntu 18.04 LTS | 18.04 | http://ubuntu.com/ubuntu/18.04

scroll@ivy:~/kvm$ sudo virt-install \
> --name=scroll-test \
> --hvm \
> --ram=2048 \
> --vcpus=2 \
> --os-variant=ubuntu18.04 \
> --os-type linux \
> --disk path=/home/scroll/kvm/scroll-test.img \
> --cdrom=/home/scroll/kvm/scroll-test-cloud-init.iso \
> --graphics vnc \
> --graphics spice \
> --noautoconsole \
> --os-type=linux \
> --network bridge:br0 \
> --network bridge:virbr0

Starting install...
Domain installation still in progress. You can reconnect to
the console to complete...

[Christian Ehrhardt  (paelzer)]

Thanks Steven for the quick reply, I didn't find any hickups with it either - your confirmation makes me confident that we can suggest it as an SRU to Bionic.

Cosmic and Disco already have this versions - setting tasks accordingly.

[Christian Ehrhardt  (paelzer)]

Uploaded to Bionic-unapproved (this time intentionally)

[Christian Ehrhardt  (paelzer)]

hmm, still waiting ...

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted osinfo-db into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/osinfo-db/0.20180929-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Christian Ehrhardt  (paelzer)]

- All autopkgtests are ok
- I tested a guest install with the new type through virt-manager - working fine
- Even the auto-detection of the ISO as 18.04 worked

Upgrade was smooth (not much active elements anyway):
$ sudo apt install osinfo-db
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  osinfo-db
1 upgraded, 0 newly installed, 0 to remove and 156 not upgraded.
Need to get 76,7 kB of archives.
After this operation, 220 kB of additional disk space will be used.
Get:1 http://de.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 osinfo-db all 0.20180929-1ubuntu0.1 [76,7 kB]
Fetched 76,7 kB in 0s (320 kB/s)
(Reading database ... 314122 files and directories currently installed.)
Preparing to unpack .../osinfo-db_0.20180929-1ubuntu0.1_all.deb ...
Unpacking osinfo-db (0.20180929-1ubuntu0.1) over (0.20180226-1) ...
Setting up osinfo-db (0.20180929-1ubuntu0.1) ...

Due to that setting verified tag

[Launchpad Janitor (janitor)]

This bug was fixed in the package osinfo-db - 0.20180929-1ubuntu0.1

---------------
osinfo-db (0.20180929-1ubuntu0.1) bionic; urgency=medium

  * SRU the more recent version to Bionic to have the newer
    release definitions available, most importantly Ubuntu
    18.04 itself as users would expect that (LP: #1770206).

osinfo-db (0.20180929-1ubuntu1) cosmic; urgency=medium

  * debian/patches/git_dccf78b83578d368fd0241cbcfc8f5c2742012fb.patch:
    (cherry-picked upstream)
    ubuntu: Bump minimal ram requirement to 2GiB. (LP: #1796037)

osinfo-db (0.20180929-1) unstable; urgency=medium

  * [478429c] New upstream version 0.20180929
    (Closes: #908973)

osinfo-db (0.20180917-1) unstable; urgency=medium

  * [13ecc03] New upstream version 0.20180917
    (Closes: #908983)
  * [4f9f362] d/control: Bump standards version to 4.2.1
  * [862e376] d/rules: Honor DEB_BUILD_OPTIONS=nocheck
  * [bf69c25] d/copyright: Avoid duplicate license section
  * [63f7c72] Switch to debhelper compat level 11
  * [c0f0f40] Don't install extra license file to /u/s/osinfo

osinfo-db (0.20180628-1) unstable; urgency=medium

  [ Guido Günther ]
  * [25145bf] Point VCS-* URLs to salsa
  * [1bcb0b3] gbp: use commit hashes in changelog
  * [fd17d3c] New upstream version 0.20180628
    (Closes: #902538)

  [ Helmut Grohne ]
  * [34d4878] Mark osinfo-db Multi-Arch: foreign (Closes: #893429)

 -- Christian Ehrhardt <email address hidden> Fri, 16 Nov 2018 07:24:05 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for osinfo-db has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.