kvm-based units are ignoring container-image-metadata-url

[Craig Bender]

Juju 2.7
Ubuntu 19.10

Units deployed using kvm are ignoring container-image-metadata-url

It also appears that kvm provisioning requires index.sjson rather than index.json as with lxd provisioned units

This makes deploying kvm-based units extremely slow or impossible to deploy if on isolated network

Units deployed using lxd are respecting this setting.

Juju status:
0 started 172.27.21.120 node03ob20 bionic default Deployed
0/kvm/0 pending pending eoan copying http://cloud-images.ubuntu.com/server/releases/eoan/release-20200107/ubuntu-19.10-server-cloudimg-amd64.img 11% (236KB/s)
0/lxd/0 started 172.27.21.127 juju-1e08fb-0-lxd-0 bionic default Container started

Juju model-config:
agent-metadata-url controller https://juju-images.orangebox.me/juju/tools
container-image-metadata-url controller https://cloud-images.orangebox.me/releases
image-metadata-url controller https://cloud-images.orangebox.me/releases

Juju debug-log:
Only log message related to provisioning kvm-based units and kvm images is:
machine-1: 19:59:52 DEBUG juju.container.kvm synchronise images for amd64 eoan released https://cloud-images.orangebox.me/releases

LXD finds image:
machine-0: 19:59:08 DEBUG juju.container.lxd Found image remotely - "cloud-images.orangebox.me/releases" "ubuntu-18.04-server-cloudimg-amd64-lxd.tar.xz" "979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185"

local simple-streams product images shows the required image and image is on disk:
curl -sSlL https://cloud-images.orangebox.me/releases/streams/v1/com.ubuntu.cloud:released:download.json|awk '/ubuntu-19.10-server-cloudimg-amd64.img/'
       "path": "server/releases/eoan/release-20191017/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191023/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191113/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191115/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191204/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191217/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20200107/ubuntu-19.10-server-cloudimg-amd64.img",
curl -sSlL https://cloud-images.orangebox.me/releases/streams/v1/com.ubuntu.cloud:released:download.sjson|awk '/ubuntu-19.10-server-cloudimg-amd64.img/'
       "path": "server/releases/eoan/release-20191017/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191023/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191113/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191115/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191204/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20191217/ubuntu-19.10-server-cloudimg-amd64.img",
       "path": "server/releases/eoan/release-20200107/ubuntu-19.10-server-cloudimg-amd64.img",

[Gábor Mészáros (gabor.meszaros)]

Apparently this is also happening in 2.6.10 based juju deployment.
LXD images are respecting the container-image-metadata-url model-config value, but the KVMs are going over proxy + cloud-images.ubuntu.com

[Gábor Mészáros (gabor.meszaros)]

subscribed ~field-high

[Gábor Mészáros (gabor.meszaros)]

adjusted to ~field-crit after internal discussion.

[Achilleas Anagnostopoulos (achilleasa)]

PR https://github.com/juju/juju/pull/11550 includes a fix for 2.7

[Gábor Mészáros (gabor.meszaros)]

Thanks all for the fix, really appreciate it!

Is there a possibility to also backport it to 2.6?

[Pen Gale (pengale)]

@gabor.meszaros: yes. There is a possibility of backporting to 2.6. I added it to our list of todos.

[Achilleas Anagnostopoulos (achilleasa)]

PR https://github.com/juju/juju/pull/11557 back-ports the fix to 2.6. Note however, that (as far as I am aware) there are no plans for a new 2.6 release and that you would probably be better off upgrading to 2.7 instead.

[Gábor Mészáros (gabor.meszaros)]

Thanks Achilleas, for now we have to stick with 2.6 unfortunately.