MAAS Breaks historical custom images

[Alan Baghumian]

Hello There!

This was reported by a client and I'm filing the bug report on their behalf.

MAAS 3.1/3.2 introduce a new condition in src/maasserver/rpc/boot.py that breaks deployment of historical images that do not contain "/" in their base_image name:

https://git.launchpad.net/maas/tree/src/maasserver/rpc/boot.py?h=3.1 (Line 230)
https://git.launchpad.net/maas/tree/src/maasserver/rpc/boot.py?h=3.2 (Line 231)

I am proposing the following change to fix this issue:

diff --git a/src/maasserver/rpc/boot.py b/src/maasserver/rpc/boot.py
index 2eae1db66..f2eac4f26 100644
--- a/src/maasserver/rpc/boot.py
+++ b/src/maasserver/rpc/boot.py
@@ -235,7 +235,10 @@ def get_boot_config_for_machine(machine, configs, purpose):
                 if install_image.base_image is not None:
                     # if the operating system is a custom ubuntu image,
                     # use the base image to install
- osystem, series = install_image.base_image.split("/")
+ if "/" not in install_image.base_image:
+ series = install_image.base_image
+ else:
+ osystem, series = install_image.base_image.split("/")

             if install_image is None or osystem != "ubuntu":
                 # Use only the commissioning osystem and series, for operating

Although re-importing the images fixes the deployment issues, some clients have hundreds of historical images in their system and re-deployment is not really an option.

This simple change resolves the issue and preserves the intended condition for the rest of the images.

It would be great if this can be applied to 3.1/3.2 and future branches. Please let me know if you have any questions.

Thank you.

[Jerzy Husakowski (jhusakowski)]

We believe the issue is related to the custom image names in the database and need to investigate more. We suspect the names may need to be aligned with the required naming schema, e.g. during migration on MAAS version upgrade.

[Alan Baghumian (alanbach)]

This is becoming more and more problematic. We have historically allowed - with older versions of MAAS - importing images without requiring the current os/series requirement.

After MAAS is upgraded to 3.1/3.2 this also causes the sync operations to break:

--- <exception caught here> ---
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
  result = result.throwExceptionIntoGenerator(g)
File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
  return g.throw(self.type, self.value, self.tb)
File "/usr/lib/python3/dist-packages/provisioningserver/rpc/boot_images.py", line 153, in _import_boot_images
  yield deferToThread(_run_import, sources, maas_url, **proxies)
File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
  result = inContext.theWork()
File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
  inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
  return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
  return func(*args,**kw)
File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 202, in wrapper
  result = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/provisioningserver/rpc/boot_images.py", line 117, in _run_import
  imported = boot_resources.import_images(sources)
File "/usr/lib/python3/dist-packages/provisioningserver/import_images/boot_resources.py", line 166, in import_images
  image_descriptions = download_all_image_descriptions(
File "/usr/lib/python3/dist-packages/provisioningserver/import_images/download_descriptions.py", line 343, in download_all_image_descriptions
  repo_boot = download_image_descriptions(
File "/usr/lib/python3/dist-packages/provisioningserver/import_images/download_descriptions.py", line 333, in download_image_descriptions
  dumper.sync(reader, rpath)
File "/usr/lib/python3/dist-packages/provisioningserver/import_images/download_descriptions.py", line 201, in sync
  super().sync(reader, path)
File "/usr/lib/python3/dist-packages/simplestreams/mirrors/__init__.py", line 86, in sync
  data = util.load_content(payload)
File "/usr/lib/python3/dist-packages/simplestreams/util.py", line 332, in load_content
  return json.loads(content)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
  return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
  obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
  raise JSONDecodeError("Expecting value", s, err.value) from None
  json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Example images that are causing this error:
{
   {
       "id": 84,
       "type": "Uploaded",
       "name": "debiantest",
       "architecture": "amd64/generic",
       "resource_uri": "/MAAS/api/2.0/boot-resources/84/",
       "subarches": "generic",
  ...

[Christian Grabowski (cgrabowski)]

There's an MP landing shortly that fixes this.

[Alan Baghumian (alanbach)]

With MAAS 3.2.6, we can now do:

$ maas $PROFILE boot-resources create name='centos/my-centos7-image' title='My Custom CentOS 7' architecture='amd64/generic' filetype='tgz' base_image='custom' content@=centos7.tar.gz

It would be really great to also allow:

$ maas $PROFILE boot-resources create name='custom/my-centos7-image' title='My Custom CentOS 7' architecture='amd64/generic' filetype='tgz' base_image='custom' content@=centos7.tar.gz

Essentially allowing name='custom/my-centos7-image' in combination with base_image='custom'

Please let me know if you have any questions.

Thanks much!