Comment 2 for bug 1908543

I believe I understand what's going wrong; thanks to @OddBloke for the tip. tests/vmtests/image_sync.py syncs from the daily link, which is now a URL redirect to stable and as a result, the *content* of the streams file is also renamed:

com.ubuntu.maas:stable:v3:download.json

which used to be

com.ubuntu.maas:daily:v3:download.json

When curtin's image_sync goes to look for the synced content, it would only ever look for the content in these previously downloaded streams data files

If we change the URL to stable, and the STREAM_BASE (which points to the file where vmtest caches which content it pulled down) then the sync works correctly.

So this:

% git diff tests/vmtests/image_sync.py
diff --git a/tests/vmtests/image_sync.py b/tests/vmtests/image_sync.py
index e460e028..9183f18a 100644
--- a/tests/vmtests/image_sync.py
+++ b/tests/vmtests/image_sync.py
@@ -31,10 +31,10 @@ def environ_get(key, default):
     long_key = 'CURTIN_VMTEST_' + key
     return os.environ.get(long_key, os.environ.get(key, default))

-
+MAAS_STREAM = environ_get("MAAS_IMAGE_STREAM", "stable")
 IMAGE_SRC_URL = environ_get(
     'IMAGE_SRC_URL',
- "http://images.maas.io/ephemeral-v3/daily/streams/v1/index.sjson")
+ "http://images.maas.io/ephemeral-v3/%s/streams/v1/index.sjson" % MAAS_STREAM)
 IMAGE_DIR = environ_get("IMAGE_DIR", "/srv/images")

 KEYRING = environ_get(
@@ -43,7 +43,7 @@ KEYRING = environ_get(
 ITEM_NAME_FILTERS = \
     ['ftype~(boot-initrd|boot-kernel|root-tgz|squashfs)']
 FORMAT_JSON = 'JSON'
-STREAM_BASE = 'com.ubuntu.maas:daily'
+STREAM_BASE = 'com.ubuntu.maas:%s' % MAAS_STREAM
 VMTEST_CONTENT_ID_PATH_MAP = {
     STREAM_BASE + ":v3:download": "streams/v1/vmtest.json",
     STREAM_BASE + ":centos-bases-download": "streams/v1/vmtest-centos.json",

Fixes things for syncing from stable.

To your second question, should curtin use stable or candidate, I believe curtin should test stable primarily and I'd like to add a jenkins job to run candidate.

The challenge is that we can't easily mix the same images repo without a bigger refactor in image_sync.py as the /srv/images/.vmtest_data/* content is coupled to the stream_base.

Depending on how frequently we think we should run against candidate (once a week?) we could sync candidate to a different IMAGE_DIR; and with a fix to accept which stream.