[SRU] openstack mirror with keystone v3 always imports new images

[Scott Moser]

[Impact]

On every execution of /etc/cron.*/glance_simplestreams_sync simplestreams will upload all the images that match the configured filters no matter of those same images were already uploaded in a previous execution.

This will potentially lead to the inability to upload new images due to not having enough free space.

[Test Case]

* deploy artful-pike
  juju deploy ./artful-pike.yaml # http://paste.ubuntu.com/p/RZqm3cGjqk/

* Wait until glance-simplestreams-sync runs the first sync-up execution.
* Verify the images are in glance running "openstack image list"
* Run the synchronization script again
  juju ssh glance-simplestreams-sync/0 sudo /etc/cron.daily/glance_simplestreams-sync

Expected results:

"openstack image list" prints the same list of images as before running the synchronization script for the 2nd time

Actual result:

"openstack image list" prints a list of images duplicates, e.g.:

$ openstack image list
+--------------------------------------+---------------------------------------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+---------------------------------------------------------------+--------+
| 7f946cbf-57e1-4704-92ea-928d8d4e9454 | auto-sync/ubuntu-trusty-14.04-amd64-server-20180404-disk1.img | active |
| 7a5afbf8-f072-49af-9629-483fc27c627a | auto-sync/ubuntu-trusty-14.04-amd64-server-20180404-disk1.img | active |
| c9a1dfbd-9e5d-4261-b43f-585e65f9733a | auto-sync/ubuntu-xenial-16.04-amd64-server-20180405-disk1.img | active |
| a731c994-61f3-43ea-b86c-227baec101e3 | auto-sync/ubuntu-xenial-16.04-amd64-server-20180405-disk1.img | active |
+--------------------------------------+---------------------------------------------------------------+--------+

[Potential Regression]

* this patch allows simplestreams to connect to swift and verify if the image was already uploaded or not, any possible regression will manifest around the ability of simplestreams to connect to swift.

[Other Info]

When using the newly added (revno 450) v3 support for mirroring to openstack simplestreams will not notice that images already exist in glace. The result is that every 'sync' will import all new images.

The issue was simply that the tenant_id was not being correctly passed through to the glance query.
It ended considering only existing images that were owned by None, which didn't match anything.

Note:
 * This bug is present in Artful when using the v3 keystone api. It is *not* present in xenial or zesty as they do not have v3 keystone support, and the code that is submitted for merge request has the fix included.

[Launchpad Janitor (janitor)]

This bug was fixed in the package simplestreams - 0.1.0~bzr454-0ubuntu1

---------------
simplestreams (0.1.0~bzr454-0ubuntu1) bionic; urgency=medium

  * New upstream snapshot.
    - Support using older swift clients (older than Ocata).
      [David Ames] (LP: #1719879)
    - Fix glance sync always uploading new image with keystone v3 auth.
      [David Ames] (LP: #1728982)
    - tox.ini: Pin versions of style check tools used in flake8 env.
    - flake8: remove usage of bare except.

 -- Scott Moser <email address hidden> Tue, 31 Oct 2017 10:15:41 -0400

[Łukasz Zemczak (sil2100)]

Could this bug be updated to include the SRU-required information in its description? Thank you!

https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

[Łukasz Zemczak (sil2100)]

An upload of simplestreams to artful-proposed has been rejected from the upload queue for the following reason: "No SRU information provided in the bug. No update from thetime. Please re-upload after the required bits are filled-in.".

[Scott Moser (smoser)]

This bug is believed to be fixed in simplestreams in version 0.1.0. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[Rafael David Tinoco (rafaeldtinoco)]

For Xenial SRU, please read summary at:

https://bugs.launchpad.net/ubuntu/+source/simplestreams/+bug/1686437/comments/19

[Robie Basak (racb)]

An upload of simplestreams to xenial-proposed has been rejected from the upload queue for the following reason: "See https://bugs.launchpad.net/charms/+source/glance-simplestreams-sync/+bug/1611987/comments/21".

[Rafael David Tinoco (rafaeldtinoco)]

TL;DR version:

We are going to fix Bionic simplestreams package and provide this Bionic simplestreams package in Ubuntu Cloud Archive for Xenial. With that, we are fixing Xenial simplestreams behavior (to work with keystone v3) only if end-user enables Ubuntu Cloud Archive.

Conversation about this topic:

https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/simplestreams/+git/simplestreams/+merge/373030/comments/980111