[SRU] openstack mirror with keystone v3 always imports new images

Bug #1728982 reported by Scott Moser on 2017-10-31
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
simplestreams
High
David Ames
simplestreams (Ubuntu)
Medium
Unassigned
Xenial
Undecided
Unassigned
Artful
Medium
Unassigned
Bionic
Medium
Unassigned
Disco
Undecided
Unassigned

Bug Description

[Impact]

On every execution of /etc/cron.*/glance_simplestreams_sync simplestreams will upload all the images that match the configured filters no matter of those same images were already uploaded in a previous execution.

This will potentially lead to the inability to upload new images due to not having enough free space.

[Test Case]

* deploy artful-pike
  juju deploy ./artful-pike.yaml # http://paste.ubuntu.com/p/RZqm3cGjqk/

* Wait until glance-simplestreams-sync runs the first sync-up execution.
* Verify the images are in glance running "openstack image list"
* Run the synchronization script again
  juju ssh glance-simplestreams-sync/0 sudo /etc/cron.daily/glance_simplestreams-sync

Expected results:

"openstack image list" prints the same list of images as before running the synchronization script for the 2nd time

Actual result:

"openstack image list" prints a list of images duplicates, e.g.:

$ openstack image list
+--------------------------------------+---------------------------------------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+---------------------------------------------------------------+--------+
| 7f946cbf-57e1-4704-92ea-928d8d4e9454 | auto-sync/ubuntu-trusty-14.04-amd64-server-20180404-disk1.img | active |
| 7a5afbf8-f072-49af-9629-483fc27c627a | auto-sync/ubuntu-trusty-14.04-amd64-server-20180404-disk1.img | active |
| c9a1dfbd-9e5d-4261-b43f-585e65f9733a | auto-sync/ubuntu-xenial-16.04-amd64-server-20180405-disk1.img | active |
| a731c994-61f3-43ea-b86c-227baec101e3 | auto-sync/ubuntu-xenial-16.04-amd64-server-20180405-disk1.img | active |
+--------------------------------------+---------------------------------------------------------------+--------+

[Potential Regression]

* this patch allows simplestreams to connect to swift and verify if the image was already uploaded or not, any possible regression will manifest around the ability of simplestreams to connect to swift.

[Other Info]

When using the newly added (revno 450) v3 support for mirroring to openstack simplestreams will not notice that images already exist in glace. The result is that every 'sync' will import all new images.

The issue was simply that the tenant_id was not being correctly passed through to the glance query.
It ended considering only existing images that were owned by None, which didn't match anything.

Note:
 * This bug is present in Artful when using the v3 keystone api. It is *not* present in xenial or zesty as they do not have v3 keystone support, and the code that is submitted for merge request has the fix included.

Related branches

Scott Moser (smoser) on 2017-10-31
Changed in simplestreams:
status: New → In Progress
importance: Undecided → High
assignee: nobody → David Ames (thedac)
Scott Moser (smoser) on 2017-10-31
Changed in simplestreams:
status: In Progress → Fix Committed
Scott Moser (smoser) on 2017-11-10
Changed in simplestreams (Ubuntu):
status: New → Fix Released
importance: Undecided → High
Changed in simplestreams (Ubuntu Artful):
status: New → Confirmed
Changed in simplestreams (Ubuntu Bionic):
status: Fix Released → Confirmed
Changed in simplestreams (Ubuntu Artful):
importance: Undecided → Medium
description: updated
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package simplestreams - 0.1.0~bzr454-0ubuntu1

---------------
simplestreams (0.1.0~bzr454-0ubuntu1) bionic; urgency=medium

  * New upstream snapshot.
    - Support using older swift clients (older than Ocata).
      [David Ames] (LP: #1719879)
    - Fix glance sync always uploading new image with keystone v3 auth.
      [David Ames] (LP: #1728982)
    - tox.ini: Pin versions of style check tools used in flake8 env.
    - flake8: remove usage of bare except.

 -- Scott Moser <email address hidden> Tue, 31 Oct 2017 10:15:41 -0400

Changed in simplestreams (Ubuntu Bionic):
status: Confirmed → Fix Released
tags: added: cpe-onsite
Łukasz Zemczak (sil2100) wrote :

Could this bug be updated to include the SRU-required information in its description? Thank you!

https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

An upload of simplestreams to artful-proposed has been rejected from the upload queue for the following reason: "No SRU information provided in the bug. No update from thetime. Please re-upload after the required bits are filled-in.".

Felipe Reyes (freyes) on 2018-04-24
description: updated
summary: - openstack mirror with keystone v3 always imports new images
+ [SRU] openstack mirror with keystone v3 always imports new images

This bug is believed to be fixed in simplestreams in version 0.1.0. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in simplestreams:
status: Fix Committed → Fix Released
Changed in simplestreams (Ubuntu Artful):
status: Confirmed → Won't Fix
Changed in simplestreams (Ubuntu Xenial):
status: New → Fix Released
status: Fix Released → In Progress
assignee: nobody → Rafael David Tinoco (rafaeldtinoco)
Changed in simplestreams (Ubuntu Disco):
status: New → Fix Released
Changed in simplestreams (Ubuntu Bionic):
importance: High → Medium
Changed in simplestreams (Ubuntu):
importance: High → Medium

An upload of simplestreams to xenial-proposed has been rejected from the upload queue for the following reason: "See https://bugs.launchpad.net/charms/+source/glance-simplestreams-sync/+bug/1611987/comments/21".

Changed in simplestreams (Ubuntu Xenial):
status: In Progress → Won't Fix
Changed in simplestreams (Ubuntu Bionic):
status: Fix Released → In Progress
Changed in simplestreams (Ubuntu Xenial):
assignee: Rafael David Tinoco (rafaeldtinoco) → nobody
Changed in simplestreams (Ubuntu Bionic):
assignee: nobody → Rafael David Tinoco (rafaeldtinoco)

TL;DR version:

We are going to fix Bionic simplestreams package and provide this Bionic simplestreams package in Ubuntu Cloud Archive for Xenial. With that, we are fixing Xenial simplestreams behavior (to work with keystone v3) only if end-user enables Ubuntu Cloud Archive.

Conversation about this topic:

https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/simplestreams/+git/simplestreams/+merge/373030/comments/980111

Changed in simplestreams (Ubuntu Bionic):
status: In Progress → Fix Released
assignee: Rafael David Tinoco (rafaeldtinoco) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers