Comment 6 for bug 1583276

This is easily reproducible, and it behaves the same with python-glanceclient 2.0.0 from xenial.

If I kick off the glance-simplestreams-sync.py script, and restart glance-api right after message along the lines of

  DEBUG * 05-19 09:16:22 [PID:36034] * glanceclient.common.http * curl -g -i -X POST -H 'x-image-meta-property-source_content_id: com.ubuntu.cloud:released:download' -H 'x-image-meta-property-version_name: 20160516' -H 'x-image-meta-property-os_version: 14.04' -H 'x-image-meta-checksum: e903d04033149e800fc90400dee1abb8' -H 'x-image-meta-container_format: bare' -H 'x-image-meta-property-content_id: auto.sync' -H 'x-image-meta-property-architecture: x86_64' -H 'Accept: */*' -H 'X-Auth-Token: {SHA1}906818b8a051308fa657b016479038d65e086b8c' -H 'x-image-meta-property-os_distro: ubuntu' -H 'x-image-meta-property-item_name: disk1.img' -H 'x-image-meta-property-product_name: com.ubuntu.cloud:server:14.04:amd64' -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'x-image-meta-is_public: True' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 259850752' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-disk_format: qcow2' -H 'x-image-meta-name: auto-sync/ubuntu-trusty-14.04-amd64-server-20160516-disk1.img' http://10.44.203.31:9292/v1/images

shows up in /var/log/glance-simplestreams-sync.log, I get the above CommunicationError, and image is left-over in a "saving" state (also visible from "Images" tab in the Openstack Dashboard).

Re-running the glance-simplestreams-sync script doesn't "fix" this particular image.

Now, let's see what should be done:

 1. Ideally, glance itself would drop the image that has not been fully uploaded (in glance/api/v1/images.py: _upload_and_activate method - it attempts to do that, but I am guessing that service restart is not that graceful and just outright kills the process)
 2. python-simplestreams could treat "status" != "active" images as incomplete and re-download and re-publish them.

I've tested a patch to do 2 (https://pastebin.canonical.com/156799/), and when re-running the gs3 script, it results in the "saving" image remaining there, but a correct image that matches the "saving" one appearing as "active" one (http://people.canonical.com/~danilo/tmp/images-saving+active.png). I'll propose this for inclusion in python-simplestreams, but not before I do another round of testing, and ensure juju bootstrap still works even with such incomplete images potentially listed in the index file (I hope they aren't, though).