regression, sync-images action no longer working

[Frode Nordahl]

After commit 009c8a7b929c9b961c9cf388c122cffe6f6ae41c was applied to master the sync-images action is no longer working.

The same behavior can be displayed by executing commands manually on the unit:

Executing this will return immediately without performing any work:
# /usr/share/glance-simplestreams-sync/glance-simplestreams-sync.sh
/usr/share/glance-simplestreams-sync/glance_simplestreams_sync.py:119: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  confobj = yaml.load(f)
/usr/lib/python3/dist-packages/keystoneauth1/adapter.py:235: UserWarning: Using keystoneclient sessions has been deprecated. Please update your software to use keystoneauth1.
  warnings.warn('Using keystoneclient sessions has been deprecated. '
/usr/share/glance-simplestreams-sync/glance_simplestreams_sync.py:119: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  confobj = yaml.load(f)

# cd /usr/share/glance-simplestreams-sync/
# patch -Rp2 < revert the glance_simplestreams_sync.py part of 009c8a7b929c9b961c9cf388c122cffe6f6ae41c

# /usr/share/glance-simplestreams-sync/glance-simplestreams-sync.sh
/usr/share/glance-simplestreams-sync/glance_simplestreams_sync.py:109: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  confobj = yaml.load(f)
/usr/lib/python3/dist-packages/keystoneauth1/adapter.py:235: UserWarning: Using keystoneclient sessions has been deprecated. Please update your software to use keystoneauth1.
  warnings.warn('Using keystoneclient sessions has been deprecated. '
created 50a060cb-6f58-4207-8255-be5befb9adc8: auto-sync/ubuntu-trusty-14.04-amd64-server-20191107-disk1.img
created cbb70087-2b1e-404d-bc52-c6944eed218b: auto-sync/ubuntu-xenial-16.04-amd64-server-20210429-disk1.img
created 91be131b-3052-4b7e-b632-e6beb1d78f4c: auto-sync/ubuntu-bionic-18.04-amd64-server-20210817-disk1.img
created 69a9d41d-9c27-40fa-8edd-b19a98dbccea: auto-sync/ubuntu-focal-20.04-amd64-server-20210825-disk1.img
sending incremental file list
streams/
streams/v1/
streams/v1/auto.sync.json
streams/v1/index.json

sent 1,579 bytes received 66 bytes 3,290.00 bytes/sec
total size is 4,502 speedup is 2.74
/usr/share/glance-simplestreams-sync/glance_simplestreams_sync.py:109: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  confobj = yaml.load(f)

[Dmitrii Shcherbakov (dmitriis)]

Successfully ran gss func tests (focal-ussuri) with the following proxy settings applied right after Zaza created the model:

juju model-config -m zaza-0c1292704187 | grep proxy
apt-ftp-proxy default ""
apt-http-proxy default ""
apt-https-proxy default ""
apt-no-proxy default ""
ftp-proxy default ""
http-proxy default ""
https-proxy default ""
juju-ftp-proxy default ""
juju-http-proxy model http://squid.internal:3128
juju-https-proxy model http://squid.internal:3128
juju-no-proxy default 127.0.0.1,localhost,::1
no-proxy default 127.0.0.1,localhost,::1
proxy-ssh default false
snap-http-proxy default ""
snap-https-proxy default ""
snap-store-proxy default ""
snap-store-proxy-url default ""

[Dmitrii Shcherbakov (dmitriis)]

Juju status for #1

[Dmitrii Shcherbakov (dmitriis)]

/var/log/glace-simplestreams-sync.log for #1

[Dmitrii Shcherbakov (dmitriis)]

/var/log/glance-simplestreams-sync.log after running `juju run-action glance-simplestreams-sync/0 sync-images --wait` with proxy settings reset.

[Frode Nordahl (fnordahl)]

Thx for checking, that is interesting. I guess one difference in the other gates using this is that they do not have ceph nor ceph-radosgw and as such set the use_swift configuration option to 'false'. Could that be relevant?

[Dmitrii Shcherbakov (dmitriis)]

Should help: https://review.opendev.org/c/openstack/charm-glance-simplestreams-sync/+/806579

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-glance-simplestreams-sync/+/806579
Committed: https://opendev.org/openstack/charm-glance-simplestreams-sync/commit/d1f8c4ba4cec4324ef9edc91c357672349176dfc
Submitter: "Zuul (22348)"
Branch: master

commit d1f8c4ba4cec4324ef9edc91c357672349176dfc
Author: Dmitrii Shcherbakov <email address hidden>
Date: Mon Aug 30 16:17:32 2021 +0300

    Remove local apache2 install, rework use_swift

    The functionality of use_swift where a local apache2 instance was set up
    for metadata was deprecated for removal and this change actually removes
    this. However, the option itself can still be used as an indicator
    whether Swift is intended to be used for simplestreams metadata or not.
    If the usage is explicitly requested, the charm but Swift endpoints are
    not yet present, the charm will now will set the unit state to
    maintenance in addition to local logging that was done previously.

    Swift presence checking is now endpoint-based instead of being
    service-based (no use for the case where endpoints are not set up while
    the service is in the catalog).

    The lack of swift presence is also ignored for the purposes of
    generating proxy setting context - if the endpoint is not there there is
    not point in trying to generate NO_PROXY rules for it.

    This change also makes test bundles use ceph as a storage medium for
    glance since out of space errors were encountered during testing as the
    glance unit was running out of space for image storage.

    Closes-Bug: #1942047
    Closes-Bug: #1934563
    Closes-Bug: #1938069

    Change-Id: I6519b1449806ad19ee4545501bc4631c9f8e535f

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/21.04
Review: https://review.opendev.org/c/openstack/charm-glance-simplestreams-sync/+/810811

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-glance-simplestreams-sync/+/810811
Committed: https://opendev.org/openstack/charm-glance-simplestreams-sync/commit/87d8f0b30909702c67799c27f985d6b325d4fc0e
Submitter: "Zuul (22348)"
Branch: stable/21.04

commit 87d8f0b30909702c67799c27f985d6b325d4fc0e
Author: Dmitrii Shcherbakov <email address hidden>
Date: Mon Aug 30 16:17:32 2021 +0300

    Remove local apache2 install, rework use_swift

    The functionality of use_swift where a local apache2 instance was set up
    for metadata was deprecated for removal and this change actually removes
    this. However, the option itself can still be used as an indicator
    whether Swift is intended to be used for simplestreams metadata or not.
    If the usage is explicitly requested, the charm but Swift endpoints are
    not yet present, the charm will now will set the unit state to
    maintenance in addition to local logging that was done previously.

    Swift presence checking is now endpoint-based instead of being
    service-based (no use for the case where endpoints are not set up while
    the service is in the catalog).

    The lack of swift presence is also ignored for the purposes of
    generating proxy setting context - if the endpoint is not there there is
    not point in trying to generate NO_PROXY rules for it.

    This change also makes test bundles use ceph as a storage medium for
    glance since out of space errors were encountered during testing as the
    glance unit was running out of space for image storage.

    Closes-Bug: #1942047
    Closes-Bug: #1934563
    Closes-Bug: #1938069

    Change-Id: I6519b1449806ad19ee4545501bc4631c9f8e535f
    (cherry picked from commit d1f8c4ba4cec4324ef9edc91c357672349176dfc)