S3 API Broken For >= Kilo

[Edward Hope-Morley]

Seems we got ahead of ourselves in commit 25a2ea7f and broke the S3 api by changing the swift3 middleware config to something that is not supported until the Newton version (1.11) of swift-plugin-s3. In the swift-proxy charm for >= Kilo we currently have:

--------------------
[filter:s3token]
paste.filter_factory = keystonemiddleware.s3_token:filter_factory
auth_uri = {{ auth_protocol }}://{{ keystone_host }}:{{ auth_port }}

[filter:swift3]
use = egg:swift3#swift3
--------------------

but that is broken in (at least) two ways; firstly swift3 (swift-plugin-s3) didn't switch to use auth_uri until 1.11 which is not available until Newton and secondly, when you switch to 1.11 you then need to change the s3token config to:

--------------------
[filter:s3token]
use = egg:swift3#s3token
auth_uri = http://127.0.0.1:35357/

[filter:swift3]
use = egg:swift3#swift3
--------------------

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/527763

[Edward Hope-Morley (hopem)]

Ok so there is an added issue here in that successive releases of ubuntu have newer versions of swift-plugin-s3 BUT the Ubuntu cloud archive does not (until Queens which has 1.12-1). So, I will need to modify my patch so that the same config is used until Queens and change at a later time for prior Xenial UCA releases should this ever get resolved.

[0] https://launchpad.net/ubuntu/+source/swift-plugin-s3
[1] http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/queens_versions.html

[Edward Hope-Morley (hopem)]

And more issues... It seems that since Newton->Pike UCA do have newer deps such as python-keystonemiddleware it is not be (easily) possible to fix this properly for those versions. So I can and will fix for Kilo -> Mitaka but I will have to leave Newton -> Pike as they are for now and hopefully Queens will be released with working config.

[Edward Hope-Morley (hopem)]

And it seems that Trusty Mitaka UCA will also fall foul to this until the UCA packages are updated.

[James Page (james-page)]

Raising tasks for the UCA; Queens is already underway.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/527763
Committed: https://git.openstack.org/cgit/openstack/charm-swift-proxy/commit/?id=a256263c79236be8adc0ea5176c0e4512c5a135b
Submitter: Zuul
Branch: master

commit a256263c79236be8adc0ea5176c0e4512c5a135b
Author: Edward Hope-Morley <email address hidden>
Date: Wed Dec 13 19:10:56 2017 +0000

    Fix S3 API for >= Kilo

    From Kilo onwards the swift-proxy charm is
    misconfiguring the swift3 middleware such that
    the api is unable to respond to any requests.
    We fix this by providing working config for
    Kilo onwards.

    NOTE: see LP for full explanation but due to
    problems with package version mismatches in the
    UCA this patch only fixes Trusty Kilo, (L is
    EOL) and Xenial Mitaka.

    Change-Id: Ice5690e7f06ffc78dd20b53b67dffc6bd72b2613
    Closes-Bug: 1738063

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/17.11
Review: https://review.openstack.org/528221

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528221
Committed: https://git.openstack.org/cgit/openstack/charm-swift-proxy/commit/?id=37cfb1c504f1dbde40ff619b654355352618c939
Submitter: Zuul
Branch: stable/17.11

commit 37cfb1c504f1dbde40ff619b654355352618c939
Author: Edward Hope-Morley <email address hidden>
Date: Wed Dec 13 19:10:56 2017 +0000

    Fix S3 API for >= Kilo

    From Kilo onwards the swift-proxy charm is
    misconfiguring the swift3 middleware such that
    the api is unable to respond to any requests.
    We fix this by providing working config for
    Kilo onwards.

    NOTE: see LP for full explanation but due to
    problems with package version mismatches in the
    UCA this patch only fixes Trusty Kilo, (L is
    EOL) and Xenial Mitaka.

    Change-Id: Ice5690e7f06ffc78dd20b53b67dffc6bd72b2613
    Closes-Bug: 1738063
    (cherry picked from commit a256263c79236be8adc0ea5176c0e4512c5a135b)

[Corey Bryant (corey.bryant)]

Hi Edward,

I've backported the following to corresponding cloud archive proposed pockets for testing:

newton: swift-plugin-s3 1.10-3~cloud0
ocata: swift-plugin-s3 1.11-2~cloud0
pike: swift-plugin-s3 1.11-2~cloud0

Thanks,
Corey

[Edward Hope-Morley (hopem)]

@corey.bryant thanks I will test those today. Don't forget that we also need to make Trusty Mitaka UCA parity with Xenial (which has 1.10-1)

[Edward Hope-Morley (hopem)]

@corey.bryant actually I see in http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/mitaka_versions.html that 1.10-1~cloud0 was uploaded but it failed to build (and weirdly is reported as existing in -proposed even though it is not). I also note that the N/O/P builds are are in -staging but not -proposed.

[Corey Bryant (corey.bryant)]

There was a hash mismatch in the cloud archive with ocata and pike having the same versions. That's been fixed and the packages are now syncing to proposed.

I need to look into the build failure for Mitaka.

[Corey Bryant (corey.bryant)]

swift-plugin-s3 1.10-1~cloud1 is available now in mitaka-proposed.

[Corey Bryant (corey.bryant)]

Newton is end of life.