Generated config files are completely wrong

[Thomas Goirand]

The files generated using oslo-config-generator are completely wrong. For example, it is missing [keystone_authtoken] and many more. This shows on the example config in git (ie: etc/glance-api.conf in Glance's git repo).

I believe the generator's config files is missing --namespace keystonemiddleware.auth_token (maybe instead of keystoneclient.middleware.auth_token).

IMO, this is a critical issue, which should be addressed with highest priority. This blocks me from testing Liberty rc1 in Debian.

[Erno Kuvaja (jokke)]

Thanks for the bug Thomas,

Mind to elaborate "and many more.", would make fixing easier to know what else we actually did miss.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/228401

[Ian Cordasco (icordasc)]

Note, this is still a very incomplete bug report. The review that has been submitted is marked as a WIP and will not progress until this does. (Thanks Erno for reacting swiftly.)

[Thomas Goirand (thomas-goirand)]

Obviously, the patch at https://review.openstack.org/228401 is correct, and should be merged.

I still have a glance auth against keystone issue, but IMO, this maybe isn't related. I'll investigate asap.

[Thomas Goirand (thomas-goirand)]

After fixing the generation (the way patch #228401 does) in the Debian package, I still can't do "glance image-list":

# glance image-list
WARNING: The client is falling back to v1 because the accessing to v2 failed. This behavior will be removed in future versions
Invalid OpenStack Identity credentials.

And here's the corresponding output of glance-api.log and keystone.log:

==> /var/log/keystone/keystone.log <==
2015-09-29 09:04:45.554 18789 INFO keystone.common.wsgi [req-0cce70a5-e0fd-4bc2-8f7a-64b3a0f364d2 - - - - -] GET http://117.121.243.214:5000/v2.0/
2015-09-29 09:04:45.556 18789 INFO eventlet.wsgi.server [req-0cce70a5-e0fd-4bc2-8f7a-64b3a0f364d2 - - - - -] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /v2.0/ HTTP/1.1" 200 560 0.002729
2015-09-29 09:04:45.566 18789 INFO keystone.common.wsgi [req-c43c4e7a-bb44-4982-966b-f69c4172a9ee - - - - -] POST http://117.121.243.214:5000/v2.0/tokens

==> /var/log/glance/glance-api.log <==
2015-09-29 09:04:45.715 16450 INFO eventlet.wsgi.server [-] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /versions HTTP/1.1" 200 820 0.000618

==> /var/log/keystone/keystone.log <==
2015-09-29 09:04:45.720 18789 INFO eventlet.wsgi.server [req-c43c4e7a-bb44-4982-966b-f69c4172a9ee - - - - -] 117.121.243.214 - - [29/Sep/2015 09:04:45] "POST /v2.0/tokens HTTP/1.1" 200 3232 0.154477

==> /var/log/glance/glance-api.log <==
2015-09-29 09:04:45.752 16450 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
2015-09-29 09:04:45.756 16450 INFO eventlet.wsgi.server [-] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /v2/schemas/image HTTP/1.1" 401 573 0.038755
2015-09-29 09:04:45.799 16450 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
2015-09-29 09:04:45.810 16450 INFO eventlet.wsgi.server [-] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /v2/schemas/metadefs/namespace HTTP/1.1" 401 573 0.045735
2015-09-29 09:04:45.847 16450 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
2015-09-29 09:04:45.855 16450 INFO eventlet.wsgi.server [-] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /v2/schemas/metadefs/resource_type HTTP/1.1" 401 573 0.044272

==> /var/log/keystone/keystone.log <==
2015-09-29 09:04:45.891 18789 INFO keystone.common.wsgi [req-2634ccb7-7d7b-42b8-997a-3b3d79c75f29 - - - - -] GET http://117.121.243.214:5000/v2.0/
2015-09-29 09:04:45.893 18789 INFO eventlet.wsgi.server [req-2634ccb7-7d7b-42b8-997a-3b3d79c75f29 - - - - -] 117.121.243.214 - - [29/Sep/2015 09:04:45] "GET /v2.0/ HTTP/1.1" 200 560 0.002421
2015-09-29 09:04:45.899 18789 INFO keystone.common.wsgi [req-bf3db99d-08a4-4312-978a-23f85c649bab - - - - -] POST http://117.121.243.214:5000/v2.0/tokens
2015-09-29 09:04:46.012 18789 INFO eventlet.wsgi.server [req-bf3db99d-08a4-4312-978a-23f85c649bab - - - - -] 117.121.243.214 - - [29/Sep/2015 09:04:46] "POST /v2.0/tokens HTTP/1.1" 200 3232 0.113956

==> /var/log/glance/glance-api.log <==
2015-09-29 09:04:46.027 16450 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
2015-09-29 09:04:46.035 16450 INFO eventlet.wsgi.server [-] 117.121.243.214 - - [29/Sep/2015 09:04:46] "GET /v1/images/detail?sort_key=name&sort_dir=asc&...

[Thomas Goirand (thomas-goirand)]

I've found the remaining issue with Glance. It was an issue with http vs https connection to Keystone. Once I set auth_protocol = http then it works. So I guess this bug can be closed.

Sorry for the noise.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228401
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=b1d2d938d282ccd51986e57a638f8ea5bec56b0f
Submitter: Jenkins
Branch: master

commit b1d2d938d282ccd51986e57a638f8ea5bec56b0f
Author: Erno Kuvaja <email address hidden>
Date: Mon Sep 28 10:44:38 2015 +0000

    Return missing authtoken options

    Example configs were missing keystone_authtoken section after moving
    to generated config files. This change returns that to generation.

    Closes-Bug: #1500361

    Change-Id: I6ee82c38061d483cea7254d155d9a72436880e84

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/229672

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229672
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=b7fb5bf0f89f657ead98024ee0168f2c2fa7a776
Submitter: Jenkins
Branch: stable/liberty

commit b7fb5bf0f89f657ead98024ee0168f2c2fa7a776
Author: Erno Kuvaja <email address hidden>
Date: Mon Sep 28 10:44:38 2015 +0000

    Return missing authtoken options

    Example configs were missing keystone_authtoken section after moving
    to generated config files. This change returns that to generation.

    Closes-Bug: #1500361

    Change-Id: I6ee82c38061d483cea7254d155d9a72436880e84
    (cherry picked from commit b1d2d938d282ccd51986e57a638f8ea5bec56b0f)

[Nikhil Komawar (nikhil-komawar)]

RC2 is yet to be out, so I updated the status to Fix Committed from FIx Released. This will be released when RC2 is out; that is either tomorrow (Friday Oct 2) or next week.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/235346

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/235346
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=431fa95ecf92a9ebb7082f829c0a99f958363cc3
Submitter: Jenkins
Branch: master

commit 69516fad5f651a085a047a337a05c58b39023c1b
Author: Mike Fedosin <email address hidden>
Date: Mon Oct 12 15:34:54 2015 +0300

    Add 'deactivated' status to image schema.

    New 'deactivated' status was introduced in Kilo release,
    but it doesn't listed in available image statuses in the schema.

    It leads to issues on the client side, when it can't validate
    the image with this status against the schema and returns the error.

    Change-Id: I5ec264614ae7ecf54b846ad0600442a18c61d24c
    Closes-bug: #1505218
    Related-bug: #1505134

commit c5b6901527b8b4a1250bdc179405c8af66fbae7e
Author: Mike Fedosin <email address hidden>
Date: Tue Oct 13 00:33:27 2015 +0300

    Add testresources and testscenarios used by oslo.db fixture

    If we use oslo.db fixtures, we'll need these 2 packages or
    the next version of oslo.db release will break us.

    Change-Id: I7c0d2f6dabc20bd4ff0d29d3b47b948aa24ea56b
    Closes-Bug: #1503501

commit fc32f0554de0ba7773d98e6828da157ca7c66002
Author: Mike Fedosin <email address hidden>
Date: Sun Sep 20 17:01:22 2015 +0300

    Cleanup chunks for deleted image if token expired

    In patch I47229b366c25367ec1bd48aec684e0880f3dfe60 it was
    introduced the logic that if image was deleted during file
    upload when we want to update image status from 'saving'
    to 'active' it's expected to get Duplicate error and delete
    stale chunks after that. But if user's token is expired
    there will be Unathorized exception and chunks will stay
    in store and clog it.
    And when, the upload operation for such an image is
    completed the operator configured quota can be exceeded.

    This patch fixes the issue of left over chunks for an image
    which was deleted from saving status, by correctly handle
    auth exceptions from registry server.

    Partial-bug: #1498163

    Change-Id: I17a66eca55bfb83107046910e69c4da01415deec

commit ca8d909a61ba335805d8d17070230ce9478a000d
Author: Stuart McLaren <email address hidden>
Date: Wed Sep 30 16:54:12 2015 +0000

    Download forbidden when get_image_location is set.

    When using v2 an attempt to download an image would return a 403 if the
    get_image_location policy was set.

    Note: We had been returning both 404 and 204 when no data was
    available. There was no way to detect the 404 case without trying to
    access the image locations so I've standardized on 204.

    Change-Id: I658b08a35d3a8cb8a7096baf716ccb3d6e7d9abf
    Closes-bug: 1501672
    (cherry picked from commit b47f625443c3b46483506926f31fee42478705d4)

commit ebdf076cc9bd5d9239cdc96c6e7cecc72f852bbb
Author: Mike Fedosin <email address hidden>
Date: Thu Oct 1 18:28:48 2015 +0300

    Catch NotAuthenticated exception in import task

    If glance uses registry as data_api then it's possible
    that token may expire during image import task and Glance
    will have NotUauthenticated exception.

    This code adds...

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/glance 12.0.0.0b1 development milestone.