flaskification

[Morgan Fainberg]

Moving keystone to Flask away from it's home-grown WSGI framework is a long-term plan. The major reasons for this is to ensure we have an easy way for folks start contributing.

This will include a number of improvements including:

* moving to flask-restful for API definitions

* all routable paths will be owned by the base prefix (e.g. keystone.api.user will own everything under /user/)

* Paste Deploy removed

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/574953

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/574735
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=f18d54fe16dcc91a0d9d144ad15c524a679e3cc5
Submitter: Zuul
Branch: master

commit f18d54fe16dcc91a0d9d144ad15c524a679e3cc5
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 12 06:59:38 2018 -0700

    Store JSON Home Resources off the composing router

    JSON Home Resources must be stored in a location other
    than the router for Flask as we are not composing routers
    in the same way as we used to with the home-grown webob
    based WSGI setup.

    Partial-Bug: #1776504
    Closes-Bug: #1776506
    Change-Id: I292ea9e923ff2f49041dfd417994bcdd797d0520

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/576637

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/576639

[OpenStack Infra (hudson-openstack)]

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/574953

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/578189

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/578190

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/578216

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/574414
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8dd2235cbc08f11036a898342e00bc2a5f13f79d
Submitter: Zuul
Branch: master

commit 8dd2235cbc08f11036a898342e00bc2a5f13f79d
Author: Morgan Fainberg <email address hidden>
Date: Mon Jun 11 09:49:05 2018 -0700

    Add Flask-RESTful and update flask minimum(s)

    Keystone's move to flask requires more than the general Flask library
    as Keystone is a RESTful (ish) API. We will be using Flask-RESTful for
    the easier mechanism to implment a REST API than standard flask
    blueprints.

    This also increases the base flask minimum requirements to unblock
    requirements updates (flask has been updated in g-r to minimum of
    1.0.2)

    Partial-Bug: #1776504
    Change-Id: I398acad439f4e525df3ca4e17fdd3e3ba90d58cc

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/574502
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ecf721a3c176daf67d00536c48e80e78bded1af6
Submitter: Zuul
Branch: master

commit ecf721a3c176daf67d00536c48e80e78bded1af6
Author: Morgan Fainberg <email address hidden>
Date: Mon Jun 11 17:25:45 2018 -0700

    Keystone adheres to public_endpoint opt only

    With the complete removal of the v2.0 API, keystone no longer
    differentiates between admin and public endpoints. This change
    deprecates the "admin_endpoint" configuration option and converts
    keystone over to only using the public endpoint. The "admin" endpoint
    was only used for unit testing purposes.

    This change does not clean up all code related, it is aimed to make
    the most minimal set of changes eliminating the use of the
    "admin_endpoint" configuration option.

    Partial-Bug: #1776504
    Change-Id: I08f6f8ae078d65203bd95c43c80367dd3489be48

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579342

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579746

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579928

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579930

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579985

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/574736
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3e3ba18bfaf82711689a45ae03efdba78e6e856e
Submitter: Zuul
Branch: master

commit 3e3ba18bfaf82711689a45ae03efdba78e6e856e
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 12 06:42:21 2018 -0700

    Convert json_home and version discovery to Flask

    Move the JSON Home Document and Version Discovery Documents out of
    the webob-based mapper and into Flask.

    This change removes the keystone.version.controller and
    keystone.version.router modules as they have been moved into
    keystone.api.discovery.

    The keystone.api.discovery module is somewhat specialized as there
    are no "resources" and it must handle multiple types of responses
    based upon the ACCEPTS header (JSON Home or JSON). In lieu of the
    flask-RESTful mechanisms, keystone.api.discovery utilizes bare
    flask blueprint and functions. Minor scaffolding work has been done
    to ensure the discovery blueprint can be loaded via the loader loop
    in keystone.server.flask.application (a stub object in
    keystone.api.discovery).

    Partial-Bug: #1776504
    Change-Id: Ib25380cefdbb7147661bb9853de7872a837322e0

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/576637
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3330c228ad030e6d3d987f1f360ba5ce0c460333
Submitter: Zuul
Branch: master

commit 3330c228ad030e6d3d987f1f360ba5ce0c460333
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 19 12:55:24 2018 -0700

    Add support for before and after request functions

    In flask it is possible to register functions that will run before
    and after every request, these are useful for ensuring certain
    things occur (setting up environments) and protecting against
    cases such as un-enforced API calls not explicitly white-listed.

    Partial-Bug: #1776504
    Change-Id: I0138a4676c1e257cd0c238bbe11a520446c299cb

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/577587
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1e870abd693088d19db023d731310a1f33f2c831
Submitter: Zuul
Branch: master

commit 1e870abd693088d19db023d731310a1f33f2c831
Author: Morgan Fainberg <email address hidden>
Date: Fri Jun 22 17:27:36 2018 -0700

    Don't replace the whole app just the wsgi_app backing

    Do not replace the entire app when wrapping with middleware. It is
    important to maintain all the flask-functionality on the app object
    and ensure any/all test client calls go through the entire stack of
    app and middleware.

    Partial-Bug: #1776504
    Change-Id: I928d08e96b4c79807ad8c312ba17359c54b67fa0

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/577627
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1bcc8a11c8f3fba70cdc4b8c6992b20a423aba48
Submitter: Zuul
Branch: master

commit 1bcc8a11c8f3fba70cdc4b8c6992b20a423aba48
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 23 14:33:24 2018 -0700

    Make it easy to identify a 404 from Flask

    This change makes (for test purposes) the Flask app return a 418
    instead of a 404 if the path is unrouted. This allows easy
    identification (programmatically) if the 404 is issued from Flask
    or is a handled 404 such as "UserNotFound"

    Parital-Bug: #1776504
    Change-Id: I0475d9b6315250d9c3384be63c14a81fbd1c7b7c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/578216
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=b77b5845d58b64babd9111832e239f47fc4766a2
Submitter: Zuul
Branch: master

commit b77b5845d58b64babd9111832e239f47fc4766a2
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 14:46:01 2018 -0700

    Address minor comments to 404 error detection

    Addressing minor comments to the 404 detection mechanism.

    Change-Id: Id8e2b5d5f811495694965d75a32eebb2a59779f2
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582449

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582450

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/576639
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bb3b15bbf02993545d234d827dbb72ca24695e8d
Submitter: Zuul
Branch: master

commit bb3b15bbf02993545d234d827dbb72ca24695e8d
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 23 14:33:50 2018 -0700

    Implement base for new RBAC Enforcer

    Create the base implementation of the RBAC enforcer with compat code
    for the legacy mechanism via @protected decorators.

    Change-Id: I80662d9b23e706b720d56670cb849318e951a3b4
    Parital-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/578189
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=fb0299f661f590bcaddcb3d363fbec4ea6df7548
Submitter: Zuul
Branch: master

commit fb0299f661f590bcaddcb3d363fbec4ea6df7548
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 10:26:20 2018 -0700

    Add support for enforce_call to set value on flask.g

    When enforce_call is called, flask.g has the appropriate value set
    in the known attribute location so that it can be determined if
    enforce_call was used within a REST API request.

    Change-Id: I08ecd2be0a80248df7041596437adb6238835153
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579746
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1caba2a44858ab4a6d405603cd357bb1c5e51a77
Submitter: Zuul
Branch: master

commit 1caba2a44858ab4a6d405603cd357bb1c5e51a77
Author: Morgan Fainberg <email address hidden>
Date: Sun Jul 1 10:26:15 2018 -0700

    Move keystone.server.common to keystone.server

    The common functions are mostly removed and these make sense to
    be part of the __init__.py. This also simplifies imports and
    eliminates odd import errors due to duplicated names and masking
    the names e.g. (and keystone.server.flask needs the functions
    from keystone.server.common):

    keystone.server.flask.common
    keystone.server.common

    Change-Id: Ie586fd45e10c8a1c8db3d9a64f949c97004814d9
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579342
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=6d1456061defc016529b3c7c89edb7618b117267
Submitter: Zuul
Branch: master

commit 6d1456061defc016529b3c7c89edb7618b117267
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 30 08:17:33 2018 -0700

    Address minor comments from initial impl RBACEnforcer

    RBACEnforcer had a couple minor comments and this addresses all
    except for adding explicit system-scope enforcement tests.

    * Invalid "action" (policy rule name) now correctly raises
      Forbidden instead of Unauthorized (it can never be authorized
      as the code is incorrect, and a 500 "ISE" should never be expected).
      This is equivalent behavior to an unknown rule in the oslo.policy
      enforcer.

    * Added more expressive errors/warnings/exceptions for invalid actions.

    * Code comments added where missing

    Change-Id: I4895242555d2e9761e035d879ca2dfeb6d9b387c
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582724

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582725

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582726

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/578190
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=16be22b428fd0c7c34dd48417878f7d37ad1e2ac
Submitter: Zuul
Branch: master

commit 16be22b428fd0c7c34dd48417878f7d37ad1e2ac
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 13:02:01 2018 -0700

    Flesh out and add testing for flask_RESTful scaffolding

    Add in support for JSON Home documents, a ResourceBase implementing
    basic functionality, and full testing of the new flask_RESTful
    scaffolding.

    Change-Id: I5bcc8660b68c0b39a2110089f6c67531769d14ef
    Parital-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579928
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9387dfd4cc39fa6fbbf66147ced880dffa6408bd
Submitter: Zuul
Branch: master

commit 9387dfd4cc39fa6fbbf66147ced880dffa6408bd
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 10:10:53 2018 -0700

    Make keystone.server.flask more interesting for importing

    Importing keystone.server.flask now exposes all the relevant bits
    from the sub modules to develop APIs without needing to understand
    all the underlying modules. __all__ has also be setup in a meaningful
    way to allow for `from keystone.server.flask import *` and have
    all the needed objects to start developing APIs for keystone.

    Change-Id: Iab22cabb71c6690e6ffb0c9de68ed8437c4848de
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579930
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0f4fd2e4f38d4e5e6aea5dfd7c3d124e8965edb0
Submitter: Zuul
Branch: master

commit 0f4fd2e4f38d4e5e6aea5dfd7c3d124e8965edb0
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 10:12:40 2018 -0700

    Fix keystone.common.rbac_enforcer.__init__.py exporting

    keystone.common.rbac_enforcer now exports (and has __all__ setup)
    for the RBACEnforcer object. This limits the underlying module
    knowledge needed to effectively work with the new flask RBACEnforcer.

    Change-Id: I48cafa779ca1895113222a2ebfef6d738a906473
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579985
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=dbc2ac06b8236c446a4bb2817add4f253bf2034d
Submitter: Zuul
Branch: master

commit dbc2ac06b8236c446a4bb2817add4f253bf2034d
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 13:54:40 2018 -0700

    Do not use flask.g imported as g

    Always explicitly use flask.g as `flask.g` and do not do `from flask
    import g`.

    Change-Id: I3737115717d8a5040c3e299d206fbddd43b33aa6
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583278

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583356

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583357

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583358

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583373

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583374

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/582449
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0d6b427fcc7f6b9b2a06bdd9a7af37064e7ddd51
Submitter: Zuul
Branch: master

commit 0d6b427fcc7f6b9b2a06bdd9a7af37064e7ddd51
Author: Morgan Fainberg <email address hidden>
Date: Thu Jul 12 16:41:10 2018 -0700

    Don't allow legacy and native flask to share paths

    When a path-prefix is moved to flask native dispatching, no longer
    allow that path prefix to be registered with the legacy dispatch
    middleware. This will ensure the entire Keystone path is moved
    and prevent bad behavior due to both dispatchers needing ot handle
    a URL.

    Change-Id: Ice800abf80a725349d6450b742a2c48238e11e6e
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/603451

[OpenStack Infra (hudson-openstack)]

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/602808

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/603461

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/603964

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/601881
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8697da7da650e2db29136ce48053ee7c234e3b62
Submitter: Zuul
Branch: master

commit 8697da7da650e2db29136ce48053ee7c234e3b62
Author: Gage Hugo <email address hidden>
Date: Tue Sep 11 17:26:24 2018 -0600

    Add build_target arguement to enforcer

    This change adds in a new arguement "build_target" which takes
    in a passed function to build the enforement target after
    the authentication check. This is to avoid leaking existance
    data when determining scope.

    Change-Id: I9aab71dd0032d40aa2f2e088b529af08b112671f
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/603964
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d027dd2ba9134629e067b24503cc7f61d75b692d
Submitter: Zuul
Branch: master

commit d027dd2ba9134629e067b24503cc7f61d75b692d
Author: wangxiyuan <email address hidden>
Date: Thu Sep 20 11:25:21 2018 +0800

    Add hint back

    When do Flask refactor, some list APIs missed "hints"
    for wrap_collection.

    Add them back for list resource so that filters can work as
    before

    Change-Id: If32ca8d798c95436f72aa55ac7060e8eb7540de7
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/601882
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0c71cdd23bd2a7e4f7ec1a5ecec91f3ed7457d00
Submitter: Zuul
Branch: stable/rocky

commit 0c71cdd23bd2a7e4f7ec1a5ecec91f3ed7457d00
Author: morgan fainberg <email address hidden>
Date: Tue Sep 11 16:03:54 2018 -0700

    Ensure view args is in policy dict

    The policy_dict (in enforcement) was not populating the view args
    in a similar manner to the old style @protected decorator. This
    change ensures that we mirror the old behavior (required for
    proper use of v3cloud policy).

    Conflicts:
        keystone/tests/unit/common/test_rbac_enforcer.py

    Change-Id: Ida9009a95a874be9cc60c3152d4e3225726562eb
    Partial-Bug: #1776504
    Closes-Bug: #1792047
    (cherry picked from commit 4975b79e8174587f7639347939cf679460d4896b)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/608756

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609071

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609159

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609500

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609535

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609548

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/603461
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d97832e8e826e37171b727072c720a9b589998dd
Submitter: Zuul
Branch: master

commit d97832e8e826e37171b727072c720a9b589998dd
Author: morgan fainberg <email address hidden>
Date: Tue Sep 18 10:54:59 2018 -0700

    Convert auth to flask native dispatching

    Convert the /auth paths to flask native dispatching.

    A minor change to additional_urls was implemented to ensure all
    urls are added at once instead of individually (causing an over-
    write issue within flask as a single resource may only have a
    single set of URL mappings).

    Alternate URLs now support adding alternate JSON Home rel links.
    This is to support the case of OS-FEDERATION auth routes moving
    to /auth. The old JSON Home entries must exist but reference
    the new paths.

    This port includes the following test changes (needed due to the
    way flask handles requests and the way requests are passed through
    the auth system):

    * Implemented keystone.common.render_token (module)
      containing render_token_response_from_model and use it instead
      of keystone.common.controller.render_token_response_from_model.

      Minor differences occur in render_token_response_from_model in
      the keystone.common.render_token module, this is simply
      for referencing data from flask instead of the request object.

    * Test cases have been modified to no longer rely on the auth
      controller(s) directly

    * Test cases now use "make_request" as a context manager
      since authenticate/authenticate_for_token directly
      reference the flask contexts and must have an explicit
      context pushed.

    * Test cases no longer pass request objects into methods
      such as authenticate/authenticate_for_token or similar
      methods on the auth plugins

    * Test cases for federation reference the token model now
      where possible instead of the rendered token response.
      Rendered token responses are generated where needed.

    * Auth Plugin Configuration is done in test core as well.
      This is because Auth controller does not exist.

    NOTE: This is a massive change, but must of these changes
    were now easily uncoupled because of how far reaching auth
    is.

    Change-Id: I636928102875760726cc3493775a2be48e774fd7
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/608756
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=54b6227c1fdec1e768570ba696841b00f4e0a549
Submitter: Zuul
Branch: master

commit 54b6227c1fdec1e768570ba696841b00f4e0a549
Author: Morgan Fainberg <email address hidden>
Date: Mon Oct 8 12:28:42 2018 -0700

    Auth flask conversion cleanup

    * Superfluous call to setup token authentication has been removed from
      keystone.server.flask.core

    * Base SAML assertion function has been extracted from
      keystone.api.auth and moved to keystone.api._shared.saml

    Change-Id: Idfa62bf1aea81ef5b4c6f564397e6a0d3ae60417
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609796

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609804

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609805

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609815

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609836

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609837

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609838

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/609839

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/610129

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609071
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=86f968163ea90dfac515b5e59f20edf8c5554cee
Submitter: Zuul
Branch: master

commit 86f968163ea90dfac515b5e59f20edf8c5554cee
Author: Morgan Fainberg <email address hidden>
Date: Mon Oct 8 14:40:56 2018 -0700

    Convert /v3/users to flask native dispatching

    Convert /v3/users to use flask native dispatching.

    The following test changes were required:

    * Application Credentials did not have the plural form
      in the JSON Home document. The JSON Home document was
      corrected both in code and in tests.

    * Application Credentials "patch" test needed to be
      refactored to look for METHOD_NOT_ALLOWED instead
      of NOT FOUND for invalid/unimplemented methods.
      The "assertValidErrorResponse" method was
      insufficient and the test now uses the flask
      test_client mechanism instead.

    Change-Id: Iedaf405d11450b11e2d1fcdfae45ccb8eeb6f255
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/603451
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=46380baeb76fc624bf8b286ce403656767c403cf
Submitter: Zuul
Branch: master

commit 46380baeb76fc624bf8b286ce403656767c403cf
Author: Harry Rybacki <email address hidden>
Date: Tue Sep 18 12:03:52 2018 -0400

    Convert projects API to Flask

    Convert the projects API to Flask native dispatching.

    Change-Id: I3406284acfb7950b701f6a98a3a173a427415f97
    Co-Authored-By: Morgan Fainberg <email address hidden>
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609159
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0ab08e75b237dd287554e5c86642efcf8292396c
Submitter: Zuul
Branch: master

commit 0ab08e75b237dd287554e5c86642efcf8292396c
Author: Morgan Fainberg <email address hidden>
Date: Tue Oct 9 14:36:20 2018 -0700

    Remove skip for test_locked_out_user_sends_notification

    Remove the explicit skip added pre-user conversion to flask for
    test_locked_out_user_sends_notification.

    Change-Id: I5f2e83f24dca2774fd5abaacf5885a2dab1ae759
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609500
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=35c9bb7eff51ebd6101b4a1d233b984cbf68fca8
Submitter: Zuul
Branch: master

commit 35c9bb7eff51ebd6101b4a1d233b984cbf68fca8
Author: Morgan Fainberg <email address hidden>
Date: Tue Oct 9 17:15:50 2018 -0700

    Convert S3 and EC2 auth to flask native dispatching

    Convert S3 and EC2 auth to flask native dispatching.

    Test changes required:

    * Eliminate direct reference of the EC2 / S3 controllers, originally
      this direct reference was to verify signature checking. Since
      signature checking is an @staticmethod now, direct reference of
      the API resources covers everything.

    * Direct import of keystone.common.controller - due to an oddity in
      how our WSGI code work(s) in test, if nothing imports the common
      controller module, the tests fail using the oslo import_class
      mechanism.

    Change-Id: I06e95957b3ea3a55b0da28959548bd5eb628c70b
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609535
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ee9b035cf17f0070dfb392505e44ea2961e2cf4b
Submitter: Zuul
Branch: master

commit ee9b035cf17f0070dfb392505e44ea2961e2cf4b
Author: Morgan Fainberg <email address hidden>
Date: Wed Oct 10 13:12:32 2018 -0700

    Replace JSON Body middleware with flask-native func

    Replace the JSON Body middleware with flask-native before-request
    function.

    The body filtering and storing data in
    request.environ['openstack.params'] was not used in the code base and
    has been dropped.

    Test Changes:

    * JSON Body middleware has been removed, no testing of the removed code

    * JSON Body Before Request Method has been implemented and associated
      testing (mirroring the JSON Body middleware code).

    * Test entry points no longer looks for JSON Body middleware.

    Change-Id: I84491865870b6bf2b8f094b524ee8b77510f0054
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609548
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=e666839bc170ee3a529e11a78500d34357c8f6ff
Submitter: Zuul
Branch: master

commit e666839bc170ee3a529e11a78500d34357c8f6ff
Author: Morgan Fainberg <email address hidden>
Date: Wed Oct 10 14:23:12 2018 -0700

    Cleanup keystone.server.flask.application

    Remove a chunk of the compat code for legacy dispatching. This moves
    the logging about the request to it's own before_request function.

    Change-Id: I0b1a4ca9a95489e410f055ff47f3399feba3a8f1
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609796
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9b052e4c059893791dd7adc4aa32d40ac098e788
Submitter: Zuul
Branch: master

commit 9b052e4c059893791dd7adc4aa32d40ac098e788
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 12:32:59 2018 -0700

    Register exceptions with a Flask Error Handler

    Exceptions are now handled in the Flask APP instead of in the
    legacy webob Application code (at this point that code was living
    in the URL Normalizing Middleware). All Keystone API exceptions
    (derived from keystone.exception.Error) are automatically
    registered on definition with the
    keystone.exception.KEYSTONE_API_EXCEPTIONS set. This set is
    processed once the app is created in keystone.server.application
    to the flask-friendly handler.

    TypeError and generic Exception are registered to an explicit
    error handler that converts TypeError to ValidationError (BAD_REQUEST)
    and all other Exceptions to UnexpectedError (INTERNAL SERVER ERROR).
    These exceptions are then emitted in a "jsonify-ed" manner to the
    client.

    Two other minor changes were required:

    * Unenforced API decorator had it's core functionality split into
      a dedicated function that can be called in the case of an error
      being raised in a "before_request" function (such as validation
      in the JSON Body before request func.

    * The JSON Body before request func now explicitly sets the
      api to "unenforced_ok" if it is raising an exception. This
      prevents the flask "was this API enforced" assertion from failing
      because @unenforced_api was never run (the ValidationError was
      raised prior to the resource's method being called).

    Change-Id: I0d0ef6a774eb86b4769238ed34d7703232ce86c3
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609804
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1c05f5f88d84d90ca949bab6e3d48c5f1e35a909
Submitter: Zuul
Branch: master

commit 1c05f5f88d84d90ca949bab6e3d48c5f1e35a909
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:05:16 2018 -0700

    Make Request Logging a little better

    Use the flask.request properties instead of direct environ lookups,
    as this is more representative of what is happening in the application.

    Change-Id: Ic16c5ea26b2f526b51ef167e6f6977c72df1d06a
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609805
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=18d597f8e87a7ebee6aba920e11d64bab6dc0295
Submitter: Zuul
Branch: master

commit 18d597f8e87a7ebee6aba920e11d64bab6dc0295
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:20:36 2018 -0700

    Internally defined middleware don't use stevedore

    For internally defined middleware (URL Normalizer and AuthContext)
    Do not use stevedore to load, apply directly. This also cleans up
    a lingering entry in the setup.cfg for token_auth.

    Test Changes:

    * entry points test no longer looks for url_normalize and
    build_auth_context

    Change-Id: I58d3c23ad4f70668ada4eae94a94d3f5fe750b3b
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609815
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=848c8fa6385f690b26d22a28cafeb41c3e36b558
Submitter: Zuul
Branch: master

commit 848c8fa6385f690b26d22a28cafeb41c3e36b558
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:47:18 2018 -0700

    Convert Normalizing filter to flask native Middleware

    Normalizing filter has been converted to a flask-native style
    middleware instead of leaning on the old application logic from
    Webob. We also now strip all trailing slashes, not just a single
    traling slash.

    Test Changes:

    * test_url_middleware now tests the new middleware directly instead
      of leaning on webob and fake requests.

    Change-Id: I5f82817b61a9284b97cf6443105107150d4a1757
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609836
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=595967bba6c19b709743c2991545662fce9f6da7
Submitter: Zuul
Branch: master

commit 595967bba6c19b709743c2991545662fce9f6da7
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:59:58 2018 -0700

    Move AuthContextMiddleware

    Move AuthContextMiddleware to keystone.server.flask.request_processing
    to be more in line with the other internally defined middleware.

    Change-Id: I25b6a88f4b0dc3af306360ee4e5ec0abfe3cf812
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609837
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8a6f1759331cef666279192de1fa3ecd10de702f
Submitter: Zuul
Branch: master

commit 8a6f1759331cef666279192de1fa3ecd10de702f
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 14:06:57 2018 -0700

    Flask comment/docstring cleanup

    Address a few nits in docstrings and comments from the flask conversion
    patches.

    Change-Id: I058d50168c8e5fa566bd98d7dba101ae9e4f1684
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609838
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ce93950f44b76b409a59d346758360df2a94cdc5
Submitter: Zuul
Branch: master

commit ce93950f44b76b409a59d346758360df2a94cdc5
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 14:30:54 2018 -0700

    Cleanup test_wsgi

    Cleanup test_wsgi and remove un-needed tests. Rename to test_app_config
    to more closely match what is, in-fact, tested now.

    Change-Id: Ifb9dd311da85597d03671a49cf43db43888ad869
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/610129
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=5c70aef2dacf801ccc147be4450e5985f545a855
Submitter: Zuul
Branch: master

commit 5c70aef2dacf801ccc147be4450e5985f545a855
Author: Morgan Fainberg <email address hidden>
Date: Fri Oct 12 10:30:04 2018 -0700

    Make collection_key and member_key raise if unset

    Instead of populating with __UNUSED__ or other silly string, make
    direct use of "collection_key" or "member_key" raise a ValueError
    if they are unset and referenced.

    Change-Id: Idf4f4df9d933317fff96a474cdf23d758ebdfa8c
    Partial-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/609839
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=184c84ae76b1e4dc6b9907caf0cfa0d65bf8ecc5
Submitter: Zuul
Branch: master

commit 184c84ae76b1e4dc6b9907caf0cfa0d65bf8ecc5
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 15:16:02 2018 -0700

    Remove pre-flask legacy code

    This removes common.controller, common.extension, common.router, and
    common.wsgi. Relevant code from common.wsgi (used by AuthContext) was
    moved into keystone.server.flask.request_processing.middleware.auth_context.

    keystone.api.discovery now uses keystone.flask.base_url

    test_middleware and test_exception were modified to reflect the changes
    to the remaining code from keystone.common.wsgi

    keystone.common.authorization only holds a couple constants for auth
    work now.

    Routes is removed from requirements.txt

    Release-Note for migration to flask added.

    Change-Id: I81563b6a49c8f12ecade058a9483f3b6f070dc72
    Closes-Bug: #1776504

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 15.0.0.0rc1 release candidate.