Sync new policy from oslo

[Shuangtai Tian]

The oslo has changed the common policy for a long time, using a Enforer class to replace the old check function .In order to sync the common policy to nova, we have to rewrite the nova policy and the related unittests.

[Shuangtai Tian (shuangtai-tian)]

https://review.openstack.org/#/c/76829/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/97729

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/97731

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/76829
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a88d9d5936aabc52046ed3ae566741422bfaed78
Submitter: Jenkins
Branch: master

commit a88d9d5936aabc52046ed3ae566741422bfaed78
Author: Shuangtai Tian <email address hidden>
Date: Wed Mar 5 19:10:32 2014 +0800

    Rewrite nova policy to use the new changes of common policy

    The oslo has rewritten the common policy for a long time, using a
    Enforer class to replace the old check function. In order to sync the
    common policy to nova, We have to also changed the nova policy and
    related unittests.
    Sync policy and relate module from Oslo.
       Some related change-id:Ife909bdf3277ef33c2fb1eae16ae261fa6374c63
                    change-id:Ife84189be4b86a3ee90da4539ff2dbed125be23d

    Closes-Bug: #1288178

    Change-Id: Ic106851a24cfdc9d847f7d0e0d8a6ecbff888e05

[Matt Riedemann (mriedem)]

Since this merged in nova yesterday https://review.openstack.org/#/c/76829/ we're failing unit tests all over the check queue:

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiUG9saWN5Tm90QXV0aG9yaXplZFxcOiBQb2xpY3kgZG9lc25cXCd0IGFsbG93IGV4YW1wbGVcXDpnZXRfaHR0cCB0byBiZSBwZXJmb3JtZWRcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQwMjA4Mjc0MzA4N30=

[Matt Riedemann (mriedem)]

e-r query for the nova unit test race: https://review.openstack.org/#/c/98502/

[Joe Gordon (jogo)]

Since the new policy is causing errors, reverting it

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/98559

[Shuangtai Tian (shuangtai-tian)]

ooh, I found the oslo has been used the import six.moves.urllib.request instead of urllib2,
;(, I don't know why did the tests always success in my environment' tox.
sorry for this. thanks for reverting.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/98679

[OpenStack Infra (hudson-openstack)]

Change abandoned by Joe Gordon (<email address hidden>) on branch: master
Review: https://review.openstack.org/98559
Reason: We now have a proper fix

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/98679
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b35dc6108087466a4ed845b26b8ba0352cd1ea65
Submitter: Jenkins
Branch: master

commit b35dc6108087466a4ed845b26b8ba0352cd1ea65
Author: shuangtai <email address hidden>
Date: Mon Jun 9 10:27:04 2014 +0800

    fix nova test_enforce_http_true unit test

    When the nova policy patch was merged, causing
    nova.tests.test_policy.PolicyTestCase.test_enforce_http_true to fail
    randomly in the gate. The oslo has used the six.moves.urllib.request
    instead of urllib2, but the patch didn't sync this in test_policy.

    This related commit a88d9d5936aabc52046ed3ae566741422bfaed78.
    Related-Bug: #1288178

    Change-Id: I51b6bbf37b062edcf4c4b5f65adb8bfecd4a2eef

[John Garbutt (johngarbutt)]

Do we need this in nova now, given this one has merged?
https://review.openstack.org/#/c/98679/

[Shuangtai Tian (shuangtai-tian)]

if the test run successfully, we can close it now. I don't know whether we need to revert Matt's patch ？
https://review.openstack.org/#/c/98502/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/97729
Committed: https://git.openstack.org/cgit/openstack/tuskar/commit/?id=1a6382d855d94ef0e3feb090197bf0e892303955
Submitter: Jenkins
Branch: master

commit 1a6382d855d94ef0e3feb090197bf0e892303955
Author: Yongli He <email address hidden>
Date: Wed Jun 4 16:06:01 2014 +0800

    Rewrite policy to use the new changes of common policy

    The oslo has rewritten the common policy for a long time, using a
    Enforer class to replace the old check function. In order to sync the
    common policy, We have to also changed the tuskar policy and
    related unittests.
    Sync policy and relate module from Oslo.
       Some related change-id:Ife909bdf3277ef33c2fb1eae16ae261fa6374c63
                    change-id:Ife84189be4b86a3ee90da4539ff2dbed125be23d
    Closes-Bug: #1288178

    Change-Id: Ie4d8d31031d44ce46788d00624740825c04c7e88

[Marios Andreou (marios-b)]

For Tuskar, I am trying to set as fix-released as this was included in 0.4.7 released last week - this is commit 1a6382d855d94ef0e3feb090197bf0e892303955 in Tuskar. I can't set as fixreleased here, keeps timing out,

thanks! marios

[Doug Hellmann (doug-hellmann)]

I don't see anything obvious that needs to be done in oslo for this to be closed out, so I'm going to remove the bug from the oslo project. If I'm wrong, please add it back with a quick description of the work needed. Thanks!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/123673

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/126265

[OpenStack Infra (hudson-openstack)]

Change abandoned by Lucas Alvares Gomes (<email address hidden>) on branch: master
Review: https://review.openstack.org/97731

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/123673
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d83731a96d3b47fab1295de1fb53cfe30576619c
Submitter: Jenkins
Branch: master

commit d83731a96d3b47fab1295de1fb53cfe30576619c
Author: Elena Ezhova <email address hidden>
Date: Thu Sep 18 11:53:24 2014 +0400

    Updated policy module from oslo-incubator

    Common policy has not been synced with oslo-incubator for a
    long time and is seriously outdated.

    This change pulls in fresh code from oslo-incubator which
    introduces the Enforcer class to replace the old check function.

    Rewrite neutron.policy using naming conventions and approach
    that was set in Nova and amend related unit tests.
    Remove neutron.common.exceptions.PolicyNotAuthorized and switch
    to neutron.openstack.common.policy.PolicyNotAuthorized.
    Drop Neutron specific policy_file option since now it is defined
    in oslo-incubator policy module.

    Change log:
    4ca5091 Fixes nits in module policy
    262fc82 Correct default rule name for policy.Enforcer
    9e8b9f6 Minor fixes in policy module
    6c706c5 Delete graduated serialization files
    5d40e14 Remove code that moved to oslo.i18n
    aebb58f Fix typo to show correct log message
    bb410d9 Use MultiStrOpt for policy_dirs
    33f44bf Add support for policy configration directories
    2b966f9 Fix deletion of cached file for policy enforcer
    238e601 Make policy debug logging less verbose
    fe3389e Improve help strings
    15722f1 Adds a flag to determine whether to reload the rules in policy
    5d1f15a Documenting policy.json syntax
    fcf517d Update oslo log messages with translation domains
    e038d89 Fix policy tests for parallel testing
    0da5de6 Allow policy.json resource vs constant check
    e4b2334 Replaces use of urlutils with six in policy module
    8b2b0b7 Use hacking import_exceptions for gettextutils._
    0d8f18b Use urlutils functions instead of urllib/urllib2
    12bcdb7 Remove vim header
    9ef9fec Use six.string_type instead of basestring
    4bfb7a2 Apply six for metaclass
    1538c80 ConfigFileNotFoundError with proper argument
    33533b0 Keystone user can't perform revoke_token
    64bb5e2 Fix wrong argument in openstack common policy
    b7edc99 Fix missing argument bug in oslo common policy
    3626b6d Fix policy default_rule issue
    7bf8ee9 Allow use of hacking 0.6.0 and enable new checks
    e4ac367 Fix missing argument bug in oslo common policy
    1a2df89 Enable H302 hacking check
    7119e29 Enable hacking H404 test.
    6d27681 Enable H306 hacking check.
    1091b4f Reduce duplicated code related to policies

    Closes-Bug: #1288178
    Change-Id: I87ee30e2b64ec6b07faa84a231fd5f7eb925d501

[Louis Taylor (kragniz)]

This is being addressed in glance with this patch: https://review.openstack.org/#/c/127923

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/127923
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=bdc4951d29ce803891bee85f993e9a01893827e6
Submitter: Jenkins
Branch: master

commit bdc4951d29ce803891bee85f993e9a01893827e6
Author: Zhi Yan Liu <email address hidden>
Date: Mon Oct 13 18:06:56 2014 +0800

    Update glance.openstack.common.policy and cleanup

    1. Sync glance.openstack.common.policy up to latest
    version from oslo-inc.

    2. Clean useless modules which depended by policy
    module and pinned gettextutils module there. For
    latter one, we are going to use glance.i18n instead.
     * jsonutils
     * strutils

    docImpact

    Closes-bug: #1288178
    Closes-bug: #1387973
    Partial-bug: #1381870
    Change-Id: I84511ab1ee600e618985448dfbfbdc26cb130370
    Signed-off-by: Zhi Yan Liu <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126265
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=ec7f26a35270097210df7ff57e950d3e6b6fc3ec
Submitter: Jenkins
Branch: master

commit ec7f26a35270097210df7ff57e950d3e6b6fc3ec
Author: Roman Prykhodchenko <email address hidden>
Date: Thu Oct 2 03:53:33 2014 +0200

    Refactoring for Ironic policy

     - Synchronised OSLO policy and its dependencies to the
       newest version.
     - Replace policy-related code in Ironic with a proper
       configuration of the common policy engine;

    related commit 07e9b32a95352c25a611a93d215878a8b9a36b71
            commit b19af0806f0e2dffc83607d39a88e408928da72c
            commit 2324c77549f0affeda854ac3e7a500097450bb6a
            commit a51469326e84ed977ecc4e57fd3d46cdc21aa08f
            commit fde1e156a38633ce9018569145390bce2047fea8
            commit e700d926f7d8fe2f57e53b93361aaf281bebc8ed
            commit 65e3d8c9773880094c0a4c164e046fae9cb7a5d9
            commit 5d1f15a7785b2597eb9db5700ace9625bd2d44dd
            commit fcf517d72cb81f972fad20caa9ff0341e9b4aa9c
            commit e038d896174ada12c4d8b1ddafda2834d9ed0b14

    Change-Id: I4ede79ec7e56a6a7c5ca3d69d3b4fb9d2f4ada22
    Closes-Bug: #1288178

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/156412
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=1c28ead9006a6f6d48f417537d58c0cab9f984b2
Submitter: Jenkins
Branch: master

commit 1c28ead9006a6f6d48f417537d58c0cab9f984b2
Author: Jay S. Bryant <email address hidden>
Date: Mon Feb 16 17:05:01 2015 -0600

    Sync policy module from oslo-incubator

    The policy module has not had a sync since back in
    November. There have been a number of changes that
    should be pulled into Kilo.

    Current HEAD in OSLO:
    ---------------------
    commit 9bf01f9d98487cb13e3d95ad2a580fe8fc6f2479
    Date: Fri Feb 13 14:18:58 2015 -0500
    Switch from oslo.config to oslo_config

    Changes merged with this patch:
    ---------------------
    2aacb111 Change oslo.config to oslo_config
    2fbf5065 Remove oslo.log code and clean up versionutils API
    262279b1 switch to oslo_serialization
    07e9b32a Improving docstrings for policy API
    e67f5cd0 Merge "Don't log missing policy.d as a warning"
    99d991ce Merge "Fixed a problem with neutron http policy check"
    b19af080 Don't log missing policy.d as a warning
    2324c775 Add rule overwrite flag to Enforcer class
    6166a960 Fixed a problem with neutron http policy check

    Closes-bug: 1288178
    Change-Id: I6987029b9c15f3d35fa591014859f5f96c98f3a3