REST API relies on policies being initialized after RESOURCE_ATTRIBUTE_MAP is processed, does nothing to ensure it.

[Brent Eagles]

A race condition exists where policies may be loaded and processed before the neutron extensions are loaded and the RESOURCE_ATTRIBUTE_MAP is populated. This causes problems in system behaviour dependent on neutron specific policy checks. Policies are loaded at on demand, and if the call instigating the loading of policies happens prematurely this can cause certain neutron specific policy checks to not be setup properly as the required mappings from policy to check implementations has not been established.

Related bugs:

https://bugs.launchpad.net/neutron/+bug/1254555
https://bugs.launchpad.net/neutron/+bug/1251982
https://bugs.launchpad.net/neutron/+bug/1280738

[Brent Eagles (beagles)]

Video demonstrating how to reproduce http://youtu.be/-nL1gyL3KL8.

[Eugene Nikanorov (enikanorov)]

Can you please describe steps to reproduce in text form?
I think there was several attempts to fix the issue so I wonder if it really still exists.

[Brent Eagles (beagles)]

Here, in general form, are the steps used to reproduce this issue as demonstrated in the video:

1. simulate high agent load by changing report_interval to 1
2. simulate startup processing load by putting a sleep immediately after the line containing:
        ext_mgr = extensions.PluginAwareExtensionManager.get_instance()
   in neutron/api/v2/router.py
3. stop agents
4. stop neutron server
5. start agents
6. start neutron server

run nova network-list until REST API starts responding. Networks with router:external=True that should be visible due to the neutron FieldCheck on the external network extension do not succeed. Consequence is these networks are hidden from the user.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/127633
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=eeff5d06b2099ed9813091926dd8cef58680ad8f
Submitter: Jenkins
Branch: master

commit eeff5d06b2099ed9813091926dd8cef58680ad8f
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/146572

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/146603

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/146572
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=175e86977d6905e1e11766ca47e30b89c88d8827
Submitter: Jenkins
Branch: stable/juno

commit 175e86977d6905e1e11766ca47e30b89c88d8827
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555
    (cherry picked from commit eeff5d06b2099ed9813091926dd8cef58680ad8f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/146603
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c9cb001626e90239b1ffe5583ab9a4349280856c
Submitter: Jenkins
Branch: stable/icehouse

commit c9cb001626e90239b1ffe5583ab9a4349280856c
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555
    (cherry picked from commit eeff5d06b2099ed9813091926dd8cef58680ad8f)