REST API relies on policies being initialized after RESOURCE_ATTRIBUTE_MAP is processed, does nothing to ensure it.

Bug #1398566 reported by Brent Eagles on 2014-12-02
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Undecided
Brent Eagles
Icehouse
Undecided
Unassigned
Juno
Undecided
Unassigned

Bug Description

A race condition exists where policies may be loaded and processed before the neutron extensions are loaded and the RESOURCE_ATTRIBUTE_MAP is populated. This causes problems in system behaviour dependent on neutron specific policy checks. Policies are loaded at on demand, and if the call instigating the loading of policies happens prematurely this can cause certain neutron specific policy checks to not be setup properly as the required mappings from policy to check implementations has not been established.

Related bugs:

https://bugs.launchpad.net/neutron/+bug/1254555
https://bugs.launchpad.net/neutron/+bug/1251982
https://bugs.launchpad.net/neutron/+bug/1280738

Brent Eagles (beagles) on 2014-12-02
Changed in neutron:
assignee: nobody → Brent Eagles (beagles)
Brent Eagles (beagles) wrote :

Video demonstrating how to reproduce http://youtu.be/-nL1gyL3KL8.

Brent Eagles (beagles) on 2014-12-03
summary: - Policies may be loaded prematurely, causing failed policy checks
+ REST API relies on policies being initialized after
+ RESOURCE_ATTRIBUTE_MAP is processed, does nothing to ensure it.
Eugene Nikanorov (enikanorov) wrote :

Can you please describe steps to reproduce in text form?
I think there was several attempts to fix the issue so I wonder if it really still exists.

Changed in neutron:
status: New → Incomplete
Brent Eagles (beagles) wrote :

Here, in general form, are the steps used to reproduce this issue as demonstrated in the video:

1. simulate high agent load by changing report_interval to 1
2. simulate startup processing load by putting a sleep immediately after the line containing:
        ext_mgr = extensions.PluginAwareExtensionManager.get_instance()
   in neutron/api/v2/router.py
3. stop agents
4. stop neutron server
5. start agents
6. start neutron server

run nova network-list until REST API starts responding. Networks with router:external=True that should be visible due to the neutron FieldCheck on the external network extension do not succeed. Consequence is these networks are hidden from the user.

Changed in neutron:
status: Incomplete → In Progress

Reviewed: https://review.openstack.org/127633
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=eeff5d06b2099ed9813091926dd8cef58680ad8f
Submitter: Jenkins
Branch: master

commit eeff5d06b2099ed9813091926dd8cef58680ad8f
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555

Changed in neutron:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/146572
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=175e86977d6905e1e11766ca47e30b89c88d8827
Submitter: Jenkins
Branch: stable/juno

commit 175e86977d6905e1e11766ca47e30b89c88d8827
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555
    (cherry picked from commit eeff5d06b2099ed9813091926dd8cef58680ad8f)

tags: added: in-stable-juno

Reviewed: https://review.openstack.org/146603
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c9cb001626e90239b1ffe5583ab9a4349280856c
Submitter: Jenkins
Branch: stable/icehouse

commit c9cb001626e90239b1ffe5583ab9a4349280856c
Author: Brent Eagles <email address hidden>
Date: Fri Oct 10 13:27:51 2014 -0230

    Reset policies after RESOURCE_ATTRIBUTE_MAP is populated

    The REST API relies on neutron-specific policy checking logic that is
    only available after the extensions are loaded and the
    RESOURCE_ATTRIBUTE_MAP is populated. This patch resets the policies
    immediately after these steps are done. This ensures that in the event
    the policies are prematurely loaded for any reason, the on-demand
    loading of the policies will reload the policies and properly configure
    the neutron specific checks on the next policy check.

    Change-Id: Ic2ab3f0179b0c192e63af0bc4268d92aa26bdabe
    Closes-Bug: #1398566
    Related-Bug: #1254555
    (cherry picked from commit eeff5d06b2099ed9813091926dd8cef58680ad8f)

tags: added: in-stable-icehouse
Thierry Carrez (ttx) on 2015-02-05
Changed in neutron:
milestone: none → kilo-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2015-04-30
Changed in neutron:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers