neutron metering agent does not work in DVR mode

[edison peng]

* High level description:
When a user adds a metering label/metering label rule, metering agents on compute nodes will be notified to setup iptables. However, in Mitaka Neutron DVR mode, metering agents on compute nodes will not be notified and the iptables on compude nodes will not be setup.

After tracing the code, I believe the root cause is the side effect of "Improve DVR router sheduling mechanism for better performance/scalability"
https://blueprints.launchpad.net/neutron/+spec/improve-dvr-l3-agent-binding
This feature skips the binding process of L3 agents and routers. However, metering agents will only be notified based on the bindings.

* Pre-conditions:
None

* Step-by-step reproduction steps:
1. launch an instance
2. create a metering label rule based on the private ip of the instance

* Expected output:
The iptables should be setup in the namespace of the router of the network of the instance.

* Actual output:
The iptables does not change, and the metering will not work.

* Version:
Mitaka

* Environment:
Neutron DVR Mode

* Perceived severity:
It is a blocker for DVR environment

[Brian Haley (brian-haley)]

Do you have the latest stable/mitaka code? This should have been fixed with https://review.openstack.org/#/c/424285/

[edison peng (edisonpeng)]

My environment is not the latest stable/mitaka. However, I believe https://bugs.launchpad.net/neutron/+bug/1506567 is different with mine.

According to my understanding, https://bugs.launchpad.net/neutron/+bug/1506567 is, the metering agent will setup wrong iptables rule in DVR mode because of the diferences of the prefix of the external device name.

My problem is, because of the binding between routers and l3 agent is not maintain anymore(side effect of https://blueprints.launchpad.net/neutron/+spec/improve-dvr-l3-agent-binding), metering agent will not be notified to setup the iptables rule. It will do nothing for iptables on compute node.

[Swaminathan Vasudevan (swaminathan-vasudevan)]

Scheduling of routers should not affect the metering agent. At present only if there is a valid DVR serviceable port in the compute host we schedule a router.

When there is a valid port on the compute host the router will be scheduled.

Is there a hook on the neutron server by the metering agent to configure the metering agent on all scheduled nodes? Can you point me to the code where this happens in the server.

[Swaminathan Vasudevan (swaminathan-vasudevan)]

Ok, I see the problem. The l3_rpc_agent_api.py has been modified for to notify specific hosts that are hosting the routers.
But that change should also be applied to the metering_rpc_agent_api.py and so the metering label configuration will be delegated to the nodes.

[Brian Haley (brian-haley)]

Hi Dongcan - are you going to work on this bug? Feel free to add Swami and myself as reviewers. If not we can re-assign.

[Dongcan Ye (hellochosen)]

Hi, Brian Haley. I'm not start until now, feel free to re-assign to you or Swami.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/475077

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/475077
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=72338d9fad620ea5615a283d1e1d3cb7b56a5d9d
Submitter: Jenkins
Branch: master

commit 72338d9fad620ea5615a283d1e1d3cb7b56a5d9d
Author: Swaminathan Vasudevan <email address hidden>
Date: Fri Jun 16 14:01:38 2017 -0700

    DVR: Fix neutron metering agent to notify hosts hosting DVR

    DVR router updates are notified based on where they are hosted.
    But in the case of metering rpc notifier the notification to a
    specific host was not implemented and also when a call is being
    made to routers_updated_on_host, it is being ignored since it only
    implements the routers_updated function call.

    Change-Id: I39b0d43b14294a8eecf6cba230d948c2c45a0b7a
    Closes-Bug: #1682345

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/484408

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/484409

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/484408
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ba075c658142c93e67e61349735d0d91b423ed03
Submitter: Jenkins
Branch: stable/ocata

commit ba075c658142c93e67e61349735d0d91b423ed03
Author: Swaminathan Vasudevan <email address hidden>
Date: Fri Jun 16 14:01:38 2017 -0700

    DVR: Fix neutron metering agent to notify hosts hosting DVR

    DVR router updates are notified based on where they are hosted.
    But in the case of metering rpc notifier the notification to a
    specific host was not implemented and also when a call is being
    made to routers_updated_on_host, it is being ignored since it only
    implements the routers_updated function call.

    Change-Id: I39b0d43b14294a8eecf6cba230d948c2c45a0b7a
    Closes-Bug: #1682345
    (cherry picked from commit 72338d9fad620ea5615a283d1e1d3cb7b56a5d9d)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.3 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ihar Hrachyshka (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/484409
Reason: Newton is CVE only at this point, and we have concerns whether it's even of high impact. Restore if you think stable rules justify the backport.