Cached network object in DHCP agent not updated with router interface changes

[Shih-Hao Li]

In Dnsmasq, the function get_isolated_subnets() returns a list of subnets in a network if the subnet is not connected to a router.

The implementation of this function checks all the router interface ports in a cached network object passed from DHCP agent. But the cached network object is not updated when a subnet is attached to or detached from a router.

This could cause problem when a new VM is booted on a subnet just attached to a router. The VM will get the metadata route to the DHCP port instead of to the route port from Dnsmasq.

Based on current DHCP agent implementation, if DHCP agent is restarted, it will try to restart all metadata proxies. But it will skip the metadata proxy for the networks with any subnet attached to a router. Instead, DHCP agent will start a metadata-proxy for the router. If old metadata proxy processes for the networks are still running, then the metadata route to the DHCP port should still work. But consider the case when a openstack network node is restarted, then all old processes are gone. Thus DHCP agent will not start those metadata proxies for networks with attached router. This means any VM that has routing table containing a metadata route to the DHCP port will fail to reach metadata service because the corresponding metadata proxy that handle 169.254.169.254:80 is not running.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/290216

[Oleg Bondarev (obondarev)]

A good bug report would also have user impact and steps to reproduce the issue. Please add if you can. This will simplify bug triage.

[Ihar Hrachyshka (ihar-hrachyshka)]

I marked it with High since the bug results in not serving metadata for some isolated networks that were previously attached to a router.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/290216
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=aa00310eef0d2507c4da76eb5dd5130df043b0a5
Submitter: Jenkins
Branch: master

commit aa00310eef0d2507c4da76eb5dd5130df043b0a5
Author: Shih-Hao Li <email address hidden>
Date: Fri Mar 11 15:13:34 2016 -0800

    Update network object in DHCP agent with router interface changes

    In Dnsmasq, the function get_isolated_subnets() returns a list of
    subnets in a network if the subnet is not connected to a router.

    The implementation of this function checks all the router interface
    ports in a cached network object passed from DHCP agent. But the
    cached network object is not updated when a subnet is attached to
    or detached from a router.

    This patch fixes that by adding callback functions in DHCP RPC client
    to notify DHCP agent when changes happen on router interfaces.

    Closes-Bug: #1554825
    Change-Id: Ifaab163f49e0d1c5cb3eba6efa96214104647e4e

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0rc1 release candidate.

[Sreenivas (sreenivas-pothukanoori)]

While working on bug #1549860, “Hong Hui Xiao” has marked it as duplicate of this bug.

Could you please let me know to which status I have to set the bug #1549860.

Please provide the patch for bug #1554825 . We need the patch to check and confirm that this patch has resolved the issue reported in the 1549860.

[Shih-Hao Li (shihli)]

The patch is already merged, please see comment #4.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/321628

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321628
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=59236a014859e1e14a5844286903753d3f520e3e
Submitter: Jenkins
Branch: stable/liberty

commit 59236a014859e1e14a5844286903753d3f520e3e
Author: Shih-Hao Li <email address hidden>
Date: Fri Mar 11 15:13:34 2016 -0800

    Update network object in DHCP agent with router interface changes

    In Dnsmasq, the function get_isolated_subnets() returns a list of
    subnets in a network if the subnet is not connected to a router.

    The implementation of this function checks all the router interface
    ports in a cached network object passed from DHCP agent. But the
    cached network object is not updated when a subnet is attached to
    or detached from a router.

    This patch fixes that by adding callback functions in DHCP RPC client
    to notify DHCP agent when changes happen on router interfaces.

    Conflicts:
     neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py
     neutron/db/l3_db.py
     neutron/db/l3_dvr_db.py

    Local changes:
    - in a unit test, moved callback failure setup to after interface is
      added to a router, because now we call to register.notify in
      add_router_interface too, and the test intends to trigger failure mode
      for interface removal only.

    Closes-Bug: #1554825
    Change-Id: Ifaab163f49e0d1c5cb3eba6efa96214104647e4e
    (cherry picked from commit aa00310eef0d2507c4da76eb5dd5130df043b0a5)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 7.1.2 release.