Using external routers and dhcp metadata together is difficult

[Darragh O'Reilly]

When isolated_metadata=true, the dhcp agent will only push out a static route for the metadata address (169.254.169.254/32 via the dhcp ip) when the subnet is isolated. This makes sense because if the subnet is connected to a Neutron router, then the instances can get the metadata from the Neutron router namespace via their default route, and so there is no need for the static route.

Currently the dhcp agent determines that the subnet is isolated by simply checking that the subnet gateway_ip is not set.

https://github.com/openstack/neutron/blob/177bfb030e60267fb009b181e752ec6c37d9010b/neutron/agent/linux/dhcp.py#L450

enable_metadata = (
                self.conf.enable_isolated_metadata
                and not subnet.gateway_ip
                and subnet.ip_version == 4)

But this creates difficulty for users who don't want to use Neutron routers, but want to use a provider network with an external router and get the metadata via the proxy in the dhcp namespace instead. You would like to set gateway_ip to the external router, but when you do that, the agent will not push out the route.

To workaround, you can push out the default route as a host route, eg, if the external router is at 10.0.0.254:

neutron subnet-create net1 10.0.0.0/24 --name sub1 \
--no-gateway \
--host-route destination=0.0.0.0/0,nexthop=10.0.0.254 \
--allocation-pool start=10.0.0.1,end=10.0.0.253

Or you can set the gateway_ip and manually add the metadata static route to the subnet. But then you need to first determine what the dhcp IP is, or will be, and I think it can land on any of the first 3 ips of the CIDR depending on where the gateway_ip is, and if the dhcp-agent is restarted before the first instance for the network is booted.

Anyway, these workarounds are tricky and are not very obvious. It would be better if users could just do this:

neutron subnet-create net1 10.0.0.0/24 --name sub1 ---gateway_ip 10.0.0.254

This would be possible if the agent determined that the subnet was isolated by checking for the absence of a Neutron router on it, rather than just checking that it has no gateway_ip.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/50292

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/50292
Committed: http://github.com/openstack/neutron/commit/c73b54e50b62c489f04432bdbc5bee678b18226e
Submitter: Jenkins
Branch: master

commit c73b54e50b62c489f04432bdbc5bee678b18226e
Author: Darragh O'Reilly <email address hidden>
Date: Tue Oct 8 10:36:05 2013 +0000

    Simplify using external routers and metadata

    The dhcp agent only pushes out the metadata
    static route when the subnet is isolated, and it
    determines that by checking if the gateway_ip is
    not set. This makes it tricky to use external
    routers and metadata from dhcp at the same time.

    This patch changes how the dhcp agent determines
    that the subnet is isolated. It now considers it
    isolated if there is no Neutron router on it.
    This makes it straightforward to use an external
    router on a provider network and get the
    metadata from the dhcp namespace.

    Change-Id: I0e29a2f058564c267176dab26da00f6ef579808b
    Closes-Bug: 1236783

[John Dewey (retr0h)]

Would love to get this back ported to havana. We are seeing this very problem.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/77118

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/77118
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6d1678e86385d49615d77593b8ee5d1f17243357
Submitter: Jenkins
Branch: stable/havana

commit 6d1678e86385d49615d77593b8ee5d1f17243357
Author: Darragh O'Reilly <email address hidden>
Date: Tue Oct 8 10:36:05 2013 +0000

    Simplify using external routers and metadata

    The dhcp agent only pushes out the metadata
    static route when the subnet is isolated, and it
    determines that by checking if the gateway_ip is
    not set. This makes it tricky to use external
    routers and metadata from dhcp at the same time.

    This patch changes how the dhcp agent determines
    that the subnet is isolated. It now considers it
    isolated if there is no Neutron router on it.
    This makes it straightforward to use an external
    router on a provider network and get the
    metadata from the dhcp namespace.

    Change-Id: I0e29a2f058564c267176dab26da00f6ef579808b
    Closes-Bug: 1236783
    (cherry picked from c73b54e50b62c489f04432bdbc5bee678b18226e)