DHCP agent might use default gateway of unrouted subnets

[Sebastian Lohff]

When creating a network with two subnets the DHCP agent will choose any subnet that has a gateway_ip (which is allocated and set by default). In cases where only one of the subnets is attached to a router (which the DHCP agent needs to access resources outside of the internal network like an upstream DNS server), the DHCP agent will still arbitrarily choose one of the available subnets. If it chooses the subnet not on a router, service will be disrupted for that network namespace. The sort order of subnet is arbitrary, but stable, but if a subnet is being added to a network the DHCP agent might also switch over to this network.

Problem was observed with Neutron Yoga, but can be reproduced with current upstream code.

To fix this I would propose to sort subnet gateway selection to first use
subnets that have a port that matches the gateway ip of the subnet, as they'll have a higher likelihood of being connected.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/917053