Without security group rule for port 67 in egress, VMs may not be able to get an IP from DHCP

[Flavio Fernandes]

Bug 1790900 introduced changes that removed OVN ACLs
to explicitly allow DHCP traffic to reach the responder
in the OVN pipeline.

Unfortunately, that removal is only okay because traffic
within a given SG is allowed by default. In situations
where security group configuration does not allow traffic
within ports of the same group, this will stop DHCP access
too, making VMs unable to obtain an address.

So the need for explicit rules to allow DHCP needs to be
added back to networking-ovn.

[Flavio Fernandes (ffernand)]

Reference comments on bug 1790900 that also explain its undesired side effect:

https://bugs.launchpad.net/networking-ovn/+bug/1790900/comments/7

https://bugs.launchpad.net/networking-ovn/+bug/1790900/comments/12

[Flavio Fernandes (ffernand)]

We have a working fix for this issue here [1]. However, we are no longer convinced that
the code complexity introduced by the change justifies its benefits. Reasons for that are:

1) More places for a bug to live;
2) Default egress rules work just fine to give VMs access to dhcp and metadata; so this is
   an unnecessary burden in memory and configuration for most use cases;
3) If the operator wants total control over egress rules, implicitly adding SGR behind the
   scenes is exactly what we should _not_ do.

Lastly, just a note that this is a slight departure from the default behavior that one gets
when using ml2/ovs. Even though mls/ovs adds no 'implicit' rules for allowing vms to reach
metadata, it does add the dhcpv4 rules. When using ml2/ovn, operators will need to do add the
rules on his/her own. It would look along these lines:

   # Allow VM to contact dhcp server (ipv4)
   openstack security group rule create --egress --ethertype IPv4 --protocol udp --dst-port 67 ${SEC_GROUP_ID}
   # Allow VM to contact metadata server (ipv4)
   openstack security group rule create --egress --ethertype IPv4 --protocol tcp --remote-ip 169.254.169.254 ${SEC_GROUP_ID}

   # Allow VM to contact dhcp server (ipv6, non-slaac). Be aware that the remote-ip may vary depending on your use case!
   openstack security group rule create --egress --ethertype IPv6 --protocol udp --dst-port 547 --remote-ip ff02::1:2 ${SEC_GROUP_ID}
   # Allow VM to contact metadata server (ipv6)
   openstack security group rule create --egress --ethertype IPv6 --protocol tcp --remote-ip fe80::a9fe:a9fe ${SEC_GROUP_ID}

[1]: https://review.opendev.org/#/c/733033/