Make it possible to configure next hop for external VIP address

[Niklas Hagman]

For security reasons we all normally place our OpenStack cluster in internal networks that is not reachable from Internet. There is one exception and that is the external VIP address. I want to make it more flexible by making it possible to specify where next hop should be for outgoing default traffic from the external VIP address.

The patch adds keepalived features that create in the routing table a "default via xxx.xxx.xxx.xx dev xxxx.xxx" but adds a rule so it only applies to traffic from configured external VIP address.
It can be inspected with "ip rule list" and then "ip route list table xx". Simple!

== Example ==
Next hop wanted: 203.0.113.3
External VIP address: 203.0.113.22
External VIP interface. ens3f0.1435
Virtual router ID: 51

vrrp_instance kolla_internal_vip_51 {
    state BACKUP
    nopreempt
    interface ens3f0.1438
    virtual_router_id 51
    priority 2
    advert_int 1
    virtual_ipaddress {
        192.168.38.9 dev ens3f0.1438
        203.0.113.22 dev ens3f0.1435
    }
    virtual_routes {
        203.0.113.3 scope link dev ens3f0.1435
        default via 203.0.113.3 dev ens3f0.1435 table 51
    }
    virtual_rules {
        from 203.0.113.22 table 51
    }
    track_interface {
        ens3f0.1435
    }
    authentication {
        auth_type PASS
        auth_pass xxxxxxxxxxxxxxxxxxxxxxxxxxxx
    }
    track_script {
        check_alive
    }
}

There is no subnets here so whatever is reachable and allowed by external router as external VIP address is usable. There should be no problem with IPv6 address when that feature comes.

--- kolla-orig/share/kolla-ansible/etc_examples/kolla/globals.yml 2019-08-14 07:11:18.128600000 +0000
+++ kolla-patched/share/kolla-ansible/etc_examples/kolla/globals.yml 2019-08-14 07:15:55.799339824 +0000
@@ -41,6 +41,18 @@
 # internal and external requests between two VIPs.
 #kolla_external_vip_address: "{{ kolla_internal_vip_address }}"

+# Here you can specify next hop for default outgoing traffic from the external
+# VIP address. This is used if external VIP interface does not have a
+# default route or you want traffic to go against another external router.
+# Example use case is where external VIP interface has default route against a
+# router that can not handle public IP addresses but you have another router on
+# VIP interface that can. Or where external VIP interface has no previous
+# configuration or default route. For example if you configure external VIP
+# address to be "203.0.113.10" you can here configure that default outgoing
+# traffic should have next hop "192.0.2.3" or whatever external router IP
+# address that is reachable within external VIP interface.
+#kolla_external_default_router_address:
+
 # The Public address used to communicate with OpenStack as set in the public_url
 # for the endpoints that will be created. This DNS name should map to
 # kolla_external_vip_address.

--- kolla-orig/share/kolla-ansible/ansible/roles/haproxy/templates/keepalived.conf.j2 2019-08-14 07:11:18.032599000 +0000
+++ kolla-patched/share/kolla-ansible/ansible/roles/haproxy/templates/keepalived.conf.j2 2019-08-14 07:16:44.799825006 +0000
@@ -31,6 +31,15 @@
         {{ kolla_external_vip_address }} dev {{ kolla_external_vip_interface }}
 {% endif %}
     }
+{% if haproxy_enable_external_vip | bool and kolla_external_default_router_address is defined %}
+ virtual_routes {
+ {{ kolla_external_default_router_address }} scope link dev {{ kolla_external_vip_interface }}
+ default via {{ kolla_external_default_router_address }} dev {{ kolla_external_vip_interface }} table {{ keepalived_virtual_router_id }}
+ }
+ virtual_rules {
+ from {{ kolla_external_vip_address }} table {{ keepalived_virtual_router_id }}
+ }
+{% endif %}
 {% if haproxy_enable_external_vip | bool and api_interface != kolla_external_vip_interface %}
     track_interface {
         {{ kolla_external_vip_interface }}

[Mark Goddard (mgoddard)]

Hi Niklas, you can customise the keepalived configuration by placing a file in /etc/kolla/config/keepalived/keepalived.conf. Just be sure to keep it in sync with the template in kolla-ansible when you upgrade.

[Radosław Piliszek (yoctozepto)]

Hi Niklas, good idea. Propose it for Ussuri. For the time being please use Mark's tip.