multi-space deployment: neutron-api-plugin-ovn binding with neutron-api is using the public space by default and ignoring the specified default binding space

[Itai Levy]

Series: Focal
OpenStack release: Yoga

When using the following configuration for neutron ovn plugin and neutron-api:

 neutron-api-plugin-ovn:
    charm: ch:neutron-api-plugin-ovn
    channel: yoga/stable
    options:
      dns-servers: 8.8.8.8

  neutron-api:
    charm: ch:neutron-api
.
.
    channel: yoga/stable
    bindings:
      "": *oam-space
      public: *public-space
      internal: *internal-space
      shared-db: *internal-space

relations:
- - neutron-api-plugin-ovn:neutron-plugin
  - neutron-api:neutron-plugin-api-subordinate

It seems like the space being used for this relation is for some reason always the public space (10.7.208.0/24) while the default binding space OAM (192.168.24.0/24) is being ignored, as we can see as well in "juju show-unit neutron-api" output:

- relation-id: 49
    endpoint: neutron-plugin-api-subordinate
    related-endpoint: neutron-plugin
    application-data: {}
    related-units:
      neutron-api-plugin-ovn/0:
        in-scope: true
        data:
          egress-subnets: 10.7.208.13/32
          ingress-address: 10.7.208.13
          mechanism-drivers: ovn
          neutron-plugin: ovn
          private-address: 10.7.208.13
          service-plugins: metering,segments,ovn-router
          subordinate_configuration: '{"neutron-api": {"/etc/neutron/plugins/ml2/ml2_conf.ini":
            {"sections": {"ovn": [["ovn_nb_connection", "ssl:192.168.24.123:6641,ssl:192.168.24.128:6641,ssl:192.168.24.127:6641"],
            ["ovn_nb_private_key", "/etc/neutron/plugins/ml2/key_host"], ["ovn_nb_certificate",
            "/etc/neutron/plugins/ml2/cert_host"], ["ovn_nb_ca_cert", "/etc/neutron/plugins/ml2/neutron-api-plugin-ovn.crt"],
            ["ovn_sb_connection", "ssl:192.168.24.123:16642,ssl:192.168.24.128:16642,ssl:192.168.24.127:16642"],
            ["ovn_sb_private_key", "/etc/neutron/plugins/ml2/key_host"], ["ovn_sb_certificate",
            "/etc/neutron/plugins/ml2/cert_host"], ["ovn_sb_ca_cert", "/etc/neutron/plugins/ml2/neutron-api-plugin-ovn.crt"],
            ["ovn_l3_scheduler", "leastloaded"], ["ovn_metadata_enabled", true], ["enable_distributed_floating_ip",
            false], ["dns_servers", "8.8.8.8"], ["dhcp_default_lease_time", 43200],
            ["ovn_dhcp4_global_options", ""], ["ovn_dhcp6_global_options", ""], ["vhost_sock_dir",
            "/run/libvirt-vhost-user"]], "ml2_type_geneve": [["vni_ranges", "1001:2000"],
            ["max_header_size", "38"]]}}}}'
          tenant-network-types: geneve,gre,vlan,flat,local

As a result, ovn-central UFW will be configured to allow SSL connections (port 16642) from neutron-api on the public space only, and neutron-api will crash as its connections attempts over OAM space will be refused:
> > 2022-05-31 18:37:50.189 124076 ERROR neutron.service File "/usr/lib/python3.8/ssl.py", line 1170, in send
> > 2022-05-31 18:37:50.189 124076 ERROR neutron.service raise ValueError(
> > 2022-05-31 18:37:50.189 124076 ERROR neutron.service ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
> > 2022-05-31 18:37:50.189 124076 ERROR neutron.service
> > 2022-05-31 18:37:50.193 124076 CRITICAL neutron [req-1aa04fbb-2f6a-46d4-b769-8f2aafb21c1c - - - - -] Unhandled error: ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>

OVN-Central Units Firewall:
> # ufw status verbose | grep 16642
> > 16642/tcp ALLOW IN 10.7.208.13 # charm-ovn-central
> > 16642/tcp ALLOW IN 192.168.24.123 # charm-ovn-central
> > 16642/tcp ALLOW IN 192.168.24.128 # charm-ovn-central
> > 16642/tcp REJECT IN Anywhere # charm-ovn-central
> > 16642/tcp (v6) REJECT IN Anywhere (v6) # charm-ovn-central

The issue is resolved only when specifying explicitly the space to be used for this relation:

  neutron-api-plugin-ovn:
.
.
    bindings:
      "": *oam-space
      neutron-plugin: *oam-space

  neutron-api:
    charm: ch:neutron-api
.
.
    bindings:
      "": *oam-space
      public: *public-space
      internal: *internal-space
      shared-db: *internal-space
      neutron-plugin-api-subordinate: *oam-space