OVN doesn't seem to support reverse DNS lookups for instances

[Chris MacNaughton]

[Impact]
At Focal we took a stance that moving forward new deployments of Charmed OpenStack should use OVN as the SDN. For clouds upgrading, Focal is also the point for which a migration from ML2/OVS to ML2/OVN should take place before further upgrades are applied.

The current lack of support for reverse DNS lookup for IP addresses belonging to project instances would prevent users with applications depending on this feature to commence new deployments or upgrade through Focal and beyond.

[Test Plan]
Deploy a Charmed OpenStack and perform forward and reverse DNS lookups for instance names and addresses with the old and new packages to confirm.

[Regression Potential]
The instance DNS resolution in OVN works by looking up records in the OVN Southbound database. The CMS adds records for both forward and reverse lookups and the OVN controller needs to know how to look for them. The included patches have been available for several upstream releases without any reported issues. In addition we have been running internal clouds with these patches without issues.

[Original Bug Description]
There's an upstream bug that mentions this during implementation of regular DNS lookups - https://bugzilla.redhat.com/show_bug.cgi?id=1503521

[Andrew McLeod (admcleod)]

Can we assume this is also an upstream bug?

[Chris MacNaughton (chris.macnaughton)]

This is a bug in ovn itself, yes

[Launchpad Janitor (janitor)]

[Expired for charm-ovn-central because there has been no activity for 60 days.]

[Frode Nordahl (fnordahl)]

There is some work contributed upstream related to this here: https://github.com/ovn-org/ovn/pull/74

[Frode Nordahl (fnordahl)]

https://github.com/ovn-org/ovn/commit/82a4e44e308171cb545211eb2534475ef16a4c0e

[Brian Murray (brian-murray)]

The Hirsute Hippo has reached End of Life, so this bug will not be fixed for that release.

[James Page (james-page)]

@fnordahl - merged and will upload shortly - bug still needs SRU information around impact, testcase etc..

[Robie Basak (racb)]

Still awaiting SRU information please.

[Frode Nordahl (fnordahl)]

Description updated with SRU documentation, apologies for the delay.

[Corey Bryant (corey.bryant)]

Hello Chris, or anyone else affected,

Accepted ovn into wallaby-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:wallaby-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-wallaby-needed to verification-wallaby-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-wallaby-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Chris, or anyone else affected,

Accepted ovn into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ovn/20.03.2-0ubuntu0.20.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Frode Nordahl (fnordahl)]

Deployed focal-ussuri from charm-neutron-api-plugin-ovn gate tests and applied pending Neutron patch [0].

With two instances running we can see the following information in the OVN DB:
$ sudo ovn-nbctl list dns
_uuid : d51565b8-4ddb-4038-8a92-55e25247a419
external_ids : {ls_name=neutron-de1d0eb4-405a-4fc1-98ef-b93c7a2a4524}
records : {"114.0.168.192.in-addr.arpa"=zaza-neutrontests-ins-1.openstack.example, "13.0.168.192.in-addr.arpa"=zaza-neutrontests-ins-2.openstack.example, zaza-neutrontests-ins-1="192.168.0.114", zaza-neutrontests-ins-1.openstack.example="192.168.0.114", zaza-neutrontests-ins-2="192.168.0.13", zaza-neutrontests-ins-2.openstack.example="192.168.0.13"}

Control test:
From instance forward DNS lookups of other instance names work:
$ host zaza-neutrontests-ins-1.openstack.example
zaza-neutrontests-ins-1.openstack.example has address 192.168.0.114

But reverse DNS lookup of other instance IP does not work:
$ host 192.168.0.114
Host 114.0.168.192.in-addr.arpa. not found: 3(NXDOMAIN)

Upgrade to packages from -proposed:
$ juju run --application ovn-chassis 'dpkg -l | grep ovn'
- Stdout: |
    ii neutron-ovn-metadata-agent 2:16.4.2-0ubuntu1 all Neutron is a virtual network service for Openstack - OVN metadata agent
    ii ovn-common 20.03.2-0ubuntu0.20.04.3 amd64 OVN common components
    ii ovn-host 20.03.2-0ubuntu0.20.04.3 amd64 OVN host components
  UnitId: ovn-chassis/0
- Stdout: |
    ii neutron-ovn-metadata-agent 2:16.4.2-0ubuntu1 all Neutron is a virtual network service for Openstack - OVN metadata agent
    ii ovn-common 20.03.2-0ubuntu0.20.04.3 amd64 OVN common components
    ii ovn-host 20.03.2-0ubuntu0.20.04.3 amd64 OVN host components
  UnitId: ovn-chassis/1

Confirm updated packages work:
$ sudo systemctl restart systemd-resolved
$ host zaza-neutrontests-ins-1.openstack.example
zaza-neutrontests-ins-1.openstack.example has address 192.168.0.114
$ host 192.168.0.114
114.0.168.192.in-addr.arpa domain name pointer zaza-neutrontests-ins-1.openstack.example.

0: curl https://review.opendev.org/changes/openstack%2Fneutron~823631/revisions/2/patch?download|base64 -d|sudo patch -p2

[Frode Nordahl (fnordahl)]

Deployed focal-wallaby from charm-neutron-api-plugin-ovn gate tests and patched Neutron for some unresolved issues at Wallaby. The need to do that is unfortunate but this verificat
ion is about the OVN pacakges, so let's focus on them.

With two instances running we can see the following information in the OVN DB:
$ sudo ovn-nbctl list dns
_uuid : a62d480f-e5ad-421e-9e8a-ab4072232331
external_ids : {ls_name=neutron-7d2d7f7e-ebb9-4625-a2ab-8bb6647f1988}
records : {"182.0.168.192.in-addr.arpa"=zaza-neutrontests-ins-1.openstack.example, "239.0.168.192.in-addr.arpa"=zaza-neutrontests-ins-2.openstack.example, zaza-neutrontests-ins-1="192.168.0.182", zaza-neutrontests-ins-1.openstack.example="192.168.0.182", zaza-neutrontests-ins-2="192.168.0.239", zaza-neutrontests-ins-2.openstack.example="192.168.0.239"}

Control test:
From instance forward DNS lookups of other instance names work:
$ host zaza-neutrontests-ins-1.openstack.example
zaza-neutrontests-ins-1.openstack.example has address 192.168.0.182

But reverse DNS lookup of other instance IP does not work:
$ host 192.168.0.182
Host 182.0.168.192.in-addr.arpa. not found: 3(NXDOMAIN)

Upgrade to packages from -proposed:
$ juju run --application ovn-chassis 'dpkg -l | grep ovn'
- Stdout: |
    ii neutron-ovn-metadata-agent 2:18.1.1-0ubuntu2~cloud0 all Neutron is a virtual network service for Openstack - OVN metadata agent
    ii ovn-common 20.12.0-0ubuntu3.1~cloud0 amd64 OVN common components
    ii ovn-host 20.12.0-0ubuntu3.1~cloud0 amd64 OVN host components
  UnitId: ovn-chassis/0
- Stdout: |
    ii neutron-ovn-metadata-agent 2:18.1.1-0ubuntu2~cloud0 all Neutron is a virtual network service for Openstack - OVN metadata agent
    ii ovn-common 20.12.0-0ubuntu3.1~cloud0 amd64 OVN common components
    ii ovn-host 20.12.0-0ubuntu3.1~cloud0 amd64 OVN host components
  UnitId: ovn-chassis/1

Confirm updated packages work:
$ sudo systemctl restart systemd-resolved
$ host zaza-neutrontests-ins-1.openstack.example
zaza-neutrontests-ins-1.openstack.example has address 192.168.0.182
$ host 192.168.0.182
182.0.168.192.in-addr.arpa domain name pointer zaza-neutrontests-ins-1.openstack.example.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ovn - 20.03.2-0ubuntu0.20.04.3

---------------
ovn (20.03.2-0ubuntu0.20.04.3) focal; urgency=medium

  [ James Troup ]
  * Fix Multicast traffic between VMs using provider networks (LP: #1957817):
   - d/p/lp-1957817-ovn-northd-Add-localnet-ports-to-Multicast_Groups-cr.patch

  [ Frode Nordahl ]
  * Add support for PTR DNS requests (LP: #1857026):
    - d/p/lp-1857026-0001-DNS-Make-DNS-lookups-case-insensitive.patch
    - d/p/lp-1857026-0002-controller-Add-support-for-PTR-DNS-requests.patch

 -- Frode Nordahl <email address hidden> Fri, 25 Feb 2022 08:10:00 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ovn has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for ovn has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package ovn - 20.12.0-0ubuntu3.1~cloud0
---------------

 ovn (20.12.0-0ubuntu3.1~cloud0) focal-wallaby; urgency=medium
 .
   * Add support for PTR DNS requests (LP: #1857026)
     - d/p/lp-1857026-controller-Add-support-for-PTR-DNS-requests.patch