[FUll Tempest job][master][periodic]3 Tempest tests related to DNS failing

[yatin]

Below 3 Tempest tests related to DNS are failing in Full tempest jobs[2] since dns-integration extension is enabled in OVN in [1], before OVN patch these tests used to be skip(check SUCCESS jobs in [2]):-

- neutron_tempest_plugin.api.test_revisions.TestRevisions.test_update_dns_domain_bumps_revision
Fails with:- AssertionError: 1 not greater than 1 while matching revision_number

- neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest.test_dns_domain_and_name
Fails with:- testtools.matchers._impl.MismatchError: 'starwars' not in u'nameserver 1.1.1.1\nnameserver 1.0.0.1\nnameserver 38.145.33.91\n'

- neutron_tempest_plugin.scenario.test_dns_integration.DNSIntegrationTests (this running only in fs020 master job as desinate-tempest-plugin is installed in standalone full tempest this is skipped as designate-tempest-plugin is not installed)
Fails with:- Details: No matching service found in the catalog.

Example logs:-
- https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/dd92512/logs/tempest.html.gz
- https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/d71a1ea/logs/stestr_results.html

neutron_tempest_plugin.scenario.test_dns_integration.DNSIntegrationTests is skipped in OVN gate [1] as well with below comment so makes sense to skip in Tripleo tempest tests as well:-
# TODO(mjozefcz): Remove this blacklist when OVS 2.12 will be released and
# https://patchwork.ozlabs.org/patch/1102717/ merged.

For other two tests and the above one can someone from ovn check and confirm the issue and plan for these tests as this is currently blocking master promotion.

[1] https://review.opendev.org/#/c/562527
[2] https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-standalone-full-tempest-master&job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/664233

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/664233
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=26f3127b2e6c710750a70048074126cfaca7ddd1
Submitter: Zuul
Branch: master

commit 26f3127b2e6c710750a70048074126cfaca7ddd1
Author: Kamil Sambor <email address hidden>
Date: Mon Jun 10 11:24:31 2019 +0200

    Skip dns related tests

    In ovn recently dns dependent tests were enabled.
    For some of them we need to add additional configuration
    and for another one we are waiting for OVS 2.12 to be
    released.

    Change-Id: I8c5a316145dda6497971be46f75370511a8f2103
    Partial-Bug: #1832166

[Kamil Sambor (ksambor)]

We can close this bug, because all mentioned tests are skipped in ovn too

[wes hayutin (weshayutin)]

/me notes that the skip does not fix the problem.. leave the bug open even if the skip is merged please

[Alex Schultz (alex-schultz)]

Can we renable this yet?

[Ronelle Landy (rlandy)]

https://review.opendev.org/#/c/688945/

[Ronelle Landy (rlandy)]

Closing this as the tests are being re-enabled.

[wes hayutin (weshayutin)]

http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021-train/fc94407/logs/stestr_results.html

http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021-master/438fdbb/logs/stestr_results.html

some of these tests are still failing in train and master

[Kamil Sambor (ksambor)]

https://review.opendev.org/#/c/698899/

[chandan kumar (chkumar246)]

Based on testing with latest tempest 23.0.0 and latest neutron tempest plugin with ovn-2.12
Below is the results from http://logs.rdoproject.org/83/24683/5/check/distgit-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset021-train/1d8507b/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz and https://review.opendev.org/#/c/688945/6/roles/validate-tempest/vars/tempest_skip_master.yml

* neutron_tempest_plugin.scenario.test_dns_integration.DNSIntegrationTests) ... SKIPPED: Designate support is required

* neutron_tempest_plugin.api.test_revisions.TestRevisions.test_update_dns_domain_bumps_revision ... SKIPPED: Server does not have DNS domain configured.

* neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest.test_dns_domain_and_name [87.592710s] ... FAILED

    testtools.matchers._impl.MismatchError: 'openstackgate.local' not in u'nameserver 1.1.1.1\nnameserver 1.0.0.1\nnameserver 38.145.33.91\n'

The last test is still failing, From this http://logs.rdoproject.org/83/24683/5/check/distgit-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset021-train/1d8507b/logs/undercloud/var/log/extra/rpm-list.txt.gz

python2-neutron-tests-tempest-0.7.1-0.20200129032133.4b58524.el7.noarch is used -> pointing to this commit https://github.com/openstack/neutron-tempest-plugin/commit/4b58524ab1e49a49c73ef2a5e24d087fc0cfa758

[Marios Andreou (marios-b)]

looking here on request, looks like the relevant network tests are still failing. More detail:

test_dns_domain_and_name and a bunch of other network related tests are still failing looking at run from today in

         * https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021-train/2bb37b5/logs/undercloud/var/log/tempest/stestr_results.html.gz

        * test_dns_domain_and_name[id-988347de-07af-471a-abfa-65aea9f452a6]
*=* 18:00:13 *=*=*= " "

        * test_add_remove_network_from_dhcp_agent[id-a0856713-6549-470c-a656-e97c8df9a14d]
 fail
        * test_dhcp_port_status_active[id-f164801e-1dd8-4b8b-b5d3-cc3ac77cfaa5]
         fail
        * test_list_networks_hosted_by_one_dhcp[id-30c48f98-e45d-4ffb-841c-b8aad57c7587]
 fail
        * test_dns_domain_and_name[id-988347de-07af-471a-abfa-65aea9f452a6]
 fail
        * test_add_remove_network_from_dhcp_agent[id-a0856713-6549-470c-a656-e97c8df9a14d]
 fail
        * test_list_networks_hosted_by_one_dhcp[id-30c48f98-e45d-4ffb-841c-b8aad57c7587]
 fail
        * test_port_security_macspoofing_port[compute,id-7c0bb1a2-d053-49a4-98f9-ca1a1d849f63,network,slow]
 fail