scenario tests tempest.scenario.test_network_v6.TestGettingAddress fails

[Hemanth Nakkina]

scenario tests under group tempest.scenario.test_network_v6.TestGettingAddress are failing (observed on job tempest-slow on nova stable/rocky patches)

Example of failure:
https://7734cba1607fe472e545-7d211d4380eeb6317dc985091f0b7f02.ssl.cf5.rackcdn.com/761824/2/check/tempest-slow/e0359e0/testr_results.html

tempest.lib.exceptions.BadRequest: Bad request
Details: {u'message': u'Bad router request: Cidr 2001:db8::/64 of subnet 417e4c02-8b8a-4b89-bee2-32d35dc61b93 overlaps with cidr 2001:db8::/64 of subnet 56123648-b56d-486c-8919-a3e97802906f.', u'detail': u'', u'type': u'BadRequest'}

Could be because of recently merged patch https://review.opendev.org/q/Id5d8ac09a38c656619f88a6f87b8f384fe4c55a8

[Lajos Katona (lajos-katona)]

The cidrs from one of the example really ovrlap:
Bad router request: Cidr 2001:db8::/64 of subnet 417e4c02-8b8a-4b89-bee2-32d35dc61b93 overlaps with cidr 2001:db8::/64 of subnet 56123648-b56d-486c-8919-a3e97802906f.'

One solution could be to use addressscopes in tempest perhaps.

[Hemanth Nakkina (hemanth-n)]

The following tempest commit should be pushed to tag 23.0.0 ( stable/rocky uses tempest branch 23.0.0 [1])

https://opendev.org/openstack/tempest/commit/4ad346e6497d4fbad9e379e8ed50730a0b6b55a3

[1] https://opendev.org/openstack/devstack/src/branch/stable/rocky/stackrc#L316

[Slawek Kaplonski (slaweq)]

IMO the problem is in the devstack's configuration (or tempest test) as it is using same IPv6 subnet 2001:db8::/64 for public network (devstack creates it) and later in the tests for private networks.
Patch https://review.opendev.org/q/Id5d8ac09a38c656619f88a6f87b8f384fe4c55a8 just fixed bug in Neutron and improved checks of the subnets to prevent such situation. I'm adding tempest as affected project as IMO bug is there, not in the Neutron code.

[Martin Kopec (mkopec)]

we have merged a patch which was supposed to address this overlap issue:
https://review.opendev.org/c/openstack/tempest/+/763621

it seems it didn't, at least not the ipv6 one

[Martin Kopec (mkopec)]

oh, is the failure happening only in stable/rocky?

[Hemanth Nakkina (hemanth-n)]

yes, the failure happens only in stable/rocky (since the commit https://opendev.org/openstack/tempest/commit/4ad346e6497d4fbad9e379e8ed50730a0b6b55a3 is not on tempest tag 23.0.0)

[Martin Kopec (mkopec)]

We discussed this during today's qa meeting:
http://eavesdrop.openstack.org/meetings/qa/2021/qa.2021-02-02-14.01.log.html#l-135

We cannot fix this for rocky as it is in EM stage and uses old tempest. Tempest does not support any EM stable branch and due to its brachless nature it is not possible for us to backport anything.

However, what we can do is we can skip the test for stable rocky and move unblock the tempest-slow.
gmann will push a review for that.

[Ghanshyam Mann (ghanshyammann)]

if the problem is fixed by https://opendev.org/openstack/tempest/commit/4ad346e6497d4fbad9e379e8ed50730a0b6b55a3 which is fixed after Tempest 23.0.0 (Tempest used in stable/rocky).

Tempest is branchless and we cannot backport the fix in older Tempest. stable/rocky is in an Extended maintenance state and Tempest master does not support it
- https://docs.openstack.org/tempest/latest/stable_branch_support_policy.html

The only way to make stable/rocky green is to skip this test.

[Slawek Kaplonski (slaweq)]

IMHO this can be maybe solved/workarounded in the u/s jobs by changing configuration of the jobs and by creating subnet in "public" network from different CIDR. Would that work for You Ghanshyam?

[Ghanshyam Mann (ghanshyammann)]

@Slawek sure if that passes the test then we can try that.

[Hemanth Nakkina (hemanth-n)]

Fix proposed - https://review.opendev.org/c/openstack/tempest/+/774764

[Martin Kopec (mkopec)]

https://review.opendev.org/c/openstack/tempest/+/774764 got merged

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 26.1.0 release.