Cannot associate floating IP with port, the fixed IP already has a floating IP

[Martin Kopec]

tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
 ----------------------------------------------------------------------------------------------------------------------------------------------

 Captured traceback:
 ~~~~~~~~~~~~~~~~~~~
     Traceback (most recent call last):

       File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 70, in wrapper
     return f(*func_args, **func_kwargs)

       File "/opt/stack/tempest/tempest/scenario/test_server_basic_ops.py", line 141, in test_server_basic_ops
    self.verify_ssh(keypair)

       File "/opt/stack/tempest/tempest/scenario/test_server_basic_ops.py", line 55, in verify_ssh
     fip = self.create_floating_ip(self.instance)

       File "/opt/stack/tempest/tempest/scenario/manager.py", line 1082, in create_floating_ip
     result = client.create_floatingip(**floatingip_kwargs)

       File "/opt/stack/tempest/tempest/lib/services/network/floating_ips_client.py", line 30, in create_floatingip
     return self.create_resource(uri, post_data)

       File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 62, in create_resource
     resp, body = self.post(req_uri, req_post_data)

       File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 300, in post
     return self.request('POST', url, extra_headers, headers, body, chunked)

       File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 742, in request
     self._error_checker(resp, resp_body)

       File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 867, in _error_checker
     raise exceptions.Conflict(resp_body, resp=resp)

     tempest.lib.exceptions.Conflict: Conflict with state of target resource
 Details: {'type': 'FloatingIPPortAlreadyAssociated', 'message': 'Cannot associate floating IP 172.24.5.149 (e599dd04-a442-4aa8-a091-95bf9cb8804e) with port bf0892f9-1e1e-4d29-b6e5-8b4c6a347dcd using fixed IP 10.100.0.4, as that fixed IP already has a floating IP on external network 24f99b76-a37e-4bc1-ba55-bb5d39a77095.', 'detail': ''}

The failure was seen in https://zuul.openstack.org/builds?job_name=python-tempestconf-tempest-devstack-admin&skip=0 job.

After an investigation we found the culprit - this change caused the issue:
https://review.opendev.org/c/openstack/tempest/+/881675

The test in question has been passing just fine (even after the 881675 change was merged) - we compared the tempest.conf files in jobs voting in tempest and the ones in python-tempestconf (which hit the issue).

The main difference was that python-tempestconf jobs set the following value in tempest.conf
[network].floating_network_name = public

When floating_network_name is set, we end up in this if branch where we call create_floating_ip method:
https://opendev.org/openstack/tempest/src/commit/4054e13e65f7d11c9097946a36976bf9c6edea45/tempest/scenario/test_server_basic_ops.py#L55-L57

If we don't set flaoting_network_name we end up in the else branch where we call get_server_ip method instead:
https://opendev.org/openstack/tempest/src/commit/4054e13e65f7d11c9097946a36976bf9c6edea45/tempest/scenario/test_server_basic_ops.py#L59-L61

get_server_ip calls create_floating_ip IF CONF.validation.connect_method == floating and there is no floating ip yet:
https://opendev.org/openstack/tempest/src/commit/58e17903eb6a2f481c42270d11b89100b8f69419/tempest/scenario/manager.py#L1194-L1205

'if there is no floating ip already associated with the server' is important here - that checks never happens if we set CONF.network.floating_network_name and we go directly to creating the floating ip ... I believe that causes the above traceback, somehow.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tempest/+/884584

[Ghanshyam Mann (ghanshyammann)]

As 881675 did the change of handling the floating IPs in get_server_ip() method, we do not need to set/create the FIP in verify_ssh() method.

As Lukas mentioned in the proposed change, we do not need to create/deal with the FIP. remove the L53-L58 and just keep self.get_server_ip and it will work.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tempest/+/884584
Committed: https://opendev.org/openstack/tempest/commit/86b8b6204c08aa304b57c0758b2ae3321513614b
Submitter: "Zuul (22348)"
Branch: master

commit 86b8b6204c08aa304b57c0758b2ae3321513614b
Author: Roman Popelka <email address hidden>
Date: Mon May 29 12:06:51 2023 +0200

    [FIX] Floating IP already exists error

    if floating_network_name parameter is set in tempest.conf,
    scenario.test_server_basic_ops.verify_ssh test fails
    if server already has floating ip set, since it doesn't
    check for it's existence. This patch fixes this issue.
    More info here [1]

    Since get_server_ip() is able to return both fixed and floating ip,
    there is no reason to keep the if-else statement anymore.

    [1] https://bugs.launchpad.net/tempest/+bug/2020659

    Closes-Bug: 2020659
    Change-Id: I00a87fe488ebc383c913fa9757f98e19aa2605e9

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 35.0.0 release.