RBAC-enabled tempest test failing - in periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-rbac-master - IndexError: list index out of range

[Ronelle Landy]

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-rbac-master/ff87c9e/logs/undercloud/var/log/tempest/stestr_results.html.gz shows

neutron_tempest_plugin.scenario.admin.test_floatingip.FloatingIpTestCasesAdmin.test_two_vms_fips

fails with the following trace:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py", line 102, in test_two_vms_fips
    servers, server_ssh_clients, fips = self._create_vms(hyper, avail_zone)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py", line 81, in _create_vms
    )['ports'][0]
IndexError: list index out of range

second log: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-rbac-master/fb3a235/logs/undercloud/var/log/tempest/stestr_results.html.gz

The same test in an non-RBAC-enabled test passes on the same DLRN hash:

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/ce14b1e/logs/undercloud/var/log/tempest/stestr_results.html.gz

[Ronelle Landy (rlandy)]

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-rbac-master/ff87c9e/logs/undercloud/var/log/extra/podman/containers/ironic_neutron_agent/log/neutron/server.log.txt.gz shows some keystone errors:

2021-11-29 18:11:19.040 31 WARNING keystoneauth.identity.generic.base [-] Failed to discover available identity versions when contacting http://192.168.24.3:5000. Attempting to parse version from URL.: keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://192.168.24.3:5000: HTTPConnectionPool(host='192.168.24.3', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3a9832beb8>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))
2021-11-29 18:11:19.048 33 WARNING neutron.scheduler.dhcp_agent_scheduler [req-059da77e-29a5-4112-9059-f794fa4e78ae - - - - -] No more DHCP agents
2021-11-29 18:11:19.050 33 DEBUG neutron.scheduler.base_scheduler [req-059da77e-29a5-4112-9059-f794fa4e78ae - - - - -] Resources bound (agent type, host, resource id): schedule /usr/lib/python3.6/site-packages/neutron/scheduler/base_scheduler.py:58
2021-11-29 18:11:19.050 33 INFO neutron.db.agentschedulers_db [req-059da77e-29a5-4112-9059-f794fa4e78ae - - - - -] Failed to schedule network b1cf025a-7f80-4f38-86d4-34aa2aec0a3f, no eligible agents or it might be already scheduled by another server
2021-11-29 18:11:19.083 33 WARNING keystoneauth.identity.generic.base [-] Failed to discover available identity versions when contacting http://192.168.24.3:5000. Attempting to parse version from URL.: keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://192.168.24.3:5000: HTTPConnectionPool(host='192.168.24.3', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3a981d4a58>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))

[Lajos Katona (lajos-katona)]

Thanks for reporting this issue.
I think the problem is in neutron-tempest-plugin, as [1] uses os_admin to create server if availabilty_zones are in the call for create_server

[1]: https://opendev.org/openstack/neutron-tempest-plugin/src/branch/master/neutron_tempest_plugin/scenario/base.py#L128

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/820039

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/820039
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/c75ba5028fd1abfca557351eaf0da086599a3be8
Submitter: "Zuul (22348)"
Branch: master

commit c75ba5028fd1abfca557351eaf0da086599a3be8
Author: elajkat <email address hidden>
Date: Wed Dec 1 15:54:28 2021 +0100

    Use admin_client to list admin's servers' ports

    FloatingIpTestCasesAdmin creates VMs as admin (to be sure to boot on
    availability_zone) so to list ports for the VM we have to use
    admin_client.

    Change-Id: I1da9ccc99b70355554ace3a20a237bb4f33381ab
    Closes-Bug: #1952806

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-tempest-plugin 1.9.0 release.