Octavia Tempest Tests - Override Network/Subnet Errors

[Ayhan ilhan]

Hi,

I want override network/subnet creation but I had a some "KeyError: 'id'" and "TypeError: 'dict_values' object is not subscriptable" errors in scenario tests. When I didn't override network/subnet creation, I don't get any error.

I debugged this scenario tests and i found some bugs in code. When I changed this lines in code, I was able to run the test successfully. I am attaching below some lines that I changed.

Openstack Version: Zed
Tempest Version: tempest 35.0.1.dev35

tempest.conf is as follows:
```
...
#
# Networking
#

# When true the IPv6 tests will be run.
#test_with_ipv6 = true
test_with_ipv6 = false

# True if your cloud does not allow creating networks or specifying the boot network for instances.
#disable_boot_network = false
disable_boot_network = true

# When true, security groups will be created for the test servers.
# When false, port security will be disabled on the created networks.
#enable_security_groups = false
enable_security_groups = true

# Overrides network creation and uses this network ID for
# all tests (VIP, members, etc.). Required if test_subnet_override is set.
test_network_override = '9139bf47-xxxx-45e4-a4fb-e786c4a798b3'

# Overrides subnet creation and uses this subnet ID for all IPv4 tests (VIP, members, etc.). Optional
test_subnet_override = '8b07655f-xxxx-4091-92e5-224e26c4bfdc'
...
```

Errors:
```
Error 1:
Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/api/v2/test_member.py", line 2509, in test_UDP_SI_member_batch_update
    self._test_member_batch_update(const.UDP,

      File "/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/api/v2/test_member.py", line 2641, in _test_member_batch_update
    member1_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet[

    KeyError: 'id'

Error 2:
Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/opt/venv/tempest/lib/python3.8/site-packages/tempest/test.py", line 206, in setUpClass
    raise value.with_traceback(trace)

      File "/opt/venv/tempest/lib/python3.8/site-packages/tempest/test.py", line 199, in setUpClass
    cls.resource_setup()

      File "/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/scenario/v2/test_traffic_ops.py", line 54, in resource_setup
    super(TrafficOperationsScenarioTest, cls).resource_setup()

      File "/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/test_base.py", line 796, in resource_setup
    server_details = cls._create_webserver('lb_member_webserver1',

      File "/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/test_base.py", line 997, in _create_webserver
    instance_network = addresses.values()[0]

    TypeError: 'dict_values' object is not subscriptable
```

Debugging example:
```
$ stestr run --pdb octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest.test_healthmonitor_udp_traffic
> /etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/scenario/v2/test_traffic_ops.py(312)_test_healthmonitor_traffic()
-> if self.lb_member_1_subnet:
(Pdb) print(self.lb_member_1_subnet)
response: {'content-type': 'application/json', 'content-length': '722', 'x-openstack-request-id': 'req-d325bacb-10ac-xxxx-xxxx-7227fbd42bb9', 'date': 'Wed, 03 Jan 2024 10:29:54 GMT', 'connection': 'close', 'status': '200', 'content-location': 'https://xxxx:9696/v2.0/subnets/8b07655f-xxxx-4091-92e5-224e26c4bfdc'}
Body: {'subnet': {'id': '8b07655f-xxxx-4091-92e5-224e26c4bfdc', 'name': 'tempest-bgp-subnet-01', 'tenant_id': 'xxxxxxx48e5b15d762ad8249d20', 'network_id': '9139bf47-xxxx-45e4-a4fb-e786c4a798b3', 'ip_version': 4, 'subnetpool_id': '36bed34c-xxxx-xxxx-9482-d38526017c7f', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'gateway_ip': '10.xxx.xxx.xxx', 'cidr': '10.xxx.xxx.xxx/24', 'allocation_pools': [{'start': '10.xxx.xxx.xxx', 'end': '10.xxx.xxx.xxx'}], 'host_routes': [], 'dns_nameservers': ['xxx.xxx.xxx.xxx', 'xxx.xxx.xxx.xxx'], 'description': '', 'service_types': [], 'dns_publish_fixed_ip': False, 'tags': [], 'created_at': '2023-07-27T09:16:13Z', 'updated_at': '2023-07-27T09:16:13Z', 'revision_number': 0, 'project_id': 'xxxxxx48e5b15d762ad8249d20'}}
(Pdb) print(self.lb_member_1_subnet[const.ID])
*** KeyError: 'id'
(Pdb) print(self.lb_member_1_subnet['subnet'][const.ID])
8b07655f-xxxx-xxxx-92e5-224e26c4bfdc
```

Changes Made:
/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/test_base.py
Line998:
#instance_network = addresses.values()[0]
instance_network = list(addresses.values())[0]

Line561:
subnet_id = cls.lb_member_vip_subnet['subnet'][const.ID]

Line300:
LOG.debug('Octavia Setup: lb_member_vip_subnet = {}'.format(
                cls.lb_member_vip_subnet['subnet'][const.ID]))

/etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/api/v2/test_member.py
Line886:
#member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet[const.ID]
member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet['subnet'][const.ID]

Line1754:
#member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet[const.ID]
member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet['subnet'][const.ID]

Line2197:
#member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet[const.ID]
member_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet['subnet'][const.ID]

Line2641:
#member1_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet[const.ID]
member1_kwargs[const.SUBNET_ID] = self.lb_member_vip_subnet['subnet'][const.ID]
....

Best Regards,

[Gregory Thiemonge (gthiemonge)]

Hi Ayhan,

Thanks for reporting it,
Unfortunately we don't have CI jobs that cover the test_{network,subnet}_override parameters, so this issue was not caught earlier.

I reproduced the backtrace in my environment and I may have a patch to fix it. I was not able to fully test it, I got some issues later in the tests (probably related to my env)

The first issue should be fixed with:

diff --git a/octavia_tempest_plugin/tests/test_base.py b/octavia_tempest_plugin/tests/test_base.py
index f08cec9..83c1b0c 100644
--- a/octavia_tempest_plugin/tests/test_base.py
+++ b/octavia_tempest_plugin/tests/test_base.py
@@ -261,6 +261,7 @@ class LoadBalancerBaseTest(validators.ValidatorsMixin,
         elif CONF.load_balancer.test_network_override:
             if conf_lb.test_subnet_override:
                 override_subnet = show_subnet(conf_lb.test_subnet_override)
+ override_subnet = override_subnet.get('subnet')
             else:
                 override_subnet = None

For the second issue, could you print the value of 'addresses' just before 'instance_network = addresses.values()[0]'?

[Gregory Thiemonge (gthiemonge)]

I think you're right with the list(addresses.values())

The full patch should be:

diff --git a/octavia_tempest_plugin/tests/test_base.py b/octavia_tempest_plugin/tests/test_base.py
index f08cec9..6243d19 100644
--- a/octavia_tempest_plugin/tests/test_base.py
+++ b/octavia_tempest_plugin/tests/test_base.py
@@ -261,6 +261,7 @@ class LoadBalancerBaseTest(validators.ValidatorsMixin,
         elif CONF.load_balancer.test_network_override:
             if conf_lb.test_subnet_override:
                 override_subnet = show_subnet(conf_lb.test_subnet_override)
+ override_subnet = override_subnet.get('subnet')
             else:
                 override_subnet = None

@@ -279,6 +280,7 @@ class LoadBalancerBaseTest(validators.ValidatorsMixin,
                     conf_lb.test_IPv6_subnet_override):
                 override_ipv6_subnet = show_subnet(
                     conf_lb.test_IPv6_subnet_override)
+ override_ipv6_subnet = override_ipv6_subnet.get('subnet')
                 cls.lb_member_vip_ipv6_subnet = override_ipv6_subnet
                 cls.lb_member_1_ipv6_subnet = override_ipv6_subnet
                 cls.lb_member_2_ipv6_subnet = override_ipv6_subnet
@@ -1002,7 +1004,7 @@ class LoadBalancerBaseTestWithCompute(LoadBalancerBaseTest):

         addresses = server['addresses']
         if CONF.load_balancer.disable_boot_network:
- instance_network = addresses.values()[0]
+ instance_network = list(addresses.values())[0]
         else:
             instance_network = addresses[network['name']]
         for addr in instance_network:

[Ayhan ilhan (ayhanilhan)]

Hi,

Thank you for your quick response. I have shared list(addresses.values())[0] and addresses.values() values as below:

$ stestr run --pdb octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest.test_healthmonitor_udp_traffic
> /etc/tempest/octavia-tempest-plugin/octavia_tempest_plugin/tests/test_base.py(999)_create_webserver()
-> instance_network = list(addresses.values())[0]
(Pdb) print(addresses.values())
dict_values([[{'version': 4, 'addr': '10.xxx.xx.17', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:d2:0d:42'}]])
(Pdb) print(list(addresses.values())[0])
[{'version': 4, 'addr': '10.xxx.xx.17', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:d2:0d:42'}]

Regards,