neutron-tempest-plugin API test for list agents failure

Bug #1845330 reported by Slawek Kaplonski on 2019-09-25
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Low
Lajos Katona

Bug Description

I saw in d/s ci that it may happend in some cases that test neutron_tempest_plugin.api.admin.test_agent_management.AgentManagementTestJSON.test_list_agent may fails if e.g. in another test, like test_update_agent_description modifies same agent which is assigned to be "non modfied" agent.

Example of failure:

testtools.testresult.real._StringException: pythonlogging:'': {{{
2019-09-24 17:19:37,952 291890 INFO [tempest.lib.common.rest_client] Request (AgentManagementTestJSON:test_list_agent): 200 GET http://10.0.0.148:9696/v2.0/agents 0.594s
2019-09-24 17:19:37,953 291890 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'content-type': 'application/json', 'content-length': '11713', 'x-openstack-request-id': 'req-698a3a9e-4abe-4453-bb61-7d348032d734', 'date': 'Tue, 24 Sep 2019 17:19:37 GMT', 'connection': 'close', 'status': '200', 'content-location': 'http://10.0.0.148:9696/v2.0/agents'}
        Body: b'{"agents": [{"id": "29cbb538-fc95-4ec9-9d7d-5028a6ed7d82", "agent_type": "L3 agent", "binary": "neutron-l3-agent", "topic": "l3_agent", "host": "controller-0.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:20", "started_at": "2019-09-24 16:46:20", "heartbeat_timestamp": "2019-09-24 17:19:20", "description": "description for update agent.", "resources_synced": null, "availability_zone": "nova", "alive": true, "configurations": {"agent_mode": "legacy", "ex_gw_ports": 0, "floating_ips": 0, "gateway_external_network_id": "", "handle_internal_only_routers": true, "interface_driver": "neutron.agent.linux.interface.OVSInterfaceDriver", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0}}, {"id": "2c347a90-f262-47a2-b980-bab1aa1fb294", "agent_type": "DHCP agent", "binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "controller-1.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:16", "started_at": "2019-09-24 16:46:16", "heartbeat_timestamp": "2019-09-24 17:19:16", "description": null, "resources_synced": null, "availability_zone": "nova", "alive": true, "configurations": {"dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "dhcp_lease_duration": 86400, "log_agent_heartbeats": false, "networks": 0, "ports": 0, "subnets": 0}}, {"id": "41032db3-4508-4f93-b840-e9ca38f22479", "agent_type": "DHCP agent", "binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "controller-0.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:17", "started_at": "2019-09-24 16:46:17", "heartbeat_timestamp": "2019-09-24 17:19:17", "description": null, "resources_synced": null, "availability_zone": "nova", "alive": true, "configurations": {"dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "dhcp_lease_duration": 86400, "log_agent_heartbeats": false, "networks": 0, "ports": 0, "subnets": 0}}, {"id": "45d76247-ec5e-439d-b966-3045087e35e0", "agent_type": "Open vSwitch agent", "binary": "neutron-openvswitch-agent", "topic": "N/A", "host": "compute-0.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:30", "started_at": "2019-09-24 16:47:00", "heartbeat_timestamp": "2019-09-24 17:19:30", "description": null, "resources_synced": null, "availability_zone": null, "alive": true, "configurations": {"arp_responder_enabled": false, "bridge_mappings": {"datacentre": "br-ex", "tenant": "br-isolated"}, "datapath_type": "system", "devices": 0, "enable_distributed_routing": false, "extensions": ["qos"], "in_distributed_mode": false, "integration_bridge": "br-int", "l2_population": false, "log_agent_heartbeats": false, "ovs_capabilities": {"datapath_types": ["netdev", "system"], "iface_types": ["erspan", "geneve", "gre", "internal", "ip6erspan", "ip6gre", "lisp", "patch", "stt", "system", "tap", "vxlan"]}, "ovs_hybrid_plug": true, "resource_provider_bandwidths": {}, "resource_provider_inventory_defaults": {"allocation_ratio": 1.0, "min_unit": 1, "step_size": 1, "reserved": 0}, "tunnel_types": ["vxlan"], "tunneling_ip": "172.17.2.131", "vhostuser_socket_dir": "/var/run/openvswitch"}}, {"id": "5555b04e-ea86-498e-8c93-a83553e52bf1", "agent_type": "Metadata agent", "binary": "neutron-metadata-agent", "topic": "N/A", "host": "controller-0.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:23", "started_at": "2019-09-24 16:46:23", "heartbeat_timestamp": "2019-09-24 17:19:23", "description": null, "resources_synced": null, "availability_zone": null, "alive": true, "configurations": {"log_agent_heartbeats": false, "metadata_proxy_socket": "/var/lib/neutron/metadata_proxy", "nova_metadata_host": "overcloud.internalapi.localdomain", "nova_metadata_port": 8775}}, {"id": "948d978a-4df1-4de9-82aa-0ba2cc4a0da9", "agent_type": "Open vSwitch agent", "binary": "neutron-openvswitch-agent", "topic": "N/A", "host": "controller-1.redhat.local", "admin_state_up": true, "created_at": "2019-09-24 16:46:32", "started_at": "2019-09-24 16:47:02", "heartbeat_timestamp": "2019-09-24 17:19:32", "description": null, "resources_synced": null, "availability_zone": null, "alive":
}}}

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/api/admin/test_agent_management.py", line 42, in test_list_agent
    self.assertIn(self.agent, agents)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 417, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: {'id': '29cbb538-fc95-4ec9-9d7d-5028a6ed7d82', 'agent_type': 'L3 agent', 'binary': 'neutron-l3-agent', 'topic': 'l3_agent', 'host': 'controller-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:20', 'started_at': '2019-09-24 16:46:20', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True} not in [{'id': '29cbb538-fc95-4ec9-9d7d-5028a6ed7d82', 'agent_type': 'L3 agent', 'binary': 'neutron-l3-agent', 'topic': 'l3_agent', 'host': 'controller-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:20', 'started_at': '2019-09-24 16:46:20', 'description': 'description for update agent.', 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': '2c347a90-f262-47a2-b980-bab1aa1fb294', 'agent_type': 'DHCP agent', 'binary': 'neutron-dhcp-agent', 'topic': 'dhcp_agent', 'host': 'controller-1.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:16', 'started_at': '2019-09-24 16:46:16', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': '41032db3-4508-4f93-b840-e9ca38f22479', 'agent_type': 'DHCP agent', 'binary': 'neutron-dhcp-agent', 'topic': 'dhcp_agent', 'host': 'controller-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:17', 'started_at': '2019-09-24 16:46:17', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': '45d76247-ec5e-439d-b966-3045087e35e0', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'compute-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:30', 'started_at': '2019-09-24 16:47:00', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': '5555b04e-ea86-498e-8c93-a83553e52bf1', 'agent_type': 'Metadata agent', 'binary': 'neutron-metadata-agent', 'topic': 'N/A', 'host': 'controller-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:23', 'started_at': '2019-09-24 16:46:23', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': '948d978a-4df1-4de9-82aa-0ba2cc4a0da9', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'controller-1.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:32', 'started_at': '2019-09-24 16:47:02', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': 'a11efdd7-83ce-4449-8fd4-fbf11ed86ecc', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'controller-0.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:33', 'started_at': '2019-09-24 16:47:03', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': 'aacd3aee-caf6-403e-8890-5fce6867ce24', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'controller-2.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:32', 'started_at': '2019-09-24 16:47:02', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': 'c940f293-0e33-4586-bfce-4caba48560a4', 'agent_type': 'Metadata agent', 'binary': 'neutron-metadata-agent', 'topic': 'N/A', 'host': 'controller-2.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:23', 'started_at': '2019-09-24 16:46:23', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': 'cfce4183-5ff5-49a2-9957-54434b218afd', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'compute-1.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:29', 'started_at': '2019-09-24 16:46:59', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}, {'id': 'd8c1b0b7-a84d-48b3-be24-184c44c2123a', 'agent_type': 'DHCP agent', 'binary': 'neutron-dhcp-agent', 'topic': 'dhcp_agent', 'host': 'controller-2.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:17', 'started_at': '2019-09-24 16:46:17', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': 'dae0d572-2edb-4d39-9e7a-43206c0d234d', 'agent_type': 'L3 agent', 'binary': 'neutron-l3-agent', 'topic': 'l3_agent', 'host': 'controller-1.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:20', 'started_at': '2019-09-24 16:46:20', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': 'daf0c85a-1472-4dfb-816b-7aad7ee70a8b', 'agent_type': 'L3 agent', 'binary': 'neutron-l3-agent', 'topic': 'l3_agent', 'host': 'controller-2.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:20', 'started_at': '2019-09-24 16:46:20', 'description': None, 'resources_synced': None, 'availability_zone': 'nova', 'alive': True}, {'id': 'e7651528-ed52-489e-9e58-31105aad469c', 'agent_type': 'Metadata agent', 'binary': 'neutron-metadata-agent', 'topic': 'N/A', 'host': 'controller-1.redhat.local', 'admin_state_up': True, 'created_at': '2019-09-24 16:46:22', 'started_at': '2019-09-24 16:46:22', 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True}]

Changed in neutron:
assignee: Slawek Kaplonski (slaweq) → Lajos Katona (lajos-katona)
status: Confirmed → In Progress

Reviewed: https://review.opendev.org/686398
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=b57453f84c06faddcded02530adb58afcdc3ded6
Submitter: Zuul
Branch: master

commit b57453f84c06faddcded02530adb58afcdc3ded6
Author: elajkat <email address hidden>
Date: Thu Oct 3 15:09:05 2019 +0200

    Make one agent untouchable in test_agent_management

    In test_agent_management some tests depend on one agent to be unmodified
    which was not guaranteed as some tests updated that agent.
    The update happened because the update test assumed that neutron will
    return list of agents in different order.
    The fix is only to select always different agent for update than the one
    used for other tests.

    Change-Id: I7af97ac0ee396d2eb132459d29e938bd5f3de9aa
    Closes-Bug: #1845330

Changed in neutron:
status: In Progress → Fix Released

This issue was fixed in the openstack/neutron-tempest-plugin 0.7.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers