Wallaby fs001 failed with different tempest failures with error in neutron server logs: "ERROR neutron_lib.callbacks.manager TypeError: _handle_lb_on_ls() got an unexpected keyword argument 'context'"

[Sandeep Yadav]

Description:

Wallaby fs001 failed with different tempest failures with error in neutron server logs: "ERROR neutron_lib.callbacks.manager TypeError: _handle_lb_on_ls() got an unexpected keyword argument 'context'"

Tempest failures:-

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/66bc522/logs/undercloud/var/log/tempest/failing_tests.log.txt.gz

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/4a6e217/logs/undercloud/var/log/tempest/failing_tests.log.txt.gz

~~~
tempest.api.compute.images.test_images_oneserver_negative.ImagesOneServerNegativeTestJSON.test_create_second_image_when_first_image_is_being_saved[id-0460efcf-ee88-4f94-acef-1bf658695456,negative]
tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_pause_state[id-943bd6e8-4d7a-4904-be83-7a6cc2d4213b]
setUpClass (tempest.api.compute.floating_ips.test_floating_ips_actions_negative.FloatingIPsAssociationNegativeTestJSON)
tempest.api.compute.images.test_images.ImagesTestJSON.test_create_image_from_paused_server[id-71bcb732-0261-11e7-9086-fa163e4fa634]
tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_shelved_state[id-bb0cb402-09dd-4947-b6e5-5e7e1cfa61ad]
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesV270Test.test_create_get_list_interfaces[id-2853f095-8277-4067-92bd-9f10bd4f8e0c,network]
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port[id-73fe8f02-590d-4bf1-b184-e9ca81065051,network]
tempest.api.compute.images.test_images.ImagesTestJSON.test_delete_saving_image[id-aa06b52b-2db5-4807-b218-9441f75d74e3]
tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_suspended_state[id-1f82ebd3-8253-4f4e-b93f-de9b7df56d8b]
setUpClass (tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON)
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers[id-2f3a0127-95c7-4977-92d2-bc5aec602fb4]
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_list_security_groups_by_server[id-79517d60-535a-438f-af3d-e6feab1cbea7]
setUpClass (tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON)
tempest.api.compute.servers.test_multiple_create.MultipleCreateTestJSON.test_multiple_create[id-61e03386-89c3-449c-9bb1-a06f423fd9d1]
setUpClass (tempest.api.compute.servers.test_create_server.ServersTestManualDisk)
tempest.api.compute.servers.test_multiple_create.MultipleCreateTestJSON.test_multiple_create_with_reservation_return[id-864777fb-2f1e-44e3-b5b9-3eb6fa84f2f7]
setUpClass (tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON)
tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devices[id-a2e65a6c-66f1-4442-aaa8-498c31778d96,image,network,slow,volume]
setUpClass (tempest.api.compute.servers.test_server_addresses_negative.ServerAddressesNegativeTestJSON)
tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest_v242.test_tagged_boot_devices[id-a2e65a6c-66f1-4442-aaa8-498c31778d96,image,network,slow,volume]
tempest.api.compute.volumes.test_attach_volume_negative.AttachVolumeNegativeTest.test_attach_attached_volume_to_different_server[id-ee37a796-2afb-11e7-bc0f-fa163e65f5ce,negative]
setUpClass (tempest.api.compute.servers.test_server_rescue.ServerRescueTestJSON)
tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_server_specify_multibyte_character_name[id-defbaca5-d611-49f5-ae21-56ee25d2db49]
tempest.api.compute.volumes.test_attach_volume_negative.AttachVolumeNegativeTest.test_delete_attached_volume[id-a313b5cd-fbd0-49cc-94de-870e99f763c7,negative]
setUpClass (tempest.api.compute.servers.test_server_rescue_negative.ServerRescueNegativeTestJSON)
setUpClass (tempest.api.compute.servers.test_server_tags.ServerTagsTestJSON)
tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_server_with_ipv6_addr_only[id-38fb1d02-c3c5-41de-91d3-9bc2025a75eb]
setUpClass (tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON)
tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_with_existing_server_name[id-8fea6be7-065e-47cf-89b8-496e6f96c699]
tempest.api.compute.volumes.test_attach_volume.AttachVolumeShelveTestJSON.test_attach_volume_shelved_or_offload_server[id-13a940b6-3474-4c3c-b03f-29b89112bfee,slow]
tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume[id-52e9045a-e90d-4c0d-9087-79d657faffff,slow]
tempest.api.volume.test_volumes_snapshots.VolumesSnapshotTestJSON.test_snapshot_create_offline_delete_online[compute,id-5210a1de-85a0-11e6-bb21-641c676a5d61]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks[compute,id-1546850e-fbaa-42f5-8b5f-03d8a6a95f15,network,slow]
tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_list_get_volume_attachments[id-7fa563fe-f0f7-43eb-9e22-a1ece036b513]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
~~~

Neutron server logs:-

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/4a6e217/logs/overcloud-controller-0/var/log/containers/neutron/server.log.1.gz

~~~
2022-05-09 11:17:51.201 15 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2022-05-09 11:17:51.201 15 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3.9/site-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2022-05-09 11:17:51.201 15 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs)
2022-05-09 11:17:51.201 15 ERROR neutron_lib.callbacks.manager TypeError: _handle_lb_on_ls() got an unexpected keyword argument 'context'
2022-05-09 11:17:51.201 15 ERROR neutron_lib.callbacks.manager
~~~

[Sandeep Yadav (sandeepyadav93)]

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/4a6e217/logs/overcloud-novacompute-0/var/log/containers/nova/nova-compute.log.txt.gz

~~~
allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] Traceback (most recent call last):
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 7231, in _create_guest_with_network
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] guest = self._create_guest(
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] next(self.gen)
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 479, in wait_for_instance_event
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] actual_event = event.wait()
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] result = hub.switch()
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] return self.greenlet.switch()
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] eventlet.timeout.Timeout: 300 seconds
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419]
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] During handling of the above exception, another exception occurred:
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419]
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] Traceback (most recent call last):
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 2409, in _build_and_run_instance
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737e06f9-66ce-4d74-a441-def50a208419] self.driver.spawn(context, instance, image_meta,
2022-05-09 11:27:20.019 2 ERROR nova.compute.manager [instance: 737...