| 2022-04-05 21:00:24 |
Alexander Balderson |
bug |
|
|
added bug |
| 2022-04-05 21:04:05 |
Alexander Balderson |
description |
On a deployment of Focal Ussuri which was CIS hardened SQA had two tempest tests which failed to resize a server, and then revert the resize.
the two tests which failed were:
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm
and
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
The nova compute logs show:
: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.648 653208 ERROR nova.virt.libvirt.driver [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.697 653208 INFO os_vif [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:5f:7c,bridge_name='br-int',has_traffic_filtering=True,id=c6c15dff-9201-49e9-9d86-4ce684138f53,network=Network(611f2961-05f5-4361-a30f-bcf384865f6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc6c15dff-92')
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Setting instance vm_state to ERROR: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Traceback (most recent call last):
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 10047, in _error_out_instance_on_exception
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] yield
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5904, in _finish_resize_helper
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] network_info = self._finish_resize(context, instance, migration,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5842, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self._set_instance_info(instance, old_flavor)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.force_reraise()
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] six.reraise(self.type_, self.value, self.tb)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] raise value
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5825, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.driver.finish_migration(context, migration, instance,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 10410, in finish_migration
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] guest = self._create_domain_and_network(context, xml, instance,
...
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a]
for both tests.
our CIS rule set is
RULESET1="1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 1.1.1.5 1.1.1.6 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.1.8 1.1.9 1.1.12 1.1.13 1.1.14 1.1.18 1.1.19 1.1.20 1.1.21 1.1.22 1.1.23 1.1.24 1.2.1 1.2.2 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 1.5.2 1.5.3 1.6.1 1.6.2 1.6.3 1.6.4 1.7.1.1 1.7.1.2 1.7.1.3 1.8.1.1 1.8.1.2 1.8.1.3 1.8.1.4 1.8.1.5 1.8.1.6 1.9 1.10"
RULESET2="2.1.1 2.1.2 2.2.1.1 2.2.1.2 2.2.1.3 2.2.1.4 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.2.8 2.2.9 2.2.10 2.2.11 2.2.12 2.2.13 2.2.14 2.2.15 2.2.17 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.4"
RULESET3="3.1.2 3.2.1 3.2.2 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 3.3.6 3.3.7 3.3.8 3.3.9 3.5.1.1 3.5.1.2 3.5.1.3 3.5.1.4 3.5.1.5 3.5.1.6 3.5.1.7 3.5.2.1 3.5.2.2 3.5.2.3 3.5.2.4 3.5.2.5 3.5.2.6 3.5.2.7 3.5.2.8 3.5.2.9 3.5.2.10 3.5.3.1.1 3.5.3.1.2 3.5.3.2.1 3.5.3.2.2 3.5.3.2.3 3.5.3.2.4 3.5.3.3.1 3.5.3.3.2 3.5.3.3.3 3.5.3.3.4"
RULESET4="4.2.1.1 4.2.1.2 4.2.1.3 4.2.1.4 4.2.1.5 4.2.1.6 4.2.2.1 4.2.2.2 4.2.2.3 4.2.3 4.3 4.4"
RULESET5="5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.1.7 5.1.8 5.1.9 5.2.1 5.2.2 5.2.3 5.2.4 5.2.6 5.2.7 5.2.8 5.2.9 5.2.10 5.2.11 5.2.12 5.2.13 5.2.14 5.2.15 5.2.16 5.2.17 5.2.18 5.2.19 5.2.21 5.2.22 5.3.1 5.3.2 5.3.3 5.3.4 5.4.1.1 5.4.1.2 5.4.1.3 5.4.1.4 5.4.1.5 5.4.2 5.4.3 5.4.4 5.4.5 5.5 5.6"
RULESET6="6.1.2 6.1.3 6.1.4 6.1.5 6.1.6 6.1.7 6.1.8 6.1.9 6.1.10 6.1.11 6.1.126.1.13 6.1.14 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.2.6 6.2.7 6.2.8 6.2.9 6.2.10 6.2.11 6.2.12 6.2.13 6.2.14 6.2.15 6.2.16 6.2.17"
metal systems get the additional rules:
"4.1.1.1 4.1.1.2 4.1.1.3 4.1.1.4 4.1.2.1 4.1.2.2 4.1.2.3 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.6 4.1.7 4.1.8 4.1.9 4.1.10 4.1.11 4.1.12 4.1.13 4.1.14 4.1.15 4.1.16 4.1.17 |
On a deployment of Focal Ussuri which was CIS hardened SQA had two tempest tests which failed to resize a server, and then revert the resize.
the two tests which failed were:
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm
and
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
The nova compute logs show:
: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.648 653208 ERROR nova.virt.libvirt.driver [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.697 653208 INFO os_vif [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:5f:7c,bridge_name='br-int',has_traffic_filtering=True,id=c6c15dff-9201-49e9-9d86-4ce684138f53,network=Network(611f2961-05f5-4361-a30f-bcf384865f6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc6c15dff-92')
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Setting instance vm_state to ERROR: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Traceback (most recent call last):
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 10047, in _error_out_instance_on_exception
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] yield
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5904, in _finish_resize_helper
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] network_info = self._finish_resize(context, instance, migration,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5842, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self._set_instance_info(instance, old_flavor)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.force_reraise()
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] six.reraise(self.type_, self.value, self.tb)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] raise value
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5825, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.driver.finish_migration(context, migration, instance,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 10410, in finish_migration
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] guest = self._create_domain_and_network(context, xml, instance,
...
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a]
for both tests.
our CIS rule set is
RULESET1="1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 1.1.1.5 1.1.1.6 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.1.8 1.1.9 1.1.12 1.1.13 1.1.14 1.1.18 1.1.19 1.1.20 1.1.21 1.1.22 1.1.23 1.1.24 1.2.1 1.2.2 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 1.5.2 1.5.3 1.6.1 1.6.2 1.6.3 1.6.4 1.7.1.1 1.7.1.2 1.7.1.3 1.8.1.1 1.8.1.2 1.8.1.3 1.8.1.4 1.8.1.5 1.8.1.6 1.9 1.10"
RULESET2="2.1.1 2.1.2 2.2.1.1 2.2.1.2 2.2.1.3 2.2.1.4 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.2.8 2.2.9 2.2.10 2.2.11 2.2.12 2.2.13 2.2.14 2.2.15 2.2.17 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.4"
RULESET3="3.1.2 3.2.1 3.2.2 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 3.3.6 3.3.7 3.3.8 3.3.9 3.5.1.1 3.5.1.2 3.5.1.3 3.5.1.4 3.5.1.5 3.5.1.6 3.5.1.7 3.5.2.1 3.5.2.2 3.5.2.3 3.5.2.4 3.5.2.5 3.5.2.6 3.5.2.7 3.5.2.8 3.5.2.9 3.5.2.10 3.5.3.1.1 3.5.3.1.2 3.5.3.2.1 3.5.3.2.2 3.5.3.2.3 3.5.3.2.4 3.5.3.3.1 3.5.3.3.2 3.5.3.3.3 3.5.3.3.4"
RULESET4="4.2.1.1 4.2.1.2 4.2.1.3 4.2.1.4 4.2.1.5 4.2.1.6 4.2.2.1 4.2.2.2 4.2.2.3 4.2.3 4.3 4.4"
RULESET5="5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.1.7 5.1.8 5.1.9 5.2.1 5.2.2 5.2.3 5.2.4 5.2.6 5.2.7 5.2.8 5.2.9 5.2.10 5.2.11 5.2.12 5.2.13 5.2.14 5.2.15 5.2.16 5.2.17 5.2.18 5.2.19 5.2.21 5.2.22 5.3.1 5.3.2 5.3.3 5.3.4 5.4.1.1 5.4.1.2 5.4.1.3 5.4.1.4 5.4.1.5 5.4.2 5.4.3 5.4.4 5.4.5 5.5 5.6"
RULESET6="6.1.2 6.1.3 6.1.4 6.1.5 6.1.6 6.1.7 6.1.8 6.1.9 6.1.10 6.1.11 6.1.126.1.13 6.1.14 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.2.6 6.2.7 6.2.8 6.2.9 6.2.10 6.2.11 6.2.12 6.2.13 6.2.14 6.2.15 6.2.16 6.2.17"
metal systems get the additional rules:
"4.1.1.1 4.1.1.2 4.1.1.3 4.1.1.4 4.1.2.1 4.1.2.2 4.1.2.3 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.6 4.1.7 4.1.8 4.1.9 4.1.10 4.1.11 4.1.12 4.1.13 4.1.14 4.1.15 4.1.16 4.1.17
crashdump can be found at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/juju-crashdump-openstack-2022-04-03-03.39.08.tar.gz
with testrun at:
https://solutions.qa.canonical.com/testruns/testRun/3daa548d-79fb-4efe-84a1-7063397290a6
and bundle at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/bundle.yaml
All instances of this bug can be found at: |
|
| 2022-04-05 21:04:58 |
Alexander Balderson |
description |
On a deployment of Focal Ussuri which was CIS hardened SQA had two tempest tests which failed to resize a server, and then revert the resize.
the two tests which failed were:
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm
and
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
The nova compute logs show:
: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.648 653208 ERROR nova.virt.libvirt.driver [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.697 653208 INFO os_vif [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:5f:7c,bridge_name='br-int',has_traffic_filtering=True,id=c6c15dff-9201-49e9-9d86-4ce684138f53,network=Network(611f2961-05f5-4361-a30f-bcf384865f6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc6c15dff-92')
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Setting instance vm_state to ERROR: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Traceback (most recent call last):
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 10047, in _error_out_instance_on_exception
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] yield
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5904, in _finish_resize_helper
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] network_info = self._finish_resize(context, instance, migration,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5842, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self._set_instance_info(instance, old_flavor)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.force_reraise()
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] six.reraise(self.type_, self.value, self.tb)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] raise value
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5825, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.driver.finish_migration(context, migration, instance,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 10410, in finish_migration
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] guest = self._create_domain_and_network(context, xml, instance,
...
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a]
for both tests.
our CIS rule set is
RULESET1="1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 1.1.1.5 1.1.1.6 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.1.8 1.1.9 1.1.12 1.1.13 1.1.14 1.1.18 1.1.19 1.1.20 1.1.21 1.1.22 1.1.23 1.1.24 1.2.1 1.2.2 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 1.5.2 1.5.3 1.6.1 1.6.2 1.6.3 1.6.4 1.7.1.1 1.7.1.2 1.7.1.3 1.8.1.1 1.8.1.2 1.8.1.3 1.8.1.4 1.8.1.5 1.8.1.6 1.9 1.10"
RULESET2="2.1.1 2.1.2 2.2.1.1 2.2.1.2 2.2.1.3 2.2.1.4 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.2.8 2.2.9 2.2.10 2.2.11 2.2.12 2.2.13 2.2.14 2.2.15 2.2.17 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.4"
RULESET3="3.1.2 3.2.1 3.2.2 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 3.3.6 3.3.7 3.3.8 3.3.9 3.5.1.1 3.5.1.2 3.5.1.3 3.5.1.4 3.5.1.5 3.5.1.6 3.5.1.7 3.5.2.1 3.5.2.2 3.5.2.3 3.5.2.4 3.5.2.5 3.5.2.6 3.5.2.7 3.5.2.8 3.5.2.9 3.5.2.10 3.5.3.1.1 3.5.3.1.2 3.5.3.2.1 3.5.3.2.2 3.5.3.2.3 3.5.3.2.4 3.5.3.3.1 3.5.3.3.2 3.5.3.3.3 3.5.3.3.4"
RULESET4="4.2.1.1 4.2.1.2 4.2.1.3 4.2.1.4 4.2.1.5 4.2.1.6 4.2.2.1 4.2.2.2 4.2.2.3 4.2.3 4.3 4.4"
RULESET5="5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.1.7 5.1.8 5.1.9 5.2.1 5.2.2 5.2.3 5.2.4 5.2.6 5.2.7 5.2.8 5.2.9 5.2.10 5.2.11 5.2.12 5.2.13 5.2.14 5.2.15 5.2.16 5.2.17 5.2.18 5.2.19 5.2.21 5.2.22 5.3.1 5.3.2 5.3.3 5.3.4 5.4.1.1 5.4.1.2 5.4.1.3 5.4.1.4 5.4.1.5 5.4.2 5.4.3 5.4.4 5.4.5 5.5 5.6"
RULESET6="6.1.2 6.1.3 6.1.4 6.1.5 6.1.6 6.1.7 6.1.8 6.1.9 6.1.10 6.1.11 6.1.126.1.13 6.1.14 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.2.6 6.2.7 6.2.8 6.2.9 6.2.10 6.2.11 6.2.12 6.2.13 6.2.14 6.2.15 6.2.16 6.2.17"
metal systems get the additional rules:
"4.1.1.1 4.1.1.2 4.1.1.3 4.1.1.4 4.1.2.1 4.1.2.2 4.1.2.3 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.6 4.1.7 4.1.8 4.1.9 4.1.10 4.1.11 4.1.12 4.1.13 4.1.14 4.1.15 4.1.16 4.1.17
crashdump can be found at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/juju-crashdump-openstack-2022-04-03-03.39.08.tar.gz
with testrun at:
https://solutions.qa.canonical.com/testruns/testRun/3daa548d-79fb-4efe-84a1-7063397290a6
and bundle at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/bundle.yaml
All instances of this bug can be found at: |
On a deployment of Focal Ussuri which was CIS hardened SQA had two tempest tests which failed to resize a server, and then revert the resize.
the two tests which failed were:
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm
and
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
The nova compute logs show:
: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.648 653208 ERROR nova.virt.libvirt.driver [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Failed to start libvirt guest: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.697 653208 INFO os_vif [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:5f:7c,bridge_name='br-int',has_traffic_filtering=True,id=c6c15dff-9201-49e9-9d86-4ce684138f53,network=Network(611f2961-05f5-4361-a30f-bcf384865f6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc6c15dff-92')
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [req-b7c2648b-b61c-47b0-b965-015a39eb60a2 da22df534509496fba235127688ca2af c35da82188de4fba8f79f2d59119c4fa - f23c501bf80845fda352e6ca6e0e5bbe f23c501bf80845fda352e6ca6e0e5bbe] [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Setting instance vm_state to ERROR: libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] Traceback (most recent call last):
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 10047, in _error_out_instance_on_exception
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] yield
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5904, in _finish_resize_helper
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] network_info = self._finish_resize(context, instance, migration,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5842, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self._set_instance_info(instance, old_flavor)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.force_reraise()
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] six.reraise(self.type_, self.value, self.tb)
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] raise value
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5825, in _finish_resize
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] self.driver.finish_migration(context, migration, instance,
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 10410, in finish_migration
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] guest = self._create_domain_and_network(context, xml, instance,
...
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a] libvirt.libvirtError: Cannot access storage file '/var/lib/nova/instances/b3247fa2-fdef-4608-b661-0677fd68f96a/disk' (as uid:64055, gid:108): Permission denied
2022-04-03 03:18:09.700 653208 ERROR nova.compute.manager [instance: b3247fa2-fdef-4608-b661-0677fd68f96a]
for both tests.
our CIS rule set is
RULESET1="1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 1.1.1.5 1.1.1.6 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.1.8 1.1.9 1.1.12 1.1.13 1.1.14 1.1.18 1.1.19 1.1.20 1.1.21 1.1.22 1.1.23 1.1.24 1.2.1 1.2.2 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 1.5.2 1.5.3 1.6.1 1.6.2 1.6.3 1.6.4 1.7.1.1 1.7.1.2 1.7.1.3 1.8.1.1 1.8.1.2 1.8.1.3 1.8.1.4 1.8.1.5 1.8.1.6 1.9 1.10"
RULESET2="2.1.1 2.1.2 2.2.1.1 2.2.1.2 2.2.1.3 2.2.1.4 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.2.8 2.2.9 2.2.10 2.2.11 2.2.12 2.2.13 2.2.14 2.2.15 2.2.17 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.4"
RULESET3="3.1.2 3.2.1 3.2.2 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 3.3.6 3.3.7 3.3.8 3.3.9 3.5.1.1 3.5.1.2 3.5.1.3 3.5.1.4 3.5.1.5 3.5.1.6 3.5.1.7 3.5.2.1 3.5.2.2 3.5.2.3 3.5.2.4 3.5.2.5 3.5.2.6 3.5.2.7 3.5.2.8 3.5.2.9 3.5.2.10 3.5.3.1.1 3.5.3.1.2 3.5.3.2.1 3.5.3.2.2 3.5.3.2.3 3.5.3.2.4 3.5.3.3.1 3.5.3.3.2 3.5.3.3.3 3.5.3.3.4"
RULESET4="4.2.1.1 4.2.1.2 4.2.1.3 4.2.1.4 4.2.1.5 4.2.1.6 4.2.2.1 4.2.2.2 4.2.2.3 4.2.3 4.3 4.4"
RULESET5="5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.1.7 5.1.8 5.1.9 5.2.1 5.2.2 5.2.3 5.2.4 5.2.6 5.2.7 5.2.8 5.2.9 5.2.10 5.2.11 5.2.12 5.2.13 5.2.14 5.2.15 5.2.16 5.2.17 5.2.18 5.2.19 5.2.21 5.2.22 5.3.1 5.3.2 5.3.3 5.3.4 5.4.1.1 5.4.1.2 5.4.1.3 5.4.1.4 5.4.1.5 5.4.2 5.4.3 5.4.4 5.4.5 5.5 5.6"
RULESET6="6.1.2 6.1.3 6.1.4 6.1.5 6.1.6 6.1.7 6.1.8 6.1.9 6.1.10 6.1.11 6.1.126.1.13 6.1.14 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.2.6 6.2.7 6.2.8 6.2.9 6.2.10 6.2.11 6.2.12 6.2.13 6.2.14 6.2.15 6.2.16 6.2.17"
metal systems get the additional rules:
"4.1.1.1 4.1.1.2 4.1.1.3 4.1.1.4 4.1.2.1 4.1.2.2 4.1.2.3 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.6 4.1.7 4.1.8 4.1.9 4.1.10 4.1.11 4.1.12 4.1.13 4.1.14 4.1.15 4.1.16 4.1.17
crashdump can be found at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/juju-crashdump-openstack-2022-04-03-03.39.08.tar.gz
with testrun at:
https://solutions.qa.canonical.com/testruns/testRun/3daa548d-79fb-4efe-84a1-7063397290a6
and bundle at:
https://oil-jenkins.canonical.com/artifacts/3daa548d-79fb-4efe-84a1-7063397290a6/generated/generated/openstack/bundle.yaml
All instances of this bug can be found at:
https://solutions.qa.canonical.com/bugs/bugs/bug/1967956 |
|
| 2022-07-11 14:04:48 |
Bas de Bruijne |
tags |
cdo-qa foundations-engine |
cdo-qa cdo-tempest foundations-engine |
|
| 2022-08-26 22:28:24 |
Billy Olsen |
charm-nova-compute: status |
New |
Confirmed |
|
| 2022-08-30 10:58:53 |
Marcelo Subtil Marcal |
bug |
|
|
added subscriber Canonical Field Critical |
| 2022-08-30 19:57:28 |
Billy Olsen |
charm-nova-compute: assignee |
|
Corey Bryant (corey.bryant) |
|
| 2022-09-08 15:25:25 |
Corey Bryant |
charm-nova-compute: status |
Confirmed |
Triaged |
|
| 2022-09-08 15:25:32 |
Corey Bryant |
charm-nova-compute: importance |
Undecided |
Medium |
|
| 2022-09-14 13:15:32 |
Corey Bryant |
bug task added |
|
nova (Ubuntu) |
|
| 2022-09-14 13:15:38 |
Corey Bryant |
nova (Ubuntu): status |
New |
Triaged |
|
| 2022-09-14 13:15:40 |
Corey Bryant |
nova (Ubuntu): importance |
Undecided |
Medium |
|
| 2022-09-14 13:15:42 |
Corey Bryant |
nova (Ubuntu): assignee |
|
Corey Bryant (corey.bryant) |
|
| 2022-09-14 13:15:44 |
Corey Bryant |
charm-nova-compute: assignee |
Corey Bryant (corey.bryant) |
|
|
| 2022-09-14 13:15:46 |
Corey Bryant |
charm-nova-compute: importance |
Medium |
Undecided |
|
| 2022-09-14 13:15:50 |
Corey Bryant |
charm-nova-compute: status |
Triaged |
Invalid |
|
| 2022-09-14 13:17:19 |
Corey Bryant |
nova (Ubuntu): status |
Triaged |
Fix Released |
|
| 2022-09-22 10:10:05 |
Yoshi Kadokawa |
bug |
|
|
added subscriber Yoshi Kadokawa |
| 2023-10-10 19:29:51 |
Jeff Hillman |
bug |
|
|
added subscriber Jeff Hillman |
| 2023-10-12 18:34:58 |
Felipe Reyes |
nominated for series |
|
Ubuntu Jammy |
|
| 2023-10-12 18:34:58 |
Felipe Reyes |
bug task added |
|
nova (Ubuntu Jammy) |
|
| 2023-10-12 18:35:21 |
Felipe Reyes |
bug task added |
|
cloud-archive |
|
| 2023-10-12 18:35:30 |
Felipe Reyes |
nominated for series |
|
cloud-archive/yoga |
|
| 2023-10-12 18:35:30 |
Felipe Reyes |
bug task added |
|
cloud-archive/yoga |
|
| 2023-10-12 18:35:38 |
Felipe Reyes |
cloud-archive: status |
New |
Invalid |
|
| 2023-10-12 18:47:54 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~freyes/ubuntu/+source/nova/+git/nova/+merge/453472 |
|
| 2024-03-22 14:38:45 |
Launchpad Janitor |
nova (Ubuntu Jammy): status |
New |
Confirmed |
|
