vm resize failed due to the remains left by failed actions

Bug #1836141 reported by Spencer Yu
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Expired
Undecided
Unassigned

Bug Description

Reproduce Steps:
1. Confirm resize failed due to neutron error:
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 382, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 370, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3442, in confirm_resize |
| | File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 254, in inner |
| | return f(*args, **kwargs) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3440, in do_confirm_resize |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3465, in _confirm_resize |
| | File "/usr/lib/python2.7/site-packages/nova/network/base_api.py", line 244, in get_instance_nw_info |
| | File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 949, in _get_instance_nw_info |
| | File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1724, in _build_network_info_model |
| | File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 972, in _gather_port_ids_and_networks |
| | File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 290, in _get_available_networks |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 577, in list_networks |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 307, in list |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 320, in _pagination |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 293, in get |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 270, in retry_request |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 211, in do_request |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 185, in _handle_fault_response |
| | File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 83, in exception_handler_v20 |
| | '}

2. Resize failed due to Step 1 failed action:
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 315, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 292, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 382, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 370, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3986, in finish_resize |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3974, in finish_resize |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3940, in _finish_resize |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3935, in _finish_resize |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7012, in finish_migration |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2972, in _create_image |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 872, in create_snap |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 365, in create_snap |
| | File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute |
| | six.reraise(c, e, tb) |
| | File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker |
| | rv = meth(*args, **kwargs) |
| | File "/usr/lib/python2.7/site-packages/rbd.py", line 594, in create_snap |
| | raise make_ex(ret, \'error creating snapshot %s from %s\' % (name, self.name)) |
| | '}

3. Resize failed due to before failed actions:
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 315, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 292, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 382, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 370, in decorated_function |
| | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3841, in resize_instance |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6912, in migrate_disk_and_power_off |
| | File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6862, in migrate_disk_and_power_off |
| | File "/usr/lib/python2.7/site-packages/nova/utils.py", line 390, in execute |
| | File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 275, in execute |
| | cmd=sanitized_cmd) |
| | '}

4. Remove before steps remains: nova_resize snapshot and instance resize directory, resize success.

Environment:
rbd backends

Revision history for this message
Artom Lifshitz (notartom) wrote :

I *think* you're saying that if our call to Neutron fails during confirm_resize, Nova doesn't handle it correctly, and any subsequent attempts to resize the instance fail as well. This makes sense, and is totally the kind of thing known to be wrong in that code, but looks like the tracebacks your posted got cut off, and so are missing the important bits at the end.

If I did indeed understand you correctly, please just attach new tracebacks to the bug. Otherwise, if I misunderstood your bug report, please try to clarify the steps you took, the behaviour that you expected, and the actual result that you got.

Thanks!

Changed in nova:
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]

Changed in nova:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.