Cannot rebuild a VM created from a Cinder volume backed by NetApp
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack-Ansible |
Fix Released
|
Medium
|
David | ||
Juno |
Fix Released
|
Medium
|
Jesse Pretorius | ||
Kilo |
Fix Released
|
Medium
|
Jesse Pretorius | ||
Trunk |
Fix Released
|
Medium
|
David |
Bug Description
After successfully creating a VM from a Cinder volume backed by NetApp, issuing a rebuild fails. The error returned is: `Failed to terminate process 9233 with SIGKILL: Device or resource busy`. This is occurring within SAT6-LAB02.
Looking through the Compute error logs, it looks like the DOM kill on the VM returns an error code and throws the server's status into error. As the error log also states, I'm unable to view the process which fails because it can't be found on the Compute node. Before the rebuild's issued, you can see the VM as active according to the Nova CLI.
```
root@node89_
+------
| Property | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | building |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| adminPass | 3KedpFDJ3453 |
| config_drive | |
| created | 2014-12-
| flavor | m1.small (2) |
| hostId | |
| id | 676b0cf4-
| image | Attempt to boot from volume - no image supplied |
| key_name | - |
| metadata | {} |
| name | test |
| os-extended-
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | a79197cfca65452
| updated | 2014-12-
| user_id | 2912a11dd3b6481
+------
Server building... 100% complete
Finished
---
root@node89_
+------
| ID | Name | Status | Server |
+------
| b9e8c879-
| 5f4aac88-
| c2f864f8-
| 1920c7f2-
| b7dd98cc-
+------
---
root@node89_
+------
| Property | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | LBqu8H94d7TP |
| created | 2014-12-
| flavor | m1.small (2) |
| hostId | fa32297f7a7cfbe
| id | 676b0cf4-
| image | cirros-0.3.3 (c2f864f8-
| metadata | {} |
| name | test |
| private network | 172.31.0.126 |
| progress | 0 |
| status | REBUILD |
| tenant_id | a79197cfca65452
| updated | 2014-12-
| user_id | 2912a11dd3b6481
+------
Server rebuilding... 0% complete
Error rebuilding server
ERROR (InstanceInErro
---
# /var/log/
2014-12-09 20:03:03.613 3235 AUDIT nova.compute.
2014-12-09 20:03:19.369 3235 ERROR nova.virt.
r=Failed to terminate process 22831 with SIGKILL: Device or resource busy
2014-12-09 20:03:19.370 3235 ERROR nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
nstance_
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
ance
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
ault_impl
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
it__
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.370 3235 TRACE nova.compute.
2014-12-09 20:03:19.482 3235 INFO nova.scheduler.
2014-12-09 20:03:19.639 3235 INFO nova.scheduler.
2014-12-09 20:03:19.642 3235 ERROR oslo.messaging.
```
CVE References
Changed in openstack-ansible: | |
status: | New → Triaged |
importance: | Undecided → Critical |
importance: | Critical → Medium |
milestone: | none → next |
tags: | added: upstream |
tags: |
added: upstream-bug removed: upstream |
Changed in openstack-ansible: | |
status: | Triaged → Confirmed |
Changed in openstack-ansible: | |
milestone: | next → 11.0.4 |
milestone: | 11.0.4 → none |
I was able to replicate this on cloud servers (using standard LVM backend). I'm still digging into it, but this seems to be the same issue:
https:/ /bugs.launchpad .net/nova/ +bug/1353939