Bulk delete of docker VM gets stuck
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Juniper Openstack | Status tracked in Trunk | |||||
R2.20 |
Fix Committed
|
High
|
Ignatious Johnson Christopher | |||
Trunk |
Fix Committed
|
High
|
Ignatious Johnson Christopher |
Bug Description
Ona cluster with Docker as the hyperviosor, on trying to delete a bunch of Instances which are in Error state, the operation doesnot complete , see the below trace
root@cmbu-
Request to delete server 4e930ba8-
2015-05-14 00:07:29.620 2645 AUDIT nova.compute.
2015-05-14 00:07:34.811 2645 AUDIT nova.compute.
2015-05-14 00:07:35.468 2645 AUDIT nova.compute.
2015-05-14 00:07:35.468 2645 AUDIT nova.compute.
2015-05-14 00:07:35.469 2645 AUDIT nova.compute.
2015-05-14 00:07:35.469 2645 AUDIT nova.compute.
2015-05-14 00:07:35.495 2645 INFO nova.scheduler.
2015-05-14 00:07:35.496 2645 INFO nova.compute.
2015-05-14 00:07:39.711 2645 ERROR nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.711 2645 TRACE nova.compute.
2015-05-14 00:07:39.810 2645 INFO nova.scheduler.
2015-05-14 00:07:39.942 2645 INFO nova.scheduler.
2015-05-14 00:07:39.944 2645 ERROR oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
2015-05-14 00:07:39.944 2645 TRACE oslo.messaging.
nova show 4e930ba8-
+------
| Property | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | error |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2015-05-
| fault | {"message": "UnixHTTPConnec
| | return function(self, context, *args, **kwargs) |
| | File \"/usr/
| | do_terminate_
| | File \"/usr/
| | return f(*args, **kwargs) |
| | File \"/usr/
| | self._set_
| | File \"/usr/
| | six.reraise(
| | File \"/usr/
| | self._delete_
| | File \"/usr/
| | rv = f(*args, **kwargs) |
| | File \"/usr/
| | quotas.rollback() |
| | File \"/usr/
| | six.reraise(
| | File \"/usr/
| | self._shutdown_
| | File \"/usr/
| | requested_networks) |
| | File \"/usr/
| | six.reraise(
| | File \"/usr/
| | block_device_info) |
| | File \"/usr/
| | self.soft_
| | File \"/usr/
| | container_id = self._get_
| | File \"/usr/
| | return self._find_
| | File \"/usr/
| | for info in self.list_
| | File \"/usr/
| | info = self.docker.
| | File \"/usr/
| | out = f(*args, **kwds) |
| | File \"/usr/
| | self._get(
| | File \"/usr/
| | return self.get(url, **self.
| | File \"/usr/
| | out = f(*args, **kwds) |
| | File \"/usr/
| | return self.request('GET', url, **kwargs) |
| | File \"/usr/
| | out = f(*args, **kwds) |
| | File \"/usr/
| | resp = self.send(prep, **send_kwargs) |
| | File \"/usr/
| | out = f(*args, **kwds) |
| | File \"/usr/
| | r = adapter.
| | File \"/usr/
| | raise Timeout(e) |
| | ", "created": "2015-05-
| flavor | m1.tiny (1) |
| hostId | 56fe2e4f0df4895
| id | 4e930ba8-
| image | phusion-
| key_name | - |
| metadata | {} |
| name | VM6 |
| os-extended-
| status | ERROR |
| tenant_id | 9a72e9d550f3481
| updated | 2015-05-
| user_id | fc7848f3307144e
+------
information type: | Proprietary → Public |
Reviewed: https:/ /review. opencontrail. org/10731 github. org/Juniper/ contrail- packaging/ commit/ 22ac9793701d084 4dea10bbcffdd3e abe4a2f821
Committed: http://
Submitter: Zuul
Branch: R2.20
commit 22ac9793701d084 4dea10bbcffdd3e abe4a2f821
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri May 22 16:33:26 2015 -0700
Removing veth interfaces if the contrainer launch failed
Closes-Bug: 1452151
Closes-Bug: 1454976
Using inet6 argument with ip command to configure ipv6 address to container
Closes-Bug: 1455616
Configure network using statically instead of using dhclient to avoid
it writting to the compute hosts resolv.conf
Closes-Bug: 1455168
Periodically check the vrouter agent conncetion and publish the port informaton to
the vrouter agent.
Closes-Bug: 1454655
Change-Id: I3f59055da0748d f2a8dddfd82cf2e b062377dfab