Multiple SVMs are spawned after 'FlavorDiskTooSmall' is seen.

[Ganesha HV]

Topo
====
ci-sanity setup
nested VM - 10.84.34.96
webui - http://10.84.34.96:8080
horizon - http://10.84.34.96/horizon

1]. Tried to create a analyser VM with flavor "contrail_flavor_small | 1GB RAM | 1 VCPU | 10.0GB Disk"
2]. The following error is seen and the SVM moves to ERROR state:
2015-05-22 04:04:09.852 7027 INFO nova.scheduler.filter_scheduler [req-c6a0cad2-6fb4-42b0-946b-8ca8b1551ea5 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Attempting to build 1 instance(s) uuids: [u'f5f2acdd-3317-422f-aa56-f728f7961208']
2015-05-22 04:04:09.855 7027 ERROR nova.scheduler.filter_scheduler [req-c6a0cad2-6fb4-42b0-946b-8ca8b1551ea5 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] [instance: f5f2acdd-3317-422f-aa56-f728f7961208] Error from last host: ci-oc-subslave-ubuntu-10-84-34-158-1.localdomain.com (node ci-oc-subslave-ubuntu-10-84-34-158-1.localdomain.com): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1331, in _build_instance\n set_access_ip=set_access_ip)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 396, in decorated_function\n return function(self, context, *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1743, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1740, in _spawn\n block_device_info)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2290, in spawn\n admin_pass=admin_password)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2675, in _create_image\n project_id=instance[\'project_id\'])\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 213, in cache\n *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 406, in create_image\n self.verify_base_size(base, size)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 264, in verify_base_size\n raise exception.FlavorDiskTooSmall()\n', u"FlavorDiskTooSmall: Flavor's disk is too small for requested image.\n"]
2015-05-22 04:04:09.864 7027 WARNING nova.scheduler.host_manager [req-c6a0cad2-6fb4-42b0-946b-8ca8b1551ea5 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Host has more disk space than database expected (50gb > -4gb)
2015-05-22 04:04:09.865 7027 INFO nova.filters [req-c6a0cad2-6fb4-42b0-946b-8ca8b1551ea5 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Filter RetryFilter returned 0 hosts
2015-05-22 04:04:09.866 7027 WARNING nova.scheduler.driver [req-c6a0cad2-6fb4-42b0-946b-8ca8b1551ea5 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] [instance: f5f2acdd-3317-422f-aa56-f728f7961208] Setting instance to ERROR state.
2015-05-22 04:04:38.761 7027 INFO nova.scheduler.filter_scheduler [req-db07c715-5e5a-43e2-81ba-0c1cc6f3150e 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Attempting to build 1 instance(s) uuids: [u'1fb0f4b4-e80f-4448-8319-53c06b6f2364']
2015-05-22 04:04:38.774 7027 WARNING nova.scheduler.host_manager [req-db07c715-5e5a-43e2-81ba-0c1cc6f3150e 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Host has more disk space than database expected (50gb > -24gb)
2015-05-22 04:04:38.778 7027 INFO nova.filters [req-db07c715-5e5a-43e2-81ba-0c1cc6f3150e 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] Filter RamFilter returned 0 hosts
2015-05-22 04:04:38.779 7027 WARNING nova.scheduler.driver [req-db07c715-5e5a-43e2-81ba-0c1cc6f3150e 95b12e3882ad4c4abfdae3ab927d985c 4f93580340a74ef9a68cf001163e2e52] [instance: 1fb0f4b4-e80f-4448-8319-53c06b6f2364] Setting instance to ERROR state.

3]. But the instance isn't deleted and a new one is respawned, leading to multiple SVMs in ERROR state:

<email address hidden>:/var/log/nova# nova list --all_tenants
+--------------------------------------+-----------------------------------------------------------------+--------+------------+-------------+-------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-----------------------------------------------------------------+--------+------------+-------------+-------------------------------------+
| 173ad77e-1355-4cc1-9a9c-e901fea04389 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| 37bf1df1-4d04-4b00-aca7-79b327e7b549 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| 83c73fb4-a599-41c0-8e3f-232fa305ce4e | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| 9b078baf-6c50-4e70-82a7-05cbed04ac1a | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| ae5fa61e-c0aa-4ee9-92c5-e593940e2f48 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| b4d6db6d-dbdf-4660-97c9-f3cf9f1477fb | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| d6f15579-4a1b-44bd-8eab-2d3190597df1 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| f12348d2-9489-4f47-a7bd-0bf62e1dd332 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| f213d911-0fde-4179-8957-3d97c3acb65f | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| f5f2acdd-3317-422f-aa56-f728f7961208 | default-domain__TestSVCMirror-83103711__mirror_si-93761505_1__1 | ERROR | - | NOSTATE | |
| 4a59e790-78b3-47e5-8ef7-d66a823abe27 | in_network_vm1-14429330 | ACTIVE | - | Running | in_network_vn1-67701694=43.217.56.3 |
| 5e2c5af1-94d6-4ec7-af28-a38bb18ab93e | in_network_vm2-46070160 | ACTIVE | - | Running | in_network_vn2-03533533=174.53.53.3 |
+--------------------------------------+-----------------------------------------------------------------+--------+------------+-------------+-------------------------------------+
<email address hidden>:/var/log/nova#

The setup is intact.

Debugged with Prakash.

[Ganesha HV (ganeshahv)]

In 3.0.2.0-47, I see the following error logs in the svc-monitor.log:

8.200.25:5998, ] status = Up description = Subscribe Response >>, ] description = Collector connection down >>, ] >>
06/03/2016 03:03:18 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-c6caa554-764d-4930-843e-82b00dd8749a)
06/03/2016 03:03:18 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001
06/03/2016 03:03:20 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-b77110df-ca05-4463-8b0a-89b9dff50101)
06/03/2016 03:03:20 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001
06/03/2016 03:03:21 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-04f67a5f-b624-47d8-bab9-55ff77f353c7)
06/03/2016 03:03:21 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001
06/03/2016 03:04:22 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-1a964e74-34a2-4e2b-87c5-01e2fdbac667)
06/03/2016 03:04:22 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001
06/03/2016 03:05:24 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-64363d71-d24c-4209-8f6b-c292f5289db3)
06/03/2016 03:05:24 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001
06/03/2016 03:06:24 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: nova error Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 3593011200 bytes. (HTTP 400) (Request-ID: req-ae4c7317-36b8-4473-ab3d-51947d9a6f66)
06/03/2016 03:06:24 AM [contrail-svc-monitor]: __default__ [SYS_ERR]: SvcMonitorLog: Nova vm create failed analyser001

No nova instances spawned, though:

root@nodeg25:/var/log/contrail# nova list --all_tenants
+----+------+-----------+--------+------------+-------------+----------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+----+------+-----------+--------+------------+-------------+----------+
+----+------+-----------+--------+------------+-------------+----------+

[Rudra Rugge (rrugge)]

V2 service chain does not have this issue. Hence won't fix this bug.