nova list still shows error'ed network after interface-delete of an error'ed interface

[Jay Buffington]

I am using the latest quantum and nova code from github. I created a VM with three interfaces. One interface failed to be created. When I delete the interface that had a port state of "error" (50350ce9...) with `nova interface-delete` `nova list` still shows it under "Networks"

The interface that I deleted that had state active (64f1911e...) did not show up in nova list after I deleted it.

I expected nova list to only show one interface under the "Networks" section after I deleted two of the three.

(The below is probably unreadable, so I also pasted at http://paste.openstack.org/show/39491/)

$ nova interface-list jaybuff-test
+------------+--------------------------------------+--------------------------------------+---------------+-------------+
| Port State | Port ID | Net ID | IP addresses | MAC Address |
+------------+--------------------------------------+--------------------------------------+---------------+-------------+
| ACTIVE | 64f1911e-4843-47f0-bb5a-5855ea7ac69d | d5178745-1d6f-4308-8a7b-c6ea5c72c7df | 10.1.1.3 | |
| ACTIVE | cd82324e-825f-43da-8e48-c46ede23281b | 04d07b77-96fe-4083-a628-08888389d8a1 | 10.211.250.3 | |
| ERROR | 50350ce9-2da1-4100-8d57-b85538492fdb | 1337c239-dd8d-4fbc-915f-46f26a98121b | 1.2.3.4 | |
+------------+--------------------------------------+--------------------------------------+---------------+-------------+

$ nova interface-detach jaybuff-test cd82324e-825f-43da-8e48-c46ede23281b

$ nova interface-detach jaybuff-test 50350ce9-2da1-4100-8d57-b85538492fdb

$ nova interface-list jaybuff-test
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| Port State | Port ID | Net ID | IP addresses | MAC Address |
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| ACTIVE | 64f1911e-4843-47f0-bb5a-5855ea7ac69d | d5178745-1d6f-4308-8a7b-c6ea5c72c7df | 10.1.1.3 | |
+------------+--------------------------------------+--------------------------------------+--------------+-------------+

$ nova list
+--------------------------------------+--------------+--------+----------------------------------------------------------------+
| ID | Name | Status | Networks |
+--------------------------------------+--------------+--------+----------------------------------------------------------------+
| 199cafd5-b20f-4c9f-9fc6-937315229dde | jaybuff-test | ACTIVE | public-network=1.2.3.4; tempest-private-network=10.1.1.3 |
+--------------------------------------+--------------+--------+----------------------------------------------------------------+

[Jay Buffington (jaybuff)]

It looks like this isn't related to the fact that the port was in error state. I found this in the log on the compute node any time I run interface-{attach,delete}:

2013-07-03 00:12:23,360 (nova.network.api): ERROR api update_instance_cache_with_nw_info Failed storing info cache
Traceback (most recent call last):
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/nova/network/api.py", line 84, in update_instance_cache_with_nw_info
    cache)
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/nova/db/api.py", line 818, in instance_info_cache_update
    return IMPL.instance_info_cache_update(context, instance_uuid, values)
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/nova/cmd/compute.py", line 50, in __call__
    raise exception.DBNotAllowed('nova-compute')
DBNotAllowed: nova-compute

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/37700

[Cyril Roelandt (cyril-roelandt)]

I think this patch does not completely solve this bug. I managed to get a
reproducible test case (after applying https://review.openstack.org/#/c/37030/
and https://review.openstack.org/#/c/37700/):

$ nova image-list
+--------------------------------------+---------------------------------+--------+--------+
| ID | Name | Status | Server |
+--------------------------------------+---------------------------------+--------+--------+
| 71f62aba-0b7c-4f2f-a557-d6145d2ee151 | cirros-0.3.1-x86_64-uec | ACTIVE | |
| e52f691c-05d3-429a-89c0-52deb0c013fd | cirros-0.3.1-x86_64-uec-kernel | ACTIVE | |
| ba2fe70e-553c-4d28-9aa8-c5e44f14f9cf | cirros-0.3.1-x86_64-uec-ramdisk | ACTIVE | |
+--------------------------------------+---------------------------------+--------+--------+

$ nova boot --flavor m1.nano --image 71f62aba-0b7c-4f2f-a557-d6145d2ee151 nanotest
...

$ nova list --fields name,networks
+--------------------------------------+----------+---------------------+
| ID | Name | Networks |
+--------------------------------------+----------+---------------------+
| d8d39a88-0c72-46cf-a361-b9890cfa81fc | nanotest | public=172.24.4.227 |
+--------------------------------------+----------+---------------------+

$ nova interface-list nanotest
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| Port State | Port ID | Net ID | IP addresses | MAC Address |
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| ACTIVE | 33e680fd-57c5-4cc2-9357-7c0591d82312 | a31050d7-2b87-41d1-abd8-c018ade1242c | 172.24.4.227 | |
+------------+--------------------------------------+--------------------------------------+--------------+-------------+

$ nova interface-attach --net-id a31050d7-2b87-41d1-abd8-c018ade1242c nanotest

$ nova list
+--------------------------------------+----------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+---------------------+
| d8d39a88-0c72-46cf-a361-b9890cfa81fc | nanotest | ACTIVE | None | Running | public=172.24.4.228 |
+--------------------------------------+----------+--------+------------+-------------+---------------------+

$ nova interface-list nanotest
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| Port State | Port ID | Net ID | IP addresses | MAC Address |
+------------+--------------------------------------+--------------------------------------+--------------+-------------+
| ACTIVE | 33e680fd-57c5-4cc2-9357-7c0591d82312 | a31050d7-2b87-41d1-abd8-c018ade1242c | 172.24.4.227 | |
| ACTIVE | 496604...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/37700
Committed: http://github.com/openstack/nova/commit/3379e95515dbc4f0574a9c030127ba19ba07f0e2
Submitter: Jenkins
Branch: master

commit 3379e95515dbc4f0574a9c030127ba19ba07f0e2
Author: Cyril Roelandt <email address hidden>
Date: Thu Jul 18 14:16:18 2013 +0000

    Fix DB access when refreshing the network cache.

    When calling ComputeManager.attach_interface(), the following happens:

    1) self.network_api.allocate_port_for_instance() is called from within
       ComputeManager.attach_interface();
    2) refresh_cache() is called, since it decorates allocate_port_for_instance();
    3) refresh_cache calls update_instance_cache_with_nw_info(), with
       "conductor_api=None" since 'conductor_api' cannot be found in the kwargs;
    4) update_instance_cache_with_nw_info() calls
       api.db.instance_info_cache_update();
    5) Since the call to the database should be made through the conductor, a
       DBNotAllowed exception is raised, which prevents the cache from being updated.

    This patch fixes the call to allocate_port_for_instance (step 1) so that
    'conductor_api' can be found in the kwargs (step 3).

    When using ComputeManager.detach_interface, the exact same problem arises (with
    deallocate_port_for_instance instead of allocate_port_for_instance).

    This is a first step towards fixing bug #1197192.

    Change-Id: I647795907b22790759d3be74afcaf70b712cb338