floating ips do not display in 'nova list' after association to instance

Bug #939122 reported by Anthony Young on 2012-02-23
48
This bug affects 9 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
Vish Ishaya
Essex
Undecided
Vish Ishaya
nova (Ubuntu)
Medium
Unassigned
Precise
Undecided
Unassigned
Quantal
Medium
Unassigned

Bug Description

Immediately after associating a floating ip, 'nova list' does not display the floating ip, apparently due to a cache invalidation issue.

Steps to reproduce:

> nova boot --flavor 1 --image 14701668-cd5e-4f3e-9784-6e0c093000d8 --security_groups=test_secgroup --key_name test_key test_instance
 (wait for boot, assume instance_id=109ee800-ed59-4612-951a-b0027d1cdad6)
> nova floating-ip-create
 (assume FLOATING_IP=172.24.4.225)
> nova add-floating-ip 109ee800-ed59-4612-951a-b0027d1cdad6 172.24.4.225
> nova list

Expected:

Display floating ip

Actual:

$ nova list
+--------------------------------------+---------------+--------+------------------+
| ID | Name | Status | Networks |
+--------------------------------------+---------------+--------+------------------+
| 109ee800-ed59-4612-951a-b0027d1cdad6 | test_instance | ACTIVE | private=10.0.0.2 |
+--------------------------------------+---------------+--------+------------------+

After a few minutes, the output is correct.

Related branches

comstud notes that he does not see 'updating of the cache in associate_floating_ip or _associate_floating_ip in network manager."

Thierry Carrez (ttx) on 2012-02-24
Changed in nova:
importance: Undecided → Medium
status: New → Confirmed
Thierry Carrez (ttx) wrote :

From dupe:

The instance_info_cache needs to be updated after ip assignment. What's happening is the ip is assigned, but since all of compute/api use the instance_info_cache to get at network data, it won't show up until a get_instance_nw_info is called by compute (usually during the cache heal task).

Changed in nova:
assignee: nobody → D LALITHA RANI (deevi-rani)
D LALITHA RANI (deevi-rani) wrote :

I was able to reproduce the bug and i found that the "instance_info_caches" table is not getting updated upon associating a floating ip. I thought that there is a need to update the table in compute.api.associate_floating_ip() method.

Following is the change that fixed the bug:
    def associate_floating_ip(self, context, instance, address):
        ...
        net_info = self.network_api.get_instance_nw_info(context, instance)
        LOG.debug(_("floating-ip associated with following entities %s"), net_info)
        return
    raise exception.ApiError(fail_bag)

This successfully updated entries in the database and nova list is able to reflect it.

Changed in nova:
status: Confirmed → In Progress
Changed in nova:
status: In Progress → Fix Committed
David Kranz (david-kranz) wrote :

Since this is a regression from diablo, and will result in future bug reports, perhaps it should be marked for stable essex.

Vish Ishaya (vishvananda) wrote :

Fix committed is for when the patch has merged into trunk and it will be automatically changed by gerrit.

Changed in nova:
status: Fix Committed → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/6584

Fix proposed to branch: master
Review: https://review.openstack.org/7059

This problem also exists with auto_assign_floating_ip and is not fixed by the current patch, I had to replicate the proposed solution in network/manager.py in addition to the current fix.

thank you

Scott Moser (smoser) on 2012-05-24
Changed in nova (Ubuntu):
importance: Undecided → Medium
status: New → Confirmed

Fix proposed to branch: master
Review: https://review.openstack.org/8571

Changed in nova:
assignee: D LALITHA RANI (deevi-rani) → Vish Ishaya (vishvananda)

Reviewed: https://review.openstack.org/8571
Committed: http://github.com/openstack/nova/commit/7097e01a15ecd5c28b6b771e5043f18da243607e
Submitter: Jenkins
Branch: master

commit 7097e01a15ecd5c28b6b771e5043f18da243607e
Author: deevi rani <email address hidden>
Date: Thu May 3 06:42:47 2012 -0400

    Updates the cache

    fixes bug 939122

    Following are the changes done:
    1. in compute/api.py to display the
       floating ip in nova list command upon
       its association with an instance.
    2. in network/api.py to solve the similar
       issue upon the floating ip disassocaition
    Updated version of: https://review.openstack.org/#/c/7059

    Change-Id: Ief1a3f544c3d14bd61cc0814612df3ee521ddec2

Changed in nova:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/8581
Committed: http://github.com/openstack/nova/commit/caae0e9ca3abaad1d17d995f66195ea62fe2e59a
Submitter: Jenkins
Branch: stable/essex

commit caae0e9ca3abaad1d17d995f66195ea62fe2e59a
Author: deevi rani <email address hidden>
Date: Thu May 3 06:42:47 2012 -0400

    Updates the cache

    fixes bug 939122

    Following are the changes done:
    1. in compute/api.py to display the
       floating ip in nova list command upon
       its association with an instance.
    2. in network/api.py to solve the similar
       issue upon the floating ip disassocaition
    Updated version of: https://review.openstack.org/#/c/7059

    Change-Id: Ief1a3f544c3d14bd61cc0814612df3ee521ddec2
    (cherry picked from commit 7097e01a15ecd5c28b6b771e5043f18da243607e)

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in nova (Ubuntu Precise):
status: New → Confirmed
Thierry Carrez (ttx) on 2012-07-04
Changed in nova:
milestone: none → folsom-2
status: Fix Committed → Fix Released
Chuck Short (zulcss) on 2012-08-21
Changed in nova (Ubuntu Quantal):
status: Confirmed → Fix Released

Please find the attached test log from the Ubuntu Server Team's CI infrastructure. As part of the verification process for this bug, Nova has been deployed and configured across multiple nodes using precise-proposed as an installation source. After successful bring-up and configuration of the cluster, a number of exercises and smoke tests have be invoked to ensure the updated package did not introduce any regressions. A number of test iterations were carried out to catch any possible transient errors.

Please Note the list of installed packages at the top and bottom of the report.

For records of upstream test coverage of this update, please see the Jenkins links in the comments of the relevant upstream code-review(s):

Trunk review: https://review.openstack.org/8571
Stable review: https://review.openstack.org/8581

As per the provisional Micro Release Exception granted to this package by the Technical Board, we hope this contributes toward verification of this update.

Adam Gandelman (gandelman-a) wrote :

Test coverage log.

tags: added: verification-done
Launchpad Janitor (janitor) wrote :
Download full text (5.4 KiB)

This bug was fixed in the package nova - 2012.1.3+stable-20120827-4d2a4afe-0ubuntu1

---------------
nova (2012.1.3+stable-20120827-4d2a4afe-0ubuntu1) precise-proposed; urgency=low

  * New upstream snapshot, fixes FTBFS in -proposed. (LP: #1041120)
  * Resynchronize with stable/essex (4d2a4afe):
    - [5d63601] Inappropriate exception handling on kvm live/block migration
      (LP: #917615)
    - [ae280ca] Deleted floating ips can cause instance delete to fail
      (LP: #1038266)

nova (2012.1.3+stable-20120824-86fb7362-0ubuntu1) precise-proposed; urgency=low

  * New upstream snapshot. (LP: #1041120)
  * Dropped, superseded by new snapshot:
    - debian/patches/CVE-2012-3447.patch: [d9577ce]
    - debian/patches/CVE-2012-3371.patch: [25f5bd3]
    - debian/patches/CVE-2012-3360+3361.patch: [b0feaff]
  * Resynchronize with stable/essex (86fb7362):
    - [86fb736] Libvirt driver reports incorrect error when volume-detach fails
      (LP: #1029463)
    - [272b98d] nova delete lxc-instance umounts the wrong rootfs (LP: #971621)
    - [09217ab] Block storage connections are NOT restored on system reboot
      (LP: #1036902)
    - [d9577ce] CVE-2012-3361 not fully addressed (LP: #1031311)
    - [e8ef050] pycrypto is unused and the existing code is potentially insecure
      to use (LP: #1033178)
    - [3b4ac31] cannot umount guestfs (LP: #1013689)
    - [f8255f3] qpid_heartbeat setting in ineffective (LP: #1030430)
    - [413c641] Deallocation of fixed IP occurs before security group refresh
      leading to potential security issue in error / race conditions
      (LP: #1021352)
    - [219c5ca] Race condition in network/deallocate_for_instance() leads to
      security issue (LP: #1021340)
    - [f2bc403] cleanup_file_locks does not remove stale sentinel files
      (LP: #1018586)
    - [4c7d671] Deleting Flavor currently in use by instance creates error
      (LP: #994935)
    - [7e88e39] nova testsuite errors on newer versions of python-boto (e.g.
      2.5.2) (LP: #1027984)
    - [80d3026] NoMoreFloatingIps: Zero floating ips available after repeatedly
      creating and destroying instances over time (LP: #1017418)
    - [4d74631] Launching with source groups under load produces lazy load error
      (LP: #1018721)
    - [08e5128] API 'v1.1/{tenant_id}/os-hosts' does not return a list of hosts
      (LP: #1014925)
    - [801b94a] Restarting nova-compute removes ip packet filters (LP: #1027105)
    - [f6d1f55] instance live migration should create virtual_size disk image
      (LP: #977007)
    - [4b89b4f] [nova][volumes] Exceeding volumes, gigabytes and floating_ips
      quotas returns general uninformative HTTP 500 error (LP: #1021373)
    - [6e873bc] [nova][volumes] Exceeding volumes, gigabytes and floating_ips
      quotas returns general uninformative HTTP 500 error (LP: #1021373)
    - [7b215ed] Use default qemu-img cluster size in libvirt connection driver
    - [d3a87a2] Listing flavors with marker set returns 400 (LP: #956096)
    - [cf6a85a] nova-rootwrap hardcodes paths instead of using
      /sbin:/usr/sbin:/usr/bin:/bin (LP: #1013147)
    - [2efc87c] affinity filters don't work if scheduler_hints is None
      (LP: #1007573)
  ...

Read more...

Changed in nova (Ubuntu Precise):
status: Confirmed → Fix Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Thierry Carrez (ttx) on 2012-09-27
Changed in nova:
milestone: folsom-2 → 2012.2
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related questions