volume and snapshot IDs do not correctly map to UUIDs after folsom upgrade

[Adam Gandelman]

Similar to the EC2 instance ID issue in Bug #1061166, the snapshot_id_mappings and volume_id_mappings tables were both introduced in Folsom but are not populated entirely during creation, with the deleted column ending up NULL for all rows copied during the initial migration.

After an upgrade, queries that hope to map volume or snapshot IDs to UUIDs fail because they query with a 'deleted=0' constraint. Mappings for volumes or snapshots that existed pre-upgrade cannot be found and new entries are created, polluting the table with duplicates and skewing the ID <-> UUIDs mapping.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/14372

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/14372
Committed: http://github.com/openstack/nova/commit/43bf8344488a6af1046344c255bdb2edd7450475
Submitter: Jenkins
Branch: master

commit 43bf8344488a6af1046344c255bdb2edd7450475
Author: Adam Gandelman <email address hidden>
Date: Thu Oct 11 18:49:54 2012 -0700

    read_deleted snapshot and volume id mappings.

    Since the migration that creates the volume_id_mappings and
    snapshot_id_mappings tables does not populate the 'deleted' column,
    queries to this table should not limit results to 'deleted=0'.
    Limiting to non-deleted rows results in duplicate mappings being
    created for existing instance mappings after an upgrade, and
    throws off volume and snapshot ID to UUID mapping.

    This is a stop-gap measure to ensure avoid serious breakage during
    an upgrade. The NULL columns in this table are actually unused
    currently and can be populated in later grizzly migrations, if they
    end up being used.

    Fixes bug 1065785.

    Change-Id: I893c994362a203288b9984f0ede24cbb274bfcc5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/14416

[Launchpad Janitor (janitor)]

This bug was fixed in the package nova - 2012.2-0ubuntu5

---------------
nova (2012.2-0ubuntu5) quantal-proposed; urgency=low

  [ Adam Gandelman ]
  * Move management of /var/lib/nova/volumes from nova-common to
    nova-volume. Ensure it has proper permissions. (LP: #1065320)
  * debian/patches/avoid_setuptools_git_dependency.patch: Remove
    setuptools_git from tools/pip-requires to avoid it being automatically
    added to python-nova's runtime dependencies. (LP: #1059907)

  [ Chuck Short ]
  * debian/patches/rbd-security.patch: Support override of ceph rbd
    user and secret in nova-compute. (LP: #1065883)
  * debian/patches/ubuntu/fix-libvirt-firewall-slowdown.patch: Fix
    refreshing of security groups in libvirt not to block on RPC calls.
    (LP: #1062314)
  * debian/patches/ubuntu/fix-ec2-volume-id-mappings.patch: Read deleted
    snapshot and volume id mappings. (LP: #1065785)
 -- Chuck Short <email address hidden> Fri, 12 Oct 2012 12:35:01 -0500

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/14416
Committed: http://github.com/openstack/nova/commit/32d8722d3bf86946a56afc74b9cc301a39a2bef1
Submitter: Jenkins
Branch: stable/folsom

commit 32d8722d3bf86946a56afc74b9cc301a39a2bef1
Author: Adam Gandelman <email address hidden>
Date: Thu Oct 11 18:49:54 2012 -0700

    read_deleted snapshot and volume id mappings.

    Since the migration that creates the volume_id_mappings and
    snapshot_id_mappings tables does not populate the 'deleted' column,
    queries to this table should not limit results to 'deleted=0'.
    Limiting to non-deleted rows results in duplicate mappings being
    created for existing instance mappings after an upgrade, and
    throws off volume and snapshot ID to UUID mapping.

    This is a stop-gap measure to ensure avoid serious breakage during
    an upgrade. The NULL columns in this table are actually unused
    currently and can be populated in later grizzly migrations, if they
    end up being used.

    Fixes bug 1065785.

    Change-Id: I893c994362a203288b9984f0ede24cbb274bfcc5
    (cherry picked from commit 43bf8344488a6af1046344c255bdb2edd7450475)

[Clint Byrum (clint-fewbar)]

Hello Adam, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...