libvirt, rbd imagebackend, disk.rescue not deleted when unrescued

Bug #1475652 reported by raphael.glon on 2015-07-17
50
This bug affects 7 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
melanie witt
Newton
Medium
Lee Yarwood

Bug Description

Reproduced on juno version (actually tested on a fork from 2014.2.3, sorry in advance if invalid but i think the legacy version is also concerned by it)

not tested on younger versions, but looking at the code they seem impacted too

For Rbd imagebackend only, when unrescuing an instance the disk.rescue file is actually not deleted on remote storage (only the rbd session is destroyed)

Consequence: when rescuing instance once again, it simply ignores the new rescue image and takes the old _disk.rescue image

Reproduce:

1. nova rescue instance

(take care that you are booted to the vda rescue disk -> when rescuing an instance from the same image it was spawned from (case by default), since fs uuid is the same, according to your image fstab (if entry UUID=) you can actually boot from the image you are actually trying to rescue, but this is another matter that concerns template building -> see https://bugs.launchpad.net/nova/+bug/1460536)

edit rescue image disk

2. nova unrescue instance

3. nova rescue instance -> you get back the disk.rescue spawned in 1

if confirmed, fix coming soon

Concerning fix several possibilities:
- nova.virt.libvirt.driver :LibvirtDriver-> unrescue method, not deleting the correct files
or
- nova.virt.libvirt.imagebackend:Rbd -> erase disk.rescue in create image method if already existing

Rebuild not concerned by issue, delete instance correctly deletes files on remote storage

description: updated
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/203086

Changed in nova:
assignee: nobody → raphael.glon (raphael-glon)
status: New → In Progress

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/203086
Reason: This patch has been stalled for quite a while, so I am going to abandon it to keep the code review queue sane. Please restore the change when it is ready for review.

Changed in nova:
assignee: raphael.glon (raphael-glon) → nobody
status: In Progress → Confirmed
Changed in nova:
assignee: nobody → Bartek Żurawski (bartekzurawski1)

Fix proposed to branch: master
Review: https://review.openstack.org/314928

Changed in nova:
status: Confirmed → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/318089

Change abandoned by Bartek Żurawski (<email address hidden>) on branch: master
Review: https://review.openstack.org/318089
Reason: Duplicate

melanie witt (melwitt) on 2016-07-01
tags: added: ceph
Changed in nova:
assignee: Bartek Żurawski (bartekzurawski1) → melanie witt (melwitt)

Reviewed: https://review.openstack.org/314928
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c12d388070895e40be19f4f4e5fded736a5376be
Submitter: Jenkins
Branch: master

commit c12d388070895e40be19f4f4e5fded736a5376be
Author: Bartek Zurawski <email address hidden>
Date: Tue May 10 17:31:19 2016 +0200

    Fix issue with not removing rbd rescue disk

    Currently when instance that use RBD as backend
    is rescued and next unrescued, rescue image is
    not removed, this cause issue when the same
    instance is rescued again it's use old rescue
    image not new one.

    Change-Id: Idf4086303baa4b936c90be89552ad8deb45cef3a
    Closes-Bug: #1475652

Changed in nova:
status: In Progress → Fix Released
Matt Riedemann (mriedem) on 2016-10-14
Changed in nova:
importance: Undecided → Medium

Reviewed: https://review.openstack.org/385925
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=332e11c714ced0ffc2921feddffa370ef5475949
Submitter: Jenkins
Branch: stable/newton

commit 332e11c714ced0ffc2921feddffa370ef5475949
Author: Bartek Zurawski <email address hidden>
Date: Tue May 10 17:31:19 2016 +0200

    Fix issue with not removing rbd rescue disk

    Currently when instance that use RBD as backend
    is rescued and next unrescued, rescue image is
    not removed, this cause issue when the same
    instance is rescued again it's use old rescue
    image not new one.

    Change-Id: Idf4086303baa4b936c90be89552ad8deb45cef3a
    Closes-Bug: #1475652
    (cherry picked from commit c12d388070895e40be19f4f4e5fded736a5376be)

This issue was fixed in the openstack/nova 14.0.2 release.

This issue was fixed in the openstack/nova 15.0.0.0b1 development milestone.

This issue was fixed in the openstack/nova 14.0.2 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers