Bug #1298350 “Heat is unable to detach volumes from servers” : Bugs : OpenStack Heat

Sergey Kraynev (skraynev) on 2014-03-27

Changed in heat:
assignee:	nobody → Sergey Kraynev (skraynev)

Revision history for this message

Alexander Ignatov (aignatov) wrote on 2014-03-27:

#1

Actually this issue blocks Sahara to use Heat as provisioning engine.
Related Sahara bug: https://bugs.launchpad.net/sahara/+bug/1293962

It should be note there that direct provisioning Sahara code works well in such case.

Revision history for this message

Alexander Ignatov (aignatov) wrote on 2014-03-27:

#2

Also I looked at the VolumeDetachTask https://github.com/openstack/heat/blob/master/heat/engine/resources/volume.py#L232

and noticed that its code uses Nova client to detach volumes. Probably it would be better to use Cinder client there since it has detach() action as well.

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-03-27:

#3

check my https://review.openstack.org/#/c/72681, may be that will solve your problem

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-03-27:

#4

I mangled with VolumeDetachTask there, and I think made it more obust

Andrew Lazarev (alazarev) on 2014-03-27

description:

updated

Alexander Ignatov (aignatov) on 2014-03-27

description:

updated

Steve Baker (steve-stevebaker) on 2014-03-28

Changed in heat:
milestone:	none → juno-1
importance:	Undecided → High

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-03-28:

#5

I just wonder why you have mountpoint: null in your templates?

Revision history for this message

Sergey Kraynev (skraynev) wrote on 2014-03-28:

#6

Pavlo as explanation on your comment about null mountpoint https://bugs.launchpad.net/heat/+bug/1260345

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-03-28:

#7

I was able to reproduce on local devstack with provided template, but not every time. One instance was left in Error state due to "not able co connect to Neutron, max retries exceeded" (according to nova show). Looks like a race somewhere.

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-03-28:

#8

Have tried it using https://review.openstack.org/#/c/72681/ - it does not solve this problem.

Revision history for this message

Alexander Ignatov (aignatov) wrote on 2014-03-28:

#9

See my note in bug description, more resources in use will increase probability of this issue.

Revision history for this message

Ramashri Umale (ramashri) wrote on 2014-03-28:

#10

I was facing this issue consistently with devstack and single volume attached to an instance. The root cause I found to be twofold.

1) heat is issuing multiple detach commands for same volume triggering a race condition
2) race condition in nova volume detach that puts the volume in an inconsistent state. It has been explained very well in bug https://bugs.launchpad.net/cinder/+bug/1238093.

Anyway the end result of the bug is that the heat stack shows VolumeAttachment as DELETE_FAILED. A look into cinder db shows volume status to be 'in-use' and 'detached' instead of being 'available' and detached. A look inside the instance on which volume was mounted shows the volume as having already been detached.

From a heat perspective I found that removing un-neccessary call to detach the same volume again resolved the issue for me

In the __call__ subroutine of class VolumeDetachTask in file heat/engine/resources/volume.py

change
            while vol.status in ('in-use', 'detaching'):
                logger.debug(_('%s - volume still in use') % str(self))
                yield

                try:
                    server_api.delete_server_volume(self.server_id,
                                                    self.volume_id)
                except (clients.novaclient.exceptions.BadRequest,
                        clients.novaclient.exceptions.NotFound):
                    pass
                vol.get()

to

            while vol.status in ('in-use', 'detaching'):
                logger.debug(_('%s - volume still in use') % str(self))
                yield
                vol.get()

Revision history for this message

Alexander Ignatov (aignatov) wrote on 2014-03-29:

#11

Ramashri, good finding! Also I've looked at the history of VolumeDetachTask and noticed that it uses Nova client actions to work with volumes. I think it's designed this way because in that time of DetachTask develompent Cinder client didn't have all needed rest calls to work with. Now Cinder and its client contains all needed functions, and probably Cinder::Volume and Cinder::VolumeAttachment should be rewritten to use it. As a proof I can show an example that in Sahara provisioning code works fine in the similar case just having this code:
https://github.com/openstack/sahara/blob/master/sahara/service/volumes.py#L157

Steve Baker (steve-stevebaker) on 2014-04-08

Changed in heat:
status:	New → Triaged
milestone:	juno-1 → icehouse-rc2

Sergey Kraynev (skraynev) on 2014-04-08

Changed in heat:
assignee:	Sergey Kraynev (skraynev) → nobody

Pavlo Shchelokovskyy (pshchelo) on 2014-04-08

Changed in heat:
assignee:	nobody → Pavlo Shchelokovskyy (pshchelo)

Steve Baker (steve-stevebaker) on 2014-04-09

Changed in heat:
milestone:	icehouse-rc2 → juno-1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-10: Related fix proposed to heat (master)

#12

Related fix proposed to branch: master
Review: https://review.openstack.org/86620

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-10:

#13

Related fix proposed to branch: master
Review: https://review.openstack.org/86638

OpenStack Infra (hudson-openstack) on 2014-04-11

Changed in heat:
status:	Triaged → In Progress

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-04-11:

#14

I have abandoned https://review.openstack.org/86620 (first one above) in favor of https://review.openstack.org/86638 (second one above). The latter uses native cinder API calls for attachment and detachment of volumes and seems to work even on my local devstack, so I think that it is the way to fix this bug.

Revision history for this message

Openstack Gerrit (openstack-gerrit) wrote on 2014-04-21: Fix merged to heat (master)

#15

Reviewed: https://review.openstack.org/86638
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e5853057142313b3ed99b66786d1d3164675f211
Submitter: Jenkins
Branch: master

commit e5853057142313b3ed99b66786d1d3164675f211
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Wed Apr 9 14:58:02 2014 +0000

Use cinder API for volume attach and detach

Replace nova calls for attaching and detaching volumes
with native cinder API calls.

Closes-Bug: #1298350
Change-Id: I628b4a0dd559389f66ab25b6030a9ad6fc927e39

Changed in heat:
status:	In Progress → Fix Committed

Steve Baker (steve-stevebaker) on 2014-05-27

tags:

added: icehouse-backport-potential

Thierry Carrez (ttx) on 2014-06-11

Changed in heat:
status:	Fix Committed → Fix Released

Revision history for this message

David Hill (david-hill-ubisoft) wrote on 2014-08-30:

#16

Hi guys,

I applied the patch to Icehouse and tried it. The volume gets attached only in the database ! What I mean by this is that tgt never creates the iscsi session and thus, never attaches the volume to the instance ... I`m still trying to understand what`s missing ! I`m still running cinder from icehouse so perhaps that`s my issue ! Otherwise, something`s missing somewhere.

Dave

Revision history for this message

David Hill (david-hill-ubisoft) wrote on 2014-08-31:

#17

Download full text (15.4 KiB)

Hi guys,

Me again. I've adapted this patch in order to actually do something but from the API documentation, attach and detach are only changing the metadata. I think we actually need to create the iscsi connection and tear it down once completed. Also, my patch is not clean because it's explointing a validation issue in the metadata where the volume is not longer attached according to the metadata and thus can actually delete the instance. I tried to understand how to actually delete the iscsi session (and create it) but it seems a bit complicated for my knowledge of openstack and python (I don't know python after all).

Dave

FYI

From: David Hill
Sent: 30-Aug-14 6:33 PM
To: openstack (<email address hidden>)
Subject: Heat patch for Icehouse vs Volume deletion.

Hi guys,

I’ve wrote simple patch that takes care of the volume deletion and it’s ugly. I’m posting it here so anybody with better code knowledge on heat can base his patch on it if it’s very that ugly 

For some reasons, sometimes an instance will fail to delete, this takes care of it and will retry until it disappears:
--- instance.py.orig 2014-08-30 22:07:45.259328109 +0000
+++ instance.py 2014-08-30 22:23:03.180527084 +0000
@@ -587,8 +587,13 @@
                     self.resource_id_set(None)
                     break
                 elif server.status == "ERROR":
- raise exception.Error(_("Deletion of server %s failed.") %
- server.id)
+ while server.status == "ERROR":
+ nova_utils.refresh_server(server)
+ server.reset_state()
+ server.delete()
+ if server.status == "DELETED":
+ self.resource_id_set(None)
+ break
             except clients.novaclient.exceptions.NotFound:
                 self.resource_id_set(None)
                 break

This patch is based on the following bug: https://bugs.launchpad.net/heat/+bug/1298350 => https://review.openstack.org/#/c/86638/
It didn’t work for me but this one works flawlessly.

--- volume.py.icehouse 2014-08-30 00:59:49.844290344 +0000
+++ volume.py 2014-08-30 21:58:08.769813146 +0000
@@ -151,11 +151,11 @@
                 if vol.status == 'in-use':
                     logger.warn(_('can not delete volume when in-use'))
                     raise exception.Error(_('Volume in use'))
-
- vol.delete()
- while True:
- yield
- vol.get()
+ if vol.status != 'deleting':
+ vol.delete()
+ while True:
+ yield
+ vol.get()
             except clients.cinderclient.exceptions.NotFound:
                 self.resource_id_set(None)

@@ -227,69 +227,187 @@

logger.info(_('%s - complete') % str(self))

-
-class VolumeDetachTask(object):
+class VolumeDeleteTask(object):
"""A task for detaching a volume from a Nova server."""

- def __init__(self, stack, server_id, attachment_id):
+ def __init__(sel...

Hi guys,

Me again.   I've adapted this patch in order to actually do something but from the API documentation, attach and detach are only changing the metadata.  I think we actually need to create the iscsi connection and tear it down once completed.   Also, my patch is not clean because it's explointing a validation issue in the metadata where the volume is not longer attached according to the metadata and thus can actually delete the instance.   I tried to understand how to actually delete the iscsi session (and create it) but it seems a bit complicated for my knowledge of openstack and python (I don't know python after all).

Dave

FYI

From: David Hill 
Sent: 30-Aug-14 6:33 PM
To: openstack (openstack@lists.openstack.org)
Subject: Heat patch for Icehouse vs Volume deletion.

Hi guys,

I’ve wrote simple patch that takes care of the volume deletion and it’s ugly.  I’m posting it here so anybody with better code knowledge on heat can base his patch on it if it’s very that ugly 

For some reasons, sometimes an instance will fail to delete, this takes care of it and will retry until it disappears:
--- instance.py.orig    2014-08-30 22:07:45.259328109 +0000
+++ instance.py 2014-08-30 22:23:03.180527084 +0000
@@ -587,8 +587,13 @@
                     self.resource_id_set(None)
                     break
                 elif server.status == "ERROR":
-                    raise exception.Error(_("Deletion of server %s failed.") %
-                                          server.id)
+                    while server.status == "ERROR":
+                        nova_utils.refresh_server(server)
+                       server.reset_state()
+                       server.delete()
+                        if server.status == "DELETED":
+                            self.resource_id_set(None)
+                            break
             except clients.novaclient.exceptions.NotFound:
                 self.resource_id_set(None)
                 break

This patch is based on the following bug: https://bugs.launchpad.net/heat/+bug/1298350 => https://review.openstack.org/#/c/86638/
It didn’t work for me but this one works flawlessly.

--- volume.py.icehouse               2014-08-30 00:59:49.844290344 +0000
+++ volume.py 2014-08-30 21:58:08.769813146 +0000
@@ -151,11 +151,11 @@
                 if vol.status == 'in-use':
                     logger.warn(_('can not delete volume when in-use'))
                     raise exception.Error(_('Volume in use'))
-
-                vol.delete()
-                while True:
-                    yield
-                    vol.get()
+                             if vol.status != 'deleting':
+                    vol.delete()
+                    while True:
+                        yield
+                        vol.get()
             except clients.cinderclient.exceptions.NotFound:
                 self.resource_id_set(None)

@@ -227,69 +227,187 @@

logger.info(_('%s - complete') % str(self))

-
-class VolumeDetachTask(object):
+class VolumeDeleteTask(object):
     """A task for detaching a volume from a Nova server."""

-    def __init__(self, stack, server_id, attachment_id):
+    def __init__(self, stack, volume_id):
         """
         Initialise with the stack (for obtaining the clients), and the IDs of
         the server and volume.
         """
         self.clients = stack.clients
-        self.server_id = server_id
-        self.attachment_id = attachment_id
+        self.volume_id = volume_id

def __str__(self):
         """Return a human-readable string description of the task."""
-        return _('Removing attachment %(att)s from Instance %(srv)s') % {
-            'att': self.attachment_id, 'srv': self.server_id}
+        return _('Deleting volume %(vol)s') % {
+            'vol': self.volume_id}

def __repr__(self):
         """Return a brief string description of the task."""
-        return '%s(%s -/> %s)' % (type(self).__name__,
-                                  self.attachment_id,
-                                  self.server_id)
+        return '%s(%s)' % (type(self).__name__,
+                                  self.volume_id)

def __call__(self):
         """Return a co-routine which runs the task."""
         logger.debug(str(self))

-        server_api = self.clients.nova().volumes
-
-        # get reference to the volume while it is attached
         try:
-            nova_vol = server_api.get_server_volume(self.server_id,
-                                                    self.attachment_id)
-            vol = self.clients.cinder().volumes.get(nova_vol.id)
-        except (clients.cinderclient.exceptions.NotFound,
-                clients.novaclient.exceptions.BadRequest,
-                clients.novaclient.exceptions.NotFound):
+            vol = self.clients.cinder().volumes.get(self.volume_id)
+            if vol.status in ('available'):
+                             vol.delete()
+                yield
+            while vol.status in ('available', 'deleting'):
+                logger.debug(_('%s - volume is still %s') % ( str(self), vol.status ))
+                yield
+                vol.get()
+            logger.info(_('%(name)s - status: %(status)s') % {
+                        'name': str(self), 'status': vol.status})
+
+        except clients.cinderclient.exceptions.NotFound:
             logger.warning(_('%s - volume not found') % str(self))
             return
+        except clients.cinderclient.exceptions.BadRequest:
+            msg = _('Failed to delete %(vol)s') % {
+                'vol': self.volume_id}
+            raise exception.Error(msg)

-        # detach the volume using volume_attachment
-        try:
-            server_api.delete_server_volume(self.server_id, self.attachment_id)
-        except (clients.novaclient.exceptions.BadRequest,
-                clients.novaclient.exceptions.NotFound) as e:
-            logger.warning('%(res)s - %(err)s' % {'res': str(self),
-                                                  'err': str(e)})
+        logger.info(_('%s - complete') % str(self))

-        yield
+class VolumeAttachment(resource.Resource):
+    PROPERTIES = (
+        INSTANCE_ID, VOLUME_ID, DEVICE,
+    ) = (
+        'InstanceId', 'VolumeId', 'Device',
+    )
+
+    properties_schema = {
+        INSTANCE_ID: properties.Schema(
+            properties.Schema.STRING,
+            _('The ID of the instance to which the volume attaches.'),
+            required=True,
+            update_allowed=True
+        ),
+        VOLUME_ID: properties.Schema(
+            properties.Schema.STRING,
+            _('The ID of the volume to be attached.'),
+            required=True,
+            update_allowed=True
+        ),
+        DEVICE: properties.Schema(
+            properties.Schema.STRING,
+            _('The device where the volume is exposed on the instance. This '
+              'assignment may not be honored and it is advised that the path '
+              '/dev/disk/by-id/virtio-<VolumeId> be used instead.'),
+            required=True,
+            update_allowed=True,
+            constraints=[
+                constraints.AllowedPattern('/dev/vd[b-z]'),
+            ]
+        ),
+    }
+
+    update_allowed_keys = ('Properties',)
+
+    def handle_create(self):
+        server_id = self.properties[self.INSTANCE_ID]
+        volume_id = self.properties[self.VOLUME_ID]
+        dev = self.properties[self.DEVICE]
+
+        attach_task = VolumeAttachTask(self.stack, server_id, volume_id, dev)
+        attach_runner = scheduler.TaskRunner(attach_task)
+
+        attach_runner.start()
+
+        self.resource_id_set(attach_task.attachment_id)
+
+        return attach_runner
+
+    def check_create_complete(self, attach_runner):
+        return attach_runner.step()
+
+    def handle_delete(self):
+        server_id = self.properties[self.INSTANCE_ID]
+        volume_id = self.properties[self.VOLUME_ID]
+        detach_task = VolumeDetachTask(self.stack, server_id, volume_id)
+        scheduler.TaskRunner(detach_task)()
+        delete_task = VolumeDeleteTask(self.stack,volume_id)
+        scheduler.TaskRunner(delete_task)()
+
+    def handle_update(self, json_snippet, tmpl_diff, prop_diff):
+        checkers = []
+        if prop_diff:
+            volume_id = self.properties.get(self.VOLUME_ID)
+            server_id = self.properties.get(self.INSTANCE_ID)
+            detach_task = VolumeDetachTask(self.stack, server_id, volume_id)
+            checkers.append(scheduler.TaskRunner(detach_task))
+
+            if self.VOLUME_ID in prop_diff:
+                volume_id = prop_diff.get(self.VOLUME_ID)
+            device = self.properties.get(self.DEVICE)
+            if self.DEVICE in prop_diff:
+                device = prop_diff.get(self.DEVICE)
+            if self.INSTANCE_ID in prop_diff:
+                server_id = prop_diff.get(self.INSTANCE_ID)
+            attach_task = VolumeAttachTask(self.stack, server_id,
+                                           volume_id, device)
+
+            checkers.append(scheduler.TaskRunner(attach_task))
+
+        if checkers:
+            checkers[0].start()
+        return checkers
+
+    def check_update_complete(self, checkers):
+        for checker in checkers:
+            if not checker.started():
+                checker.start()
+            if not checker.step():
+                return False
+        self.resource_id_set(checkers[-1]._task.attachment_id)
+        return True
+
+
+class VolumeDetachTask(object):
+    """A task for detaching a volume from a Nova server."""
+
+    def __init__(self, stack, server_id, volume_id):
+        """
+        Initialise with the stack (for obtaining the clients), and the IDs of
+        the server and volume.
+        """
+        self.clients = stack.clients
+        self.server_id = server_id
+        self.volume_id = volume_id
+
+    def __str__(self):
+        """Return a human-readable string description of the task."""
+        return _('Removing volume %(vol)s from Instance %(srv)s') % {
+            'vol': self.volume_id, 'srv': self.server_id}
+
+    def __repr__(self):
+        """Return a brief string description of the task."""
+        return '%s(%s -/> %s)' % (type(self).__name__,
+                                  self.volume_id,
+                                  self.server_id)
+
+    def __call__(self):
+        """Return a co-routine which runs the task."""
+        logger.debug(str(self))

try:
-            vol.get()
+            vol = self.clients.cinder().volumes.get(self.volume_id)
+            attached_to = [att['server_id'] for att in vol.attachments]
+            if self.server_id not in attached_to:
+                msg = _('Volume %(vol)s is not attached to server %(srv)s') % {
+                    'vol': self.volume_id, 'srv': self.server_id}
+                raise exception.Error(msg)
+
+            vol.detach()
+            yield
             while vol.status in ('in-use', 'detaching'):
                 logger.debug(_('%s - volume still in use') % str(self))
                 yield
-
-                try:
-                    server_api.delete_server_volume(self.server_id,
-                                                    self.attachment_id)
-                except (clients.novaclient.exceptions.BadRequest,
-                        clients.novaclient.exceptions.NotFound):
-                    pass
                 vol.get()

logger.info(_('%(name)s - status: %(status)s') % {
@@ -299,27 +417,32 @@

except clients.cinderclient.exceptions.NotFound:
             logger.warning(_('%s - volume not found') % str(self))
+            return
+        except clients.cinderclient.exceptions.BadRequest:
+            msg = _('Failed to detach %(vol)s from server %(srv)s') % {
+                'vol': self.volume_id, 'srv': self.server_id}
+            raise exception.Error(msg)

# The next check is needed for immediate reattachment when updating:
-        # as the volume info is taken from cinder, but the detach
-        # request is sent to nova, there might be some time
-        # between cinder marking volume as 'available' and
-        # nova removing attachment from it's own objects, so we
+        # there might be some time between cinder marking volume as 'available'
+        # and nova removing attachment from it's own objects, so we
         # check that nova already knows that the volume is detached
-        def server_has_attachment(server_id, attachment_id):
-            try:
-                server_api.get_server_volume(server_id, attachment_id)
-            except clients.novaclient.exceptions.NotFound:
+        server_api = self.clients.nova().volumes
+
+        def server_has_attachment(server_id, volume_id):
+            vol = self.clients.cinder().volumes.get(self.volume_id)
+            attached_to = [att['server_id'] for att in vol.attachments]
+            if self.server_id not in attached_to:
                 return False
-            return True
+            else:
+                return True

-        while server_has_attachment(self.server_id, self.attachment_id):
-            logger.info(_("Server %(srv)s still has attachment %(att)s.") %
-                        {'att': self.attachment_id, 'srv': self.server_id})
+        while server_has_attachment(self.server_id, self.volume_id):
+            logger.info(_("Server %(srv)s still has %(vol)s attached.") %
+                        {'vol': self.volume_id, 'srv': self.server_id})
             yield

-        logger.info(_("Volume %(vol)s is detached from server %(srv)s") %
-                    {'vol': vol.id, 'srv': self.server_id})
+        logger.info(_('%s - complete') % str(self))

class VolumeAttachment(resource.Resource):
@@ -376,29 +499,25 @@

def handle_delete(self):
         server_id = self.properties[self.INSTANCE_ID]
-        detach_task = VolumeDetachTask(self.stack, server_id, self.resource_id)
+        volume_id = self.properties[self.VOLUME_ID]
+        detach_task = VolumeDetachTask(self.stack, server_id, volume_id)
         scheduler.TaskRunner(detach_task)()
+        delete_task = VolumeDeleteTask(self.stack,volume_id)
+        scheduler.TaskRunner(delete_task)()

def handle_update(self, json_snippet, tmpl_diff, prop_diff):
         checkers = []
         if prop_diff:
-            # Even though some combinations of changed properties
-            # could be updated in UpdateReplace manner,
-            # we still first detach the old resource so that
-            # self.resource_id is not replaced prematurely
             volume_id = self.properties.get(self.VOLUME_ID)
+            server_id = self.properties.get(self.INSTANCE_ID)
+            detach_task = VolumeDetachTask(self.stack, server_id, volume_id)
+            checkers.append(scheduler.TaskRunner(detach_task))
+
             if self.VOLUME_ID in prop_diff:
                 volume_id = prop_diff.get(self.VOLUME_ID)
-
             device = self.properties.get(self.DEVICE)
             if self.DEVICE in prop_diff:
                 device = prop_diff.get(self.DEVICE)
-
-            server_id = self.properties.get(self.INSTANCE_ID)
-            detach_task = VolumeDetachTask(self.stack, server_id,
-                                           self.resource_id)
-            checkers.append(scheduler.TaskRunner(detach_task))
-
             if self.INSTANCE_ID in prop_diff:
                 server_id = prop_diff.get(self.INSTANCE_ID)
             attach_task = VolumeAttachTask(self.stack, server_id,
@@ -419,7 +538,6 @@
         self.resource_id_set(checkers[-1]._task.attachment_id)
         return True

-
class CinderVolume(Volume):

PROPERTIES = (

Revision history for this message

Steve Baker (steve-stevebaker) wrote on 2014-09-01:

#18

Hi David

Could you please submit this patch as a http://review.openstack.org gerrit review? (even if it is WIP)

This will make it easier for us to see and discuss what your change is doing.

Revision history for this message

David Hill (david-hill-ubisoft) wrote on 2014-09-02:

#19

Ok

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-09-03: Fix proposed to heat (stable/icehouse)

#20

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/118517

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-09-03:

#21

David, that is the wrong patch to backport. I was the original author, but then we discovered that the cinder API is indeed confusingly only operates on cinder db and not on nova servers. The patch was reverted and rewritten. Please see the following patch that really fixed it

https://review.openstack.org/#/c/89796/

(there is also a bug reference)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-09-03: Change abandoned on heat (stable/icehouse)

#22

Change abandoned by David Hill (<email address hidden>) on branch: stable/icehouse
Review: https://review.openstack.org/118517
Reason: Useless... There's so many commits in here that we easily get lost. ;)

Thierry Carrez (ttx) on 2014-10-16

Changed in heat:
milestone:	juno-1 → 2014.2

Revision history for this message

Zane Bitter (zaneb) wrote on 2014-12-05:

#23

Pavlo, can you backport the correct patch to icehouse/stable?

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-12-05:

#24

Zane, will do

Revision history for this message

Pavlo Shchelokovskyy (pshchelo) wrote on 2014-12-05:

#25

One more question - should I also align the patch to AWS (we mistakenly allow updates to AWS Volume Attachments in Icehouse), or that would too much for the backport?

Revision history for this message

Zane Bitter (zaneb) wrote on 2014-12-10:

#26

Probably better not to make changes like that in a backport if you can easily avoid it.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-12-16: Fix merged to heat (stable/icehouse)

#27

Reviewed: https://review.openstack.org/139972
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=0b984b8c13338cf2ec3836e21e71c3c955a148b4
Submitter: Jenkins
Branch: stable/icehouse

commit 0b984b8c13338cf2ec3836e21e71c3c955a148b4
Author: Sushil Kumar <email address hidden>
Date: Wed Apr 23 11:18:31 2014 +0000

Call server volume detach only once

    Changes:
     - Removed duplicate delete calls to prevent race condition.
     - Removed duplicate mocked calls from unit-test.

Closes-Bug: #1311533
Closes-Bug: #1298350

    Conflicts:
     heat/engine/resources/volume.py
     heat/tests/test_volume.py

Change-Id: I5f16c528652f12440160f03b92f41b76d1c9100c
(cherry picked from commit d1ffbd4bfde0cd6b7a82a48b9d4f59cc8b310bd8)

tags:

added: in-stable-icehouse

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Heat	Fix Released	High	Pavlo Shchelokovskyy	OpenStack Heat 2014.2 "juno"
	Icehouse	Fix Released	Undecided	Unassigned	OpenStack Heat 2014.1.4

OpenStack Heat

Heat is unable to detach volumes from servers

Bug Description

Other bug subscribers

Remote bug watches