Stack abandon deletes nested stacks

[Steven Hardy]

If you have a stack containing nested stack resources, abandon adopt is destructive wrt the child stacks.

Probably what most users desire is to abandon the parent but not destroy all the nested resources (or, at least to have an option to do this):

-bash-4.3$ heat stack-list -n
+----+------------+--------------+---------------+--------------+--------+
| id | stack_name | stack_status | creation_time | updated_time | parent |
+----+------------+--------------+---------------+--------------+--------+
+----+------------+--------------+---------------+--------------+--------+
-bash-4.3$ cat parent.yaml
heat_template_version: 2013-05-23

resources:
  nested:
    type: random.yaml
-bash-4.3$ cat random.yaml
heat_template_version: 2013-05-23

resources:
  random:
    type: OS::Heat::RandomString
-bash-4.3$ heat stack-create p1 -f parent.yaml
+--------------------------------------+------------+--------------------+---------------------+--------------+
| id | stack_name | stack_status | creation_time | updated_time |
+--------------------------------------+------------+--------------------+---------------------+--------------+
| db83e0f1-8aeb-491d-b4b9-6c2352ee95ea | p1 | CREATE_IN_PROGRESS | 2015-10-26T03:04:31 | None |
+--------------------------------------+------------+--------------------+---------------------+--------------+
-bash-4.3$ heat stack-list -n
+--------------------------------------+------------------------+-----------------+---------------------+--------------+--------------------------------------+
| id | stack_name | stack_status | creation_time | updated_time | parent |
+--------------------------------------+------------------------+-----------------+---------------------+--------------+--------------------------------------+
| db83e0f1-8aeb-491d-b4b9-6c2352ee95ea | p1 | CREATE_COMPLETE | 2015-10-26T03:04:31 | None | None |
| ef7cf115-8874-47c8-a846-c083a6660f2b | p1-nested-xcodbgvfdmk4 | CREATE_COMPLETE | 2015-10-26T03:04:31 | None | db83e0f1-8aeb-491d-b4b9-6c2352ee95ea |
+--------------------------------------+------------------------+-----------------+---------------------+--------------+--------------------------------------+
-bash-4.3$ heat stack-abandon p1 | tee abandon_data3.yaml
{
  "files": {
    "file:///opt/stack/heat-templates/hot/random.yaml": "{\"heat_template_version\": \"2013-05-23\", \"resources\": {\"random\": {\"type\": \"OS::Heat::RandomString\"}}}"
  },
  "status": "COMPLETE",
  "name": "p1",
  "tags": null,
  "stack_user_project_id": "0187766aedf34372bd243e6ce0321aaa",
  "environment": {
    "encrypted_param_names": [],
    "parameter_defaults": {},
    "parameters": {},
    "resource_registry": {
      "file:///opt/stack/heat-templates/hot/random.yaml": "file:///opt/stack/heat-templates/hot/random.yaml",
      "resources": {}
    }
  },
  "template": {
    "heat_template_version": "2013-05-23",
    "resources": {
      "nested": {
        "type": "file:///opt/stack/heat-templates/hot/random.yaml"
      }
    }
  },
  "action": "CREATE",
  "project_id": "16e9090934024521aa5203fd5bae4dd4",
  "id": "db83e0f1-8aeb-491d-b4b9-6c2352ee95ea",
  "resources": {
    "nested": {
      "files": {
        "file:///opt/stack/heat-templates/hot/random.yaml": "{\"heat_template_version\": \"2013-05-23\", \"resources\": {\"random\": {\"type\": \"OS::Heat::RandomString\"}}}"
      },
      "status": "COMPLETE",
      "name": "p1-nested-xcodbgvfdmk4",
      "tags": null,
      "stack_user_project_id": "0187766aedf34372bd243e6ce0321aaa",
      "environment": {
        "encrypted_param_names": [],
        "parameter_defaults": {},
        "parameters": {},
        "resource_registry": {
          "resources": {}
        }
      },
      "template": {
        "heat_template_version": "2013-05-23",
        "resources": {
          "random": {
            "type": "OS::Heat::RandomString"
          }
        }
      },
      "action": "CREATE",
      "project_id": "16e9090934024521aa5203fd5bae4dd4",
      "id": "ef7cf115-8874-47c8-a846-c083a6660f2b",
      "resources": {
        "random": {
          "status": "COMPLETE",
          "name": "random",
          "resource_data": {
            "value": "K008f6DV8LV96iUHyimv4AN7Ssdqd2t6"
          },
          "resource_id": "p1-nested-xcodbgvfdmk4-random-ok5uwj3b4nio",
          "action": "CREATE",
          "type": "OS::Heat::RandomString",
          "metadata": {}
        }
      }
    }
  }
}
-bash-4.3$ heat stack-list -n
+----+------------+--------------+---------------+--------------+--------+
| id | stack_name | stack_status | creation_time | updated_time | parent |
+----+------------+--------------+---------------+--------------+--------+
+----+------------+--------------+---------------+--------------+--------+
-bash-4.3$

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/239936

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/239936
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=103a09bc1c3e9f2e4894bba19963ca91b6652155
Submitter: Jenkins
Branch: master

commit 103a09bc1c3e9f2e4894bba19963ca91b6652155
Author: Peter Razumovsky <email address hidden>
Date: Wed Oct 28 16:37:06 2015 +0300

    Enable abandon option to nested resource

    Resources use abandon_in_progress option for abandoning in
    prepare_abandon method. When abandon_in_progress, resource's
    deletion policy is "Retain" and resource will not delete. Nested
    resource prepare_abandon missed enabling that option, so all nested
    stacks delete.

    Change-Id: I819566e4e1ca81193c745dcdd89217f31db41b07
    Closes-bug: #1509912

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/288426

[Zane Bitter (zaneb)]

Hmm, interesting. The way the code was written was intentional at the time when all nested stacks were handled in-memory in a single engine. We would go through the tree of stacks and mark all of the resources as about to be abandoned, and then go through all of the stacks and run a delete.

What broke it was splitting the nested stacks out to run in a separate engine in Kilo. From then on we went through and marked all the resources in the nested stack as about to be abandoned in-memory, then sent a message to another engine to delete it, and it actually deleted it for realz.

The patch above just creates the opposite problem: we went go through and mark all the resources in the nested stack as about to be abandoned, then... do nothing.

Fundamentally, marking stuff in-memory is not going to solve this problem (and this is another reason why abandon/adopt is fundamentally broken). The only real fix is to use the export step added for bug 1353670 in prepare_abandon().

[Zane Bitter (zaneb)]

Cedric pointed out on IRC that in the implementation of the fix for bug 1353670, the second step after exporting data was not, as I requested in that bug report, a DELETE, but rather an ABANDON. Since all of the other resources we delete by doing a DELETE with the DeletionPolicy forced to RETAIN, this means that to fix it we would have to somehow override the behaviour of Resource.delete() specifically in the case of a StackResource that is in the process of being abandoned. Which is horrible.

I was thinking that with that fix in we could rely on the stored state (i.e. that the nested stack was EXPORT_COMPLETE or whatever) to do the Right Thing on DELETE, but that isn't the case at all.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/309078

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/309078
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=d19c624e48ebc913dd91d99823fa238a9a34ef0e
Submitter: Jenkins
Branch: master

commit d19c624e48ebc913dd91d99823fa238a9a34ef0e
Author: Drago Rosson <email address hidden>
Date: Wed Apr 20 23:09:45 2016 +0000

    Make RPC call to abandon nested stack

    Due to stacks being processable by multiple engines, resources in a
    nested stack cannot be marked for retainment from the parent resource
    of the nested stack. This patch makes the parent resource of the nested
    stack call stack abandon instead of stack delete through the RPC client
    so that nested stacks are deleted while nested stack resources are
    retained.

    Change-Id: I7103f46d86db35e08b014d9790215740fcae441d
    Closes-bug: #1509912

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/311234

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/311642

[OpenStack Infra (hudson-openstack)]

Change abandoned by Drago (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/311234

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/311642
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=aabd192a47e2e018e6f24be17ad333121bbe247e
Submitter: Jenkins
Branch: stable/mitaka

commit aabd192a47e2e018e6f24be17ad333121bbe247e
Author: Drago Rosson <email address hidden>
Date: Wed Apr 20 23:09:45 2016 +0000

    Make RPC call to abandon nested stack

    Due to stacks being processable by multiple engines, resources in a
    nested stack cannot be marked for retainment from the parent resource
    of the nested stack. This patch makes the parent resource of the nested
    stack call stack abandon instead of stack delete through the RPC client
    so that nested stacks are deleted while nested stack resources are
    retained.

    (Modified from the commit on master to incorporate the refinemeant made
    in I47e412169f58734b8684a645470180d95e13def6.)

    Change-Id: I7103f46d86db35e08b014d9790215740fcae441d
    Closes-bug: #1509912
    (cherry picked from commit d19c624e48ebc913dd91d99823fa238a9a34ef0e)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/heat 6.1.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sergey Kraynev (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/288426
Reason: According decision on weekly team meeting, we need to abandon this patch due to EOL for liberty and broken gate state.