TemplateResource RefId is wrong
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Heat |
Fix Released
|
Medium
|
Zane Bitter |
Bug Description
The hot incantation "{get_resource: foo}" returns an AWS ARN when foo is a stack. Other resource types have parameters whose value should be the UUID of a resource. Such a value is not available to a template author that wants to refer to a resource that happens to be a stack.
Steven Hardy (shardy) wrote : | #1 |
Mike Spreitzer (mike-spreitzer) wrote : | #2 |
The cases I noticed are when "foo" is an AWS::CloudForma
Mike Spreitzer (mike-spreitzer) wrote : | #3 |
Maybe the original description of this bug was not clear. The problem concerns the interaction of two resources, let us call them A and B. The problem happens when a property of B is supposed to be given a value that is the ID of a resource (such as A); if A is a stack then the template author can not write "{get_resource: A}" as the value of the B property in question, because {get_resource: A} does not return a resource ID in this case.
Steven Hardy (shardy) wrote : | #4 |
Mike: If you're talking about the AWS stack resource, then returning an ARN is not unexpected:
http://
Like I said, what get_resource (equivalent to "Ref" in CFN templates) returns is dependent on the resource, not the template DSL you use so there is no "hot inantation" of an AWS resource.
So what I think you are saying is that there's a bug in provider resources (which are *not* AWS Stack resources), e.g template resource GetRefId has a bug here:
https:/
That is a bug, this (in the AWS Stack resource) is not:
https:/
Steven Hardy (shardy) wrote : | #5 |
Sorry, quoting with typos, need more coffee, I meant incantation there obviously ;)
summary: |
- Nested stacks have unexpected IDs + TemplateResource RefId is wrong |
Changed in heat: | |
status: | New → Confirmed |
importance: | Undecided → Medium |
assignee: | nobody → Steven Hardy (shardy) |
milestone: | none → kilo-1 |
Steven Hardy (shardy) wrote : | #6 |
btw - if for future reports you're able to provide a reproducer template that would be great, as it's generally a quicker way to understand the issue someone is having than abstract discussions about resource interactions.
Steven Hardy (shardy) wrote : | #7 |
Having discussed this with asalkeld, it seems it was a deliberate choice to make TemplateResource return an ARN, otherwise heat resource-show won't work on the provider resource.
Maybe there's some way we can resolve that, as I do think, if possible, it'd be better for get_resource to return the stack ID by default, for native OS:: resources.
Mike - it sounds like this is preventing you from doing something, but you haven't stated what, so can you please provide a concrete use-case for this, ideally a template you'd like to work but doesn't?
Zane Bitter (zaneb) wrote : | #8 |
My original vision of TemplateResource was to allow the user to specify a special Output in their template to specify the resource ID (this would allow a TemplateResource to properly stand in for some other resource type in a template).
Whether or not we think that's a good idea, it's pretty clear to me that anything that isn't an AWS:: resource should *never* return an ARN.
Mike Spreitzer (mike-spreitzer) wrote : | #9 |
Zane, regarding the first part of comment #8: was your vision that a template might output a resource ID that is the resource ID of one of the resources it contains, or a resource ID that the template synthesizes (e.g., by string munging or random string generation)? I would be surprised by a system that has the same resource ID on two different resources (e.g., a stack and also a resource contained in that stack).
This connects to the bigger picture here, which I now see I did not describe clearly enough at the beginning. The problem is not that I am personally surprised by the output of {get_resource: somestack}. The problem is that a template author has no way to refer to a stack, in places where reference is done by resource ID. The first place I noticed this is for OS::Heat:
Mike Spreitzer (mike-spreitzer) wrote : | #10 |
Sorry, that was maybe a little too sloppy. The problem is that a template author has no way to refer to a resource that is a nested stack, in places where reference is done by resource ID.
Concrete example 1:
resources:
somestack:
type: AWS::CloudForma
properties: {...stuff...}
replacer:
type: OS::Heat:
properties:
InstanceID: {get_resource: somestack}
That does not work.
Concrete example 2:
resources:
sg:
type: OS::Heat:
properties:
resource_def:
type: Something.yaml
properties: ...
removal_
Steven Hardy (shardy) wrote : | #11 |
> InstanceID: {get_resource: somestack}
Why would you ever expect this to work, given that somestack returns a unique reference (not necessarily the resource ID) to a stack, not an instance?
> resource_id_list: [ ... the result of FnGetRefId of a member
This will work, in fact it already does in the patch I posted.
Steven Hardy (shardy) wrote : | #12 |
> My original vision of TemplateResource was to allow the user to specify a special Output in their template to specify the resource ID
+100 - stevebaker and I were discussing this in the context of provider-resource abstracted SoftwareConfig resources recently, and IMO it's something we do need.
I don't think it's "same resource ID on two different resources" at all, merely giving some control to template authors who want to use provider resources for encapulation transparently. The resource has exactly one ID, as does the stack, you're just changing what get_resource references for template-author convenience.
Zane Bitter (zaneb) wrote : | #13 |
> was your vision that a template might output a resource ID that is the resource ID of one of the resources it contains, or a resource ID that the template synthesizes (e.g., by string munging or random string generation)?
I figured it would be an output, so the user could in principle do whatever they like, but the most common case would be to reference a resource ID from within the nested template. So e.g. you could specify in your environment file a mapping from OS::Nova::Server -> my_customised_
Mike Spreitzer (mike-spreitzer) wrote : | #14 |
Re concrete example 1: HARestarter can delete a nested stack. It can delete any kind of resource. Sometimes I actually want it to delete a nested stack.
Zane Bitter (zaneb) wrote : | #15 |
No design decision should ever be made on the basis of what's best for HARestarter.
That said, I don't see why your example #1 wouldn't work, unless another resource in the parent stack had the same ID.
Mike Spreitzer (mike-spreitzer) wrote : | #16 |
While developing patch set 4 of https:/
For a nested stack resource, the refid and the resource ID are different --- the former is an ARN and the latter is not.
The bug I am trying to report is that this difference causes grief.
There are several ways it could be fixed, even while retaining the fact that the refid of an AWS::CloudForma
Mike Spreitzer (mike-spreitzer) wrote : | #17 |
My concern is not for HARestarter but for users that want health maintenance. As you see, I have found a way to get it despite this bug. If you want to leave HARestarter uniquely screwed, that is OK with me if you find the workaround in https:/
Mike Spreitzer (mike-spreitzer) wrote : | #18 |
Part of my concern here is that I realized that Heat has some sharp edges. Do we really want these sharp edges? I have just done some language lawyering, and here is what I have realized.
In the Template Guide --- which is found under Python Developer documentation but acknowledged to also be relevant to users --- the section about resources (http://
Python developers see that a Resource has an "id", and fall into the habit of saying "resource ID" when we mean that field. I am not the only one who has done this; in comment #8 here, Zane followed my lead in this direction.
Of course, Resource.id does not hold what the Template Guide defines to be the resource's ID. It is actually the resource's reference ID in many cases but not all.
It is fast to look up a Resource by it's Resource.id (that being the primary key). For a general given resource reference ID, there is no general way to look up the reference except by enumerating the resources of the stack (presuming the relevant stack is already known) and comparing reference IDs. Are we even guaranteed uniqueness?
Steve Baker (steve-stevebaker) wrote : Re: [Bug 1381136] Re: TemplateResource RefId is wrong | #19 |
On 19/10/14 10:47, Mike Spreitzer wrote:
> Part of my concern here is that I realized that Heat has some sharp
> edges. Do we really want these sharp edges? I have just done some
> language lawyering, and here is what I have realized.
>
> In the Template Guide --- which is found under Python Developer
> documentation but acknowledged to also be relevant to users --- the
> section about resources
> (http://
> #resources-section) consistently uses the term "resource ID" for what I
> have been calling "resource name". I'll take that as a correction of my
> terminology. The Template Guide also has a definition of the
> get_resource intrinsic
> (http://
> #get-resource), and it consistently uses the term "resource ID" for that
> function's input and uses the term "reference ID" for that function's
> output.
Were in the process of writing the end-user focused template writing guide:
http://
Feel free to submit reviews for semantic clarifications there. Until the
new hot-guide is published I think we should write all new content in
the openstack-manuals repo and periodically sync back to the developer
template_guide (which will eventually be deleted).
Personally I'd like the spec to change <resource ID> to <resource name>
and for it to state somewhere that resource names are unique within a
stack. We already use <resource name> in other documentation artifacts:
$ heat help resource-show
usage: heat resource-show <NAME or ID> <RESOURCE>
Describe the resource.
Positional arguments:
<NAME or ID> Name or ID of stack to show the resource for.
<RESOURCE> Name of the resource to show the details for.
> Python developers see that a Resource has an "id", and fall into the
> habit of saying "resource ID" when we mean that field. I am not the
> only one who has done this; in comment #8 here, Zane followed my lead in
> this direction.
>
> Of course, Resource.id does not hold what the Template Guide defines to
> be the resource's ID. It is actually the resource's reference ID in
> many cases but not all.
>
> It is fast to look up a Resource by it's Resource.id (that being the
> primary key). For a general given resource reference ID, there is no
> general way to look up the reference except by enumerating the resources
> of the stack (presuming the relevant stack is already known) and
> comparing reference IDs. Are we even guaranteed uniqueness?
>
Developer semantics don't map at all to user semantics, so when you
submit your hot-guide changes make sure you make no reference to
Resource.id.
Mike Spreitzer (mike-spreitzer) wrote : | #20 |
I agree with switching the terminology to "resource name" from what the Template Guide defines as "resource ID". I think there are two good reasons for this. One is that "resource name" is a more natural term for this concept. They are used like what are called names in many languages and formalisms. And witness the fact that we all use this term anyway. Another is that "resource ID" is problematic. Like it or not, anyone who looks at the source sees the class named Resource and sees that it has a field named "id". Of course users do not have to know everything developers do, but having users use a term that is confusing to developers adds an unnecessary degree of difficulty.
Mike Spreitzer (mike-spreitzer) wrote : | #21 |
The issue of whether to use the term "resource name" or "resource ID" is not actually where this bug started. This bug was opened because OS::Heat:
Zane Bitter (zaneb) wrote : | #22 |
> HARestarter looks for a resource based on resource ID, but the get_resource intrinsic returns the refid.
Ah, you're quite right. Sorry, my mistake.
This is a straight up bug in HARestarter. Instead of self._find_
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master) | #23 |
Fix proposed to branch: master
Review: https:/
Changed in heat: | |
assignee: | Steven Hardy (shardy) → Zane Bitter (zaneb) |
status: | Confirmed → In Progress |
Mike Spreitzer (mike-spreitzer) wrote : | #24 |
The discussion here has wandered through another topic, about terminology used in the documentation. I am not sure whether to incorporate that issue in this bug or open a distinct bug for it.
Steve Baker (steve-stevebaker) wrote : Re: [Bug 1381136] Re: TemplateResource RefId is wrong | #25 |
On 22/10/14 06:26, Mike Spreitzer wrote:
> The discussion here has wandered through another topic, about
> terminology used in the documentation. I am not sure whether to
> incorporate that issue in this bug or open a distinct bug for it.
>
Could you raise a bug here for the hot-guide?
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master) | #26 |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit c4dd7d70b5d1122
Author: Zane Bitter <email address hidden>
Date: Tue Oct 21 13:04:52 2014 -0400
Make HARestarter do a proper lookup of the RefID
HARestarter is a primitive prototype that is barely maintained. Its
implementation still dates to the era when the resource name, physical
resource ID and RefId were regularly used interchangably.
This change substitutes Stack.resource_
(incorrect) home-grown hack, so that at least the giant red flags will be
easy to grep for.
Change-Id: I9d81dfcd5d3428
Closes-Bug: #1381136
Changed in heat: | |
status: | In Progress → Fix Committed |
Changed in heat: | |
status: | Fix Committed → Fix Released |
Changed in heat: | |
milestone: | kilo-1 → 2015.1.0 |
The return value of get_resource depends on the resource implementation, what resource type is "foo" in this case?