AssertionError when creating a stack.

[Clint Byrum]

2013-06-17 19:48:11.531 31768 INFO heat.engine.resource [-] Validating SecurityGroup "InstanceSecurityGroup"
2013-06-17 19:48:11.531 31768 INFO heat.engine.resource [-] Validating Instance "Instance"
2013-06-17 19:48:11.532 31768 ERROR heat.engine.parser [-]
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser Traceback (most recent call last):
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/parser.py", line 249, in validate
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser result = res.validate()
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/resources/instance.py", line 406, in validate
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser res = super(Instance, self).validate()
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/resource.py", line 411, in validate
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return self.properties.validate()
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/properties.py", line 179, in validate
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser self[key]
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/properties.py", line 201, in __getitem__
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser value = self.resolve(self.data[key])
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/parser.py", line 546, in resolve_runtime_data
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return resolve_runtime_data(self.t, self.resources, snippet)
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/parser.py", line 580, in resolve_runtime_data
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser template.resolve_base64])
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/parser.py", line 589, in transform
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser data = t(data)
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/template.py", line 147, in resolve_resource_refs
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return _resolve(match_resource_ref, handle_resource_ref, s)
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/template.py", line 375, in _resolve
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return [recurse(s) for s in snippet]
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/template.py", line 366, in <lambda>
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser recurse = lambda s: _resolve(match, handle, s)
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/template.py", line 372, in _resolve
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return handle(recurse(v))
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/template.py", line 145, in handle_resource_ref
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return resources[arg].FnGetRefId()
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/resources/security_group.py", line 182, in FnGetRefId
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser return self.physical_resource_name()
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser File "/home/ubuntu/src/heat-hpcloud/heat/heat/engine/resource.py", line 401, in physical_resource_name
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser assert self.id is not None
2013-06-17 19:48:11.532 31768 TRACE heat.engine.parser AssertionError

[Clint Byrum (clint-fewbar)]

The template that produced the error above is:

https://github.com/SpamapS/heat-hpcloud/blob/master/templates/simple_single_instance_only.template

[Clint Byrum (clint-fewbar)]

commit 9e2ecf756815d69e5b7fe851f4c38fa4a23e9661
Author: Zane Bitter <email address hidden>
Date: Mon Jun 17 12:24:49 2013 +0200

    Use physical resource names with a short_id

    The short_id is a random, but stable, 12-character alphanumeric ID.

    This will allow two resources of the same name to exist concurrently. That
    is necessary for resources to be replaced during an update without
    interruption.

    Change-Id: If32904a47c6f8fb651e317ce91f2a04f6eadcc0b

This commit introduced the assertion, and seems to have caused the bug. May have to revert.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/33342

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/33414

[Zane Bitter (zaneb)]

So, the issue here is that a handful of resource types (including security groups) return their physical_resource_name as either their Ref value or from Fn::GetAtt. Most resources don't, which is why this didn't show up in testing :(

The fix is to make physical_resource_name() return None during validation (when the value is not available because resource is not stored in the database until it is created), as we now do for other data that is not available because the resource has not been created.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/33414
Committed: http://github.com/openstack/heat/commit/40c4b44d02717b2aa8f420ca516ce662d2a1b978
Submitter: Jenkins
Branch: master

commit 40c4b44d02717b2aa8f420ca516ce662d2a1b978
Author: Zane Bitter <email address hidden>
Date: Tue Jun 18 10:41:40 2013 +0200

    Return None for physical resource name during validation

    During template validation, the resources have not been stored in the
    database. Therefore any resources that return self.physical_resource_name()
    in either FnGetRef() or FnGetAtt() would cause assertions whenever either
    of those functions were called during validation. Instead, return None from
    physical_resource_name() in this phase, as we do for other data that is not
    available yet during validation.

    Fixes bug #1191931

    Change-Id: I1a5b17557ae1ca2600fb930e74eb5248c82e0326

[Zane Bitter (zaneb)]

A correction to the comment above: the issue only occurs with Ref. A similar issue with Fn::GetAtt was fixed for bug #1184794.

The only types affected by this issue are AWS::IAM::User, AWS::EC2::SecurityGroup and AWS::CloudWatch::Alarm.

*However* returning None is not actually the right thing to do for either Ref or Fn::GetAtt. It should work in the case of the template linked above because the property in question (SecurityGroups) is defined as a List, and no type checking is done on the list members. If the property were a string (or if types within the list were properly checked) then this would fail. I have opened bug #1192142 to track this issue.