YAML parser cuts duplicate resource names

Bug #1412498 reported by Removed by request
36
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Heat
Triaged
Medium
Sergey Kraynev

Bug Description

If there is more then one resource with the same name, the YAML parser returns only one of those resources and Heat is left with only a single one.
This should at least be documented.

description: updated
Revision history for this message
Steven Hardy (shardy) wrote :

AFAIK this will be very hard to correctly validate, as by the time the yaml parser has processed the template, the duplicate key is gone (it uses the last of the dupes AFAICT), so Heat can't detect there was ever more than one node with the same key.

So documenting "don't do that" is probably the best we can do unless anyone has any other ideas.

Changed in heat:
status: New → Triaged
importance: Undecided → Low
milestone: none → ongoing
Zane Bitter (zaneb)
Changed in heat:
importance: Low → Medium
Revision history for this message
Zane Bitter (zaneb) wrote :

Steve is correct, the YAML library we have in Python just doesn't offer this kind of validation, and there's no way for us to detect this kind of error from its output.

That said, it *is* possible (though a little ugly) to customise the type mapping that it uses (e.g. we could replace dict with a custom type that checks for duplicate key assignments). So we could probably implement a two-phase parse, where the first uses custom types to do extra verification and then the second does a standard parse to get the data we actually use.

Note that for folks using python-heatclient the YAML is parsed in the client, so we'd have to fix it there in order to actually have an effect for most people.

Bumping up the priority a little, because (admittedly without having dug deeply into it) I think this may be do-able, and discussion on IRC suggests that it's a very common user error.

Revision history for this message
Clint Byrum (clint-fewbar) wrote :

I feel like there's another Heat bug with this concern already but can't seem to find it.

http://pyyaml.org/ticket/128

So PyYAML is being silent where it should not. The spec is quite clear on this:

http://www.yaml.org/spec/1.2/spec.html#id2763754

Anyway, personally I think there are two ways to handle this that haven't been explored fully:

1) Fix PyYAML/libyaml - Might not be as hard as the age of the bug suggests.
2) Introduce a list-based system and have a 'name' field that can be duplicate but will be detected by Heat.

Revision history for this message
Sergey Kraynev (skraynev) wrote :

Amit, I have three separated bugs about this issue.

https://bugs.launchpad.net/pyyaml/+bug/1308400
https://bugs.launchpad.net/pyyaml/+bug/1308399
https://bugs.launchpad.net/pyyaml/+bug/1308401

I think we should mark one of them as duplicated. Also I repeat information above, that pyaml hasn't fix for this issue.

Changed in heat:
assignee: nobody → Sergey Kraynev (skraynev)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.