Tech Debt: Consider nested stacks instead of generated templates

Bug #1501045 reported by Adrian Otto on 2015-09-29
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Magnum
Wishlist
Unassigned

Bug Description

Excerpt from: https://review.openstack.org/211771

Astute remarks from Steven Hardy (10:04 AM, Patch Set 5):

"So, I'll refrain from voting since I'm not directly involved with Magnum, but hopefully I can share some experiences as I think the Heat community (and particularly the TripleO community) have been down this path before, and settled on a more pure-heat model which uses the built-in composition features of heat instead of external templating.

Some time back, TripleO had an external templating tool, called merge.py, it was a python script which looked for tokens in a heat template, and generated a version with predetermined substitutions. It was used to substitute different implementations and maintain support for multiple environments, and worked not unlike a poor-mans jinja templating engine.

We found that was a difficult approach to manage long-term, because you ended up with one giant and unmaintainable template, with multiple not-immediately obvious couplings to the various templated in pieces. It was also hard to test because each piece wasn't necessarily something that could be instantiated and validated directly via Heat (so you had to stand up the giant stack and cross your fingers).

The model we've settled on now isn't perfect either, but it's more of a pure HOT approach, which makes heavy use of the heat nested stack "provider resource" interface, where you can build a tree of nested stacks (which can all potentially be individually instantiated and validated), with reusable chunks of logic each defined in a separate HOT template.

Here are some links which hopefully show how this works - you might want to consider looking into it before you get too far down the path of implementing a superset DSL w/jinja, as it could potentially work for Magnum too AFAICS.

http://docs.openstack.org/developer/heat/template_guide/composition.html

https://www.youtube.com/watch?v=fw0JhywwA1E

http://hardysteven.blogspot.co.uk/2015/05/tripleo-heat-templates-part-1-roles-and.html

I'd welcome further discussion/collaboration, if needed we could look at adding functionality to Heat to support the Magnum use-case, or aligning the TripleO and Magnum approaches so we can benefit from the "cluster deployment" pattern reuse between both - it seems like a potentially missed opportunity if this problem ends up solved in a totally implementation specific way :)"

My response:

"I really appreciate the comment from Steven Hardy. His suggested approach does sounds appropriate. If we do find it awkward to maintain the template generation (we change the templates a lot, so we should get this feedback soon) we should give this serious consideration.

My advice is to try Tom's proposed approach with an open tech debt ticket to come back at a later time and inquire with those who have needed to maintain the templates in the mean time, and take stock of what concerns they have, and if those justify the pursuit of re-organizing the templates into nested stacks."

This ticket will serve as our placeholder to revisit this as a potential point of optimization to drive up the maintainability of magnum.

Adrian Otto (aotto) on 2015-09-29
tags: added: tech-debt
Adrian Otto (aotto) on 2015-11-24
Changed in magnum:
milestone: none → mitaka-1
Changed in magnum:
status: New → Won't Fix
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers