Flatten containers configuration changes tracking under a single host path

[Bogdan Dobrelya]

I know users as of Pike keep asking it on mail lists,
like which paths to use in templates for bind-mounts? And we have blog posts and docs, but none of that really helps.

We should ditch the puppet-generated directories that had span through all of the t-h-t places around, for the simplicity and code maintenance reasons. There is a sort of extremely hard to catch/debug issues, like that is fixed here https://review.openstack.org/#/c/634649/2..3/docker/services/database/mysql.yaml. So a subtle and easy to make, highly error prone, typo may end up affecting a service badly. Really, let's just flatten all config files for a service foo into its /var/lib/config-data/foo and drop that confusing /var/lib/config-data/puppet-generated/foo. If one wants to debug things, make a copy of it and issue a diff command against it and the altered location.

Related bug 1788138

[Bogdan Dobrelya (bogdando)]

As jistr explained on IRC, "the reason was that in the config-data/foo there's a lot of files which we don't want copied into the containers. So we only look at files which changed in the duration of the puppet run, copy them into config-data/puppet-generated/foo and that then gets used for the containers."

But I hope we can *still* simplify that (and make the config dirs structure less confusing for users creating new templates of containerized services). For example, we could instead maintain full configuration of a service foo without the puppet-generated subset of it, under two paths named /var/lib/config-data/foo and /var/lib/config-data/foo.bak or *.1, or *.shadow. That would address the main issue: confusing paths to use for templates (https://review.openstack.org/#/c/634649/2..3/docker/services/database/mysql.yaml, right...). Assuming no one will use those .bak/.1/.shadow paths normally in templates or elsewhere, cuz that's internal needs only!

And then we could identify changed config files to be copied into containers by kolla extended start hook by simply running эdiff -nuaR /var/lib/config-data/foo /var/lib/config-data/foo.1 (or .bak/.shadow) or the like. And then calculating hash/crc for it just like we do currently by evaluating the "origin of time" mark, but using the diff outputs instead. I hope that could work as well.

[Bogdan Dobrelya (bogdando)]

whatever works for me, if I can put a "never use puppet-generated in tht files" into the YAML/pep checker in the end

[Bogdan Dobrelya (bogdando)]

some background from irc

mandre: the only cases where we're normally using /var/lib/config-data/<service> vs /var/lib/config-data/puppet-generated/<service> is when the container isn't started by kolla_start

[Bogdan Dobrelya (bogdando)]

so the current logic is: if the container is started by kolla_start, it must use files from puppet_generated.

My proposition is let's make all containers using kolla_start with a custom hook that somehow reworks the static puppet_generated paths into a dynamic paths created for a one-time startup source for containers. And the content that dynamic path gets is evaluated via 'diff -nuaR foo foo.shadow' executed against the *full* paths stored at the host's /var/lib/config-data path.