grokproject generates hardcoded paths

[Uli Fouquet]

When running grokproject > 1.0 some generated configuration files in etc/ contain hardcoded paths.

This blocks developers from sharing code as they have no easy way to regenerate the configuration files with their local paths.

I think using a suitable buildout-recipe to (re)generate those files (especially zope.conf) would help.

[Martijn Faassen (faassen)]

I think it's odd to run paster with a generated file under parts. Now to add a WSGI middleware component people will have to change the files in etc and then re-run buildout and then use files under parts instead? Isn't that unlike any other paster-based setup where people edit them directly? Can't we avoid hardcoded paths instead of generating them?

How do modern Zope 3 and Plone buildouts do this?

[Michael Haubenwallner (d2m)]

we made the paths absolute because people tend to run their wsgi stack from arbitrary folders, thus breaking the relative path to the grok instance (typical error: zope.conf or site.conf not found)

if we could define a default how the process is started things would be easier.
zopeproject e.g. has the deploy.ini and *.conf files in the base folder of the instance.

[Uli Fouquet (uli-gnufix)]

Hm, we're talking about two steps here. Please let me recall, how the whole thing came up.

First we had some complaints on the list, that the generated config files in etc/ were not usable in shared SVN environments. A valid criticism from my point of view. The problem were the hardcoded paths in zope.conf and zdaemon.conf. I don't know a way to avoid these, if we want the built scripts to be runnable from any other directory.

The 'pure' paster parts (deploy.ini, debug.ini) have no such problem as they have the '%(here)s' variable (handy thing!). As far as I can see also modern Zope 3 and Plone buildouts always contain absolute paths.

So I introduced a recipe that generates the configuration files from templates and thus makes generated projects better shareable between different developers via SVN.

I hope we all agree, that this step makes sense.

Please note also, that you get additional configuration power by letting the config files be generated: you can refer to other recipes, their variables etc. which might become handy, although we currently don't use this feature.

Then - step two - Brandon Rhodes stepped up and complained about the - then generated - config files. They should be put into the parts/ directory he said, as this would be the expected place for buildout generated files. Maybe this is 'odd' maybe just buildout-like. Or both.

I see that this contradicts with expectations of paster-used users, that expect the config files to be in the 'root' or 'etc/', but we can only perform one 'usual behaviour' here by default: paster-like *or* buildout-like. (I might also add, that there was enough time to bring this point on the mailing list before the last release)

Anyway it is not difficult to change the setup so that the files are generated for example in etc/ again or even in the project root: just change the paths in the local buildout.cfg to point to etc/<filename>.conf and rerun buildout. If this is wanted by the majority of users, we could also make it the default in grokproject and ignore Brandons statements.

[Martijn Faassen (faassen)]

Sorry I haven't kept track of grokproject and this particular topic.
An alpha is always a good point of discussion, though.

I remember there being a discussion about hard coded paths within
python files, which I suggested we avoid. I hadn't realized until now
they also exist in config files.

One of the main points of using paster is familiarity across
platforms. We should therefore investigate patterns in pylons,
turbogears 2, patterns past documentation itself suggestions,
repoze.bfg, modern Plone and modern zopeproject setups. If there are
commonalities in approach we should look into adopting those. A
relative path approach might indeed be common as this is the general
way for applications to do such a thing; we'll see.

Does such an investigation makes sense? Or has it already taken place
and did I miss something?

[Uli Fouquet (uli-gnufix)]

Such investigation would indeed make sense from my point of view. I am also not very happy with that complicated path.

Michael could you look into Pylons and Turbogears? I could look into repoze.bfg and Plone/Zopeproject oriented setups.

Beside this I'll have a look into how difficult it would be to introduce a $here variable into standard Zope configurations. Having this we could skip the whole trouble and make Zope configurations more similar to paster at least.

[Uli Fouquet (uli-gnufix)]

Concerning configuration there are lots of things to discover :-)

Apart from the initial problem (where do we put 'deploy.ini') I dived into trying to understand the Zope configuration process better. Phew! Also had a peek into Pylons, TurboGears and repoze.bfg, and now there seem to be more questions left then answers (or too many answers) given.

One thing I stumbled upon and just for the logs:

Multiple setup files:

Our paster setup is certainly different from others as we have at least two configuration systems with different
config file parsers:

  1) paster parsed config files *.ini (paster seems to use an extended standard ConfigParser)

  2) ZConfig parsed zope.conf

  (3) site.zcml (I don't think we can do without it)

If we also take buildout.cfg into account, we have four different configuration files with different syntax and purpose.
That's much, maybe to much for an average user.

Non-Zope paster setups seem normally to be configured by 1 (in words: one) .ini file being in the project root or
nearby.

The repoze.bfg approach might be most attractive for us, as it also setups a complete Zope environment and
does not require a zope.conf. Instead you can for instance define your Data.fs location in a standard .ini file.

The general and already discussed shortcoming of .ini files is, that they do not support nested sections as XML-like zope.conf files (i.e.: ZConfig parsed files) usually require.

But if repoze.bfg can do without it, why shouldn't Grok be able to learn from it and provide this as well? An implementation might be possible with some extensions to zope.app.wsgi. This could also make everything more compatible (at least more similar-looking) with other Python frameworks.

Well, just some thoughts while going on. Maybe someone else has brilliant ideas to share concerning configuration ;-)

[Uli Fouquet (uli-gnufix)]

Jim Fulton already expressed clearer what I wanted to say some years ago:

  http://wiki.zope.org/zope3/UseConfigParserForHighLevelConfiguration

Does anyone know, why this was not implemented?