Comment 7 for bug 407458

Revision history for this message
Uli Fouquet (uli-gnufix) wrote : Re: [Bug 407458] Re: Can not use buildout to configure BLOB storage

> Is there a reason to not just make this the default configuration?

Only the don't-change-too-much-at-once policy.

> I created a new, no-site-packages virtualenv and new grokproject,
> changed the zope.conf.in to use the BLOB storage configuration, and it
> runs with no problems and the tests all pass.

Fine, thanks for looking into this :-)

> You do get an extra line when you start the project. (Maybe this could
> be suppressed by specifying the layout.):
>
> INFO [ZODB.blob] (26577) Blob directory `=
> /opt/grok/atest/blobtest/parts/data/Data.fs/blobs` has layout marker
> set. Selected `bushy` layout.

Is there a way to suppress this message?

Beside this, this looks erraneous to me: the path should be data/blobs,
not data/Data.fs/blobs, right? Maybe we have to switch to something like
${buildout:parts-directory}/blobs.

We _could_ also set the blobs-dir to
${buildout:parts-directory}/data/blobs, but we cannot rely, that this
directory (data/) really exists. This depends on the name of the [data]
section in buildout.cfg.

> I'm not saying that this is the way to go - I'm just curious what the
> down-side would be to having it able to "just work" when you add the
> necessary packages to setup.py and rerun buildout.

Sure, we can make this setting the (enabled) default. As you certainly
used blobs more often you can better tell what downsides could result
from that.

> It is unfortunate that the BLOB storage configuration is entangled with
> the ZODB file storage configuration in the first place.
>
> It might actually be nicer if all the database and storage related
> configuration occurred in the paster templates as it is likely that one
> would have different paths for development and deployment anyway.

That's currently not possible as (AFAIK) there is no way to forward
settings from paster to zope and vice versa. And we're yet bound to
ZConfig configurations (this XML-like format) for basic Zope
configuration. I think someone was working on support for a real WSGI
pipeline in Zope and I started to build a ZConfig parser that allows
normal Python OptionParser syntax like in paster. But both pieces are
quite complex. To have this would really be nice and main configuration
parts could go into deploy.ini then.

> That being said, I am OK with the solution as it stands right now.
>
> I will try grokproject and buildout from the trunk code later today. I
> did my tests with 1.0a4 and did have some problems with the changes
> made in changeset 9926 (the double $$ in front of variables).
> Buildout passed one of the $'s into the output files, so the paths
> were broken like:
>
> path $/opt/grok/atest/blobtest/parts/data/Data.fs
>
> Maybe this is fixed somewhere else in the trunk.

Hm, this should not happen. The grokproject templates contain double
dollars to quote the dollar character. When grokproject runs, it turns
$${buildout:directory} into ${buildout:directory} which should be
written into the etc/ template files.

Then, when you run buildout of the generated project,
${buildout:directory} turns into a real path.

I normally check this by running 'buildout' in the checked-out
grokproject branch, then run bin/grokproject to create a sample project
and in this sample project everything looks okay here (except the wrong
path to the blobs dir, as mentioned above).

Well, I checked in a version with blobs enabled by default and the
blobs-dir set to parts/blobs/.