Directory layout `zeocache` selected for blob directory but marker found for layout `bushy`

[sureshvv]

1. Unable to use blobstorage

2. Have to manually edit zope.conf to remove blobstorage related lines in zeoclient

[sureshvv (sureshvv)]

This is in plone.recipe.zop2instance

[Hanno Schlichting (hannosch)]

Sounds like you started out with a direct blob/filestorage configuration and then switched to a ZEO client setup. Make sure to start in a new directory after such a change or at least remove var/blobstorage (make sure you know where your data is actually stored).

[sureshvv (sureshvv)]

Yes. I started with a non-zeo setup and moved to a zeo configuration. Can you be a little more descriptive about why this is bad and what I should do instead? Will help me write a blog post so someone else who runs into this knows what to do.

[Hanno Schlichting (hannosch)]

In a non-ZEO setup you will have a direct file storage under var/filestorage/Data.fs and blobs stored under var/blobstorage. The blobstorage can have various layouts as specified by a .layout file in it, by default using the "bushy" layout.

If you move to a ZEO setup, you will need to decide wether to use shared-blobs or use a blob zeocache. By default shared-blobs are disabled and thus a zeocache layout is used. The zeocache stores files differently, as it's optimized to hold only part of the data and not mirror the entire blob set.

If you simply update things in place, you might configure the var/blobstorage directory to be both the real blobstorage for the ZEO server and a zeocache for the instances. If you are lucky you only get a warning like you did. If you are unlucky you might overwrite all your actual blobs.

If I have to use ZEO without shared-blobs, I tend to specify the "blob-storage" option on the zope2instance part as "blob-storage = ${buildout:directory}/var/blobcache" to make its purpose clear.

[Hanno Schlichting (hannosch)]

Actually we can do something about this :) I changed the recipe in version 4.1.1, so that it uses var/blobcache as the default zeocache instead of var/blobstorage. Only if you turn on shared-blob, it defaults to var/blobstorage now. That should help in avoiding this error, if you aren't careful or didn't fully understand the various options yet.

[ToniMueller (support-oeko)]

Nope. I have tried both using the shared-blobstorage and not using it, but I always arrived at some kind of problem. Trying to add an Image (ATImage) with

shared-blob = off:

...
   * Module transaction._transaction, line 250, in savepoint
    * Module transaction._transaction, line 647, in __init__
    * Module ZODB.Connection, line 1128, in savepoint
    * Module ZODB.Connection, line 606, in _commit
    * Module ZODB.Connection, line 651, in _store_objects
    * Module ZODB.Connection, line 1293, in storeBlob
    * Module ZODB.Connection, line 1322, in _getBlobPath
    * Module ZODB.Connection, line 1335, in temporaryDirectory
    * Module ZEO.ClientStorage, line 1061, in temporaryDirectory

AttributeError: 'NoneType' object has no attribute 'temp_dir'

With shared-blob = on I also had to create the blobstorage-NN/tmp directories manually, as they were not automatically created. Trying to create an image to a ZEO-mounted Plone site would result in a traceback. After creating them, I receive this error:

Traceback (innermost last):

    * Module ZPublisher.Publish, line 135, in publish
    * Module Zope2.App.startup, line 291, in commit
    * Module transaction._manager, line 93, in commit
    * Module transaction._transaction, line 322, in commit
    * Module transaction._transaction, line 419, in _commitResources
    * Module ZODB.Connection, line 767, in tpc_vote
    * Module ZEO.ClientStorage, line 1068, in tpc_vote
    * Module ZEO.ClientStorage, line 905, in _check_serials

IOError: [Errno 2] No such file or directory: '/path/to/plone/buildout/var/blobstorage-04/tmp/0x00/0x00/0x00/0x00/0x00/0x00/0x1d/0x92/0x00cyWGX5.tmp-'

when trying to upload an image. The last part, and the name of the blob-storage dir, change, according to the specifics of the site used and the image I'm trying to upload, but the characteristics always stay the same. A find in the buildout dir shows this:
$ find var/blobstorage-04/tmp/
var/blobstorage-04/tmp/
var/blobstorage-04/tmp/0x00
var/blobstorage-04/tmp/0x00/0x00
var/blobstorage-04/tmp/0x00/0x00/0x00
var/blobstorage-04/tmp/0x00/0x00/0x00/0x00
var/blobstorage-04/tmp/0x00/0x00/0x00/0x00/0x00
var/blobstorage-04/tmp/0x00/0x00/0x00/0x00/0x00/0x00
var/blobstorage-04/tmp/0x00/0x00/0x00/0x00/0x00/0x00/0x1d
var/blobstorage-04/tmp/0x00/0x00/0x00/0x00/0x00/0x00/0x1d/0x92
$

And that's it... (bug in ZEO? in Plone?).

[ToniMueller (support-oeko)]

I forgot to add the buildout snippet to create the directories:

I had to use this:

[filestorage]
recipe = collective.recipe.filestorage
location = var/filestorage/Data%(fs_part_name)s.fs
zodb-mountpoint = /%(fs_part_name)s/mnt:/db/mnt
blob-storage = var/blobstorage-%(fs_part_name)s
parts =
    01
    02
    03
    04

I added this to the end of my 'parts' list:

[blobdirectories]
recipe = plone.recipe.command

command =
 cd ${buildout:directory}/var
 alldirs="`ls -d blobstorage*`"
 for i in $alldirs; do
     mkdir $i/tmp
 done

This then makes all the 'blobstorage*/tmp' directories that are required for uploading images.

[Hanno Schlichting (hannosch)]

@Toni: Your issue is with collective.recipe.filestorage not plone.recipe.zope2instance as the initial report was.

Please open a new issue, as your's is probably unrelated in terms of the problematic code. It would also help to see the generated zope.conf and zeo.conf files inside the parts directories.