repozo should be in skel

[Justin Alan Ryan]

When I create a production zope instance, I always copy the 'repozo.py' file from $ZOPE_HOME into bin/ and use the 'runzope' script as a template for a 'repozo' shell script which similarly sets all of the instance's Zope environment, so that I can easily run repozo backups in cron.

It would be handy for mkzopeinstance util to do this for me, as it does with zopectl and runzope. I've heard many people complain about this but couldn't find an issue in the collector. I may even create a patch myself and attach it here, but thought it would be best to file an issue first. If there is some reason why this wouldn't be accepted into Zope, I'd like to know. :)

Cheers!

-=JR

[Chris McDonough (chrism-plope)]

+1, please provide a patch if possible...

[Justin Alan Ryan (justizin)]

It's on my short list.

[Justin Alan Ryan (justizin)]

Uploaded: runzope.in

Okay, well, I'm halfway there. It's easy for normal zope instances, but the zeo creation code doesn't have $ZOPE_HOME at all. Attached is 'runzope.in', which, if added to $ZOPE_SRC/skel/bin, will cause a runzope shell script to be created with each new instance, pointing to $ZOPE_HOME/bin/repozo.py, which is put into place by make install since at least Zope 2.9, if not earlier versions.

I'm still working on a patch to mkzeoinst.py, it's a pain. I wonder if there is a poignant reason for the ZEO skel to be so much more rudimentary than normal zope instances. The implementation is a bit clever, but the dichotomy is a bit confusing IMO.

That said, I can also see why it would be a PITA to come up with a method that works for both, but it seems like a problem that only needs to be solved once.

Anyway, ehm, I can't attach runzope.in and a patch to mkzeoinst.py at the same time anyway, so I'll update again with that bit when I get around to it, just need to do an os.path.split, slice, and os.path.join on zodb3_home.

[Chris McDonough (chrism-plope)]

OK, two things:

- It would be most effective if you worked against a checkout of the Zope trunk
  (see http://www.zope.org/DevHome/CVS/ReadOnlyAccess) and uploaded the diff
  generated by "cvs diff" against the top-level directory. This way you can
  change multiple files and we can apply them as necessary.

- I think you wanted to name this file "repozo.in"?

- Is it intentional that you left the definitions of "ZOPE_HOME", "CONFIG_FILE", and
  "ZODB3_HOME", "INSTANCE_HOME" and "SOFTWARE_HOME" in the shell script? Are
  these used by repozo? If you remove them, does repozo still work? If so,
  it would probably be good to remove them and just declare only as many variables
  as necessary to run repozo successfully.

- The reason that ZEO instance creation is more rudimentary than Zope
   instance creation is that different sets of people worked on each. It's likely you
   can make changes to ZEO instance creation as necessary to support what you
   want it to do without anybody complaining.

[Justin Alan Ryan (justizin)]

Couple of things back from me:

  (a) Yes, I meant to call this repozo.in

  (b) I forgot that I need to 'svn add' to generate a diff when adding new files, I haven't slept much this week.

  (c) This version of the script works with both zope and ZEO instances, and repozo needs most of these variables to work in one or the other situation. Since I have to split the zeo and zope scripts, yeh, I'll clean that up a bit.

I'll throw something nicer tomorrow.

Cheers.

[Justin Alan Ryan (justizin)]

Uploaded: repozo.diff

Alright here is a diff against trunk. It's possible repozo.in could shrink a bit, but in both cases I wanted to make sure that repozo has all of the environment that Zope has, which seems wise. Toward the same end, each script is based on its' corresponding run script, i.e. runzope or runzeo.

This is tested and working for me with Zope trunk, python 2.4.3, linux.

[Justin Alan Ryan (justizin)]

Any feedback or movement on this ? I'm doing this in a buildout recipe right now, but it'd be nice to see this installing into zeo and zope instances by default.

[Benji York (benji)]

Justin Ryan wrote:
> Any feedback or movement on this ? I'm doing this in a buildout recipe
> right now, but it'd be nice to see this installing into zeo and zope
> instances by default.

mkzopeinstance isn't used much any more. I suspect the buildout recipe
that creates scripts for you would be a good way for you to get what you
want... if only I remembered what it's name was or how to use it.

[Hanno Schlichting (hannosch)]

These days you install ZEO either as an egg or via a buildout recipe. In both cases a repozo script will be created, which doesn't need any adjustments for Zope2.

[Justin Alan Ryan (justizin)]

as long as releases are being cut with mkzeoinstance scripts, they should behave properly. I can give the patch another look for some cleanup, it has been a while.

If this is invalid, a patch should be committed which removes mkzeoinstance, IMO.

[Hanno Schlichting (hannosch)]

mkzeoinstance lives in the zope.mkzeoinstance package for all supported versions of Zope2 (2.12.5+ that is). But the script doesn't have to do anything anymore. Installing the ZODB egg in one way or another will create the repozo script anyways.

[Justin Alan Ryan (justizin)]

I'm just saying, as long as there's a mkzeoinstance script, I think it should create a repoze script within that instance.

When using virtualenv, buildout, or possibly system-wide, the egg totally is fine. From time to time, it can be handy to install zeo by itself. Obviously, a special buildout or handcrafted virtualenv can be created for dedicated zeo servers, I just don't see why mkzeoinstance wouldn't do it, or why there would be opposition to me putting forth the effort.

Maybe this is invalid for Zope2, but appropriate for zope.mkzeoinstance? If the package becomes deprecated entirely, then it's irrelevant, but otherwise, at least the wishlist status you set a week or two ago seems reasonable.

This just doesn't seem driven by immense decisiveness about the future roadmap, it seems driven by keeping the number of open bugs low. There is still a lot of code in the wild depending on older versions of Zope2 which could accept this, as well.

It's not a huge pain, but I don't think it's an invalid concern, entirely. I have at least twice in the past 18 months had dedicated zeo servers which had to be brought up in a hurry and mkzeoinstance can be handy. It's not killing me to manually create the script as I decided above, and the collective.recipe.backup better handles multiple ZODBs, but it's not a worthless functionality, entirely.

[Justin Alan Ryan (justizin)]

Er sorry, repozo script, not repoze. :)

[Tres Seaver (tseaver)]

Installing zope.mkzeoinstance gets ZODB installed as a dependency, and
therefore gets the repozo script installed into the same directory as
mkzeoinstance::

  $ /opt/Python-2.6.5/bin/virtualenv --no-site-packages /tmp/lp143760
  New python executable in /tmp/lp143760/bin/python
  Installing setuptools............done.
  $ /tmp/lp143760/bin/easy_install zope.mkzeoinstance
  Searching for zope.mkzeoinstance
  Reading http://pypi.python.org/simple/zope.mkzeoinstance/
  ...
  Processing dependencies for zope.mkzeoinstance
  Searching for ZODB3>=3.9.4
  ...
  Best match: ZODB3 3.10.0a2
  Downloading http://pypi.python.org/packages/source/Z/ZODB3/ZODB3-3.10.0a2.tar.gz#md5=c0ad4c3a5a8cd04f2f578b145ab831be
  Processing ZODB3-3.10.0a2.tar.gz
  Running ZODB3-3.10.0a2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-N_cFcS/ZODB3-3.10.0a2/egg-dist-tmp-KaYxMT
  Adding ZODB3 3.10.0a2 to easy-install.pth file
  Installing fstail script to /tmp/lp143760/bin
  Installing zeopack script to /tmp/lp143760/bin
  Installing runzeo script to /tmp/lp143760/bin
  Installing zeopasswd script to /tmp/lp143760/bin
  Installing zeoctl script to /tmp/lp143760/bin
  Installing fsdump script to /tmp/lp143760/bin
  Installing fsrefs script to /tmp/lp143760/bin
  Installing repozo script to /tmp/lp143760/bin
  Installing fsoids script to /tmp/lp143760/bin

As with the other ZODB diagnostic / utility scripts, nothing about the
repozo script is unique to the instance created by mkzeoinstance: it
doesn't know about the config file, or about any filestorages which might
or might not be managed by that instance. There is therefore no reason to
try to copy any of those scripts into that instance.

[Justin Alan Ryan (justizin)]

Fair enough, as long as there is a repozo script on the system, you're pretty much OK. In at least one case, we've had to run separate versions of ZEO on the same box, but I'd like to leave that life behind.

I think it's valid, but I appear to be outvoted 2 to 1. We have bigger fish to fry. :)