zopectl adduser crash on 2.11.0

[Emmanuel]

zopectl adduser crash: seems to use attributes removed in Zope 2.11:

AttributeError: 'module' object has no attribute '__ac_permissions__'

zopectl> adduser foo bar

Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "/opt/scodoc/2.11.0/lib/python/Zope2/Startup/run.py", line 31, in configure
    opts = _setconfig(configfile)
  File "/opt/scodoc/2.11.0/lib/python/Zope2/Startup/run.py", line 46, in _setconfig
    opts.realize(doc="Sorry, no option docs yet.", raise_getopt_errs=0)
  File "/opt/scodoc/2.11.0/lib/python/zdaemon/zdoptions.py", line 279, in realize
    self.load_schema()
  File "/opt/scodoc/2.11.0/lib/python/zdaemon/zdoptions.py", line 327, in load_schema
    self.schema = ZConfig.loadSchema(self.schemafile)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/loader.py", line 32, in loadSchema
    return SchemaLoader().loadURL(url)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/loader.py", line 66, in loadURL
    return self.loadResource(r)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/loader.py", line 187, in loadResource
    schema = ZConfig.schema.parseResource(resource, self)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/schema.py", line 27, in parseResource
    xml.sax.parse(resource.file, parser)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py", line 31, in parse
    parser.parse(filename_or_stream)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed
    self._parser.Parse(data, isFinal)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 312, in start_element
    self._cont_handler.startElement(name, AttributesImpl(attrs))
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/schema.py", line 99, in startElement
    getattr(self, "start_" + name)(attrs)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/schema.py", line 475, in start_schema
    keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/schema.py", line 201, in get_sect_typeinfo
    datatype = self.get_datatype(attrs, "datatype", "null", base)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/schema.py", line 194, in get_datatype
    return self._registry.get(dtname)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/datatypes.py", line 398, in get
    t = self.search(name)
  File "/opt/scodoc/2.11.0/lib/python/ZConfig/datatypes.py", line 423, in search
    package = __import__(n, g, g, component)
  File "/opt/scodoc/2.11.0/lib/python/Zope2/Startup/datatypes.py", line 22, in ?
    import OFS.Uninstalled
  File "/opt/scodoc/2.11.0/lib/python/OFS/Uninstalled.py", line 16, in ?
    import SimpleItem, Globals, Acquisition
  File "/opt/scodoc/2.11.0/lib/python/OFS/SimpleItem.py", line 26, in ?
    import AccessControl.Role, AccessControl.Owned, App.Common
  File "/opt/scodoc/2.11.0/lib/python/AccessControl/Role.py", line 19, in ?
    from Globals import DTMLFile, MessageDialog, Dictionary
  File "/opt/scodoc/2.11.0/lib/python/Globals/__init__.py", line 34, in ?
    from App.special_dtml import HTML, HTMLFile, DTMLFile
  File "/opt/scodoc/2.11.0/lib/python/App/special_dtml.py", line 73, in ?
    from Shared.DC.Scripts.Bindings import Bindings
  File "/opt/scodoc/2.11.0/lib/python/Shared/DC/Scripts/Bindings.py", line 354, in ?
    InitializeClass(Bindings)
  File "/opt/scodoc/2.11.0/lib/python/App/class_init.py", line 69, in default__class_init__
    AccessControl.Permission.registerPermissions(self.__ac_permissions__)
  File "/opt/scodoc/2.11.0/lib/python/AccessControl/Permission.py", line 132, in registerPermissions
    Products.__ac_permissions__=(
AttributeError: 'module' object has no attribute '__ac_permissions__'

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Emmanuel wrote:
> Public bug reported:
>
> zopectl adduser crash: seems to use attributes removed in Zope 2.11:
>
> AttributeError: 'module' object has no attribute '__ac_permissions__'

I can't reproduce this problem using the released tarball for 2.11::

 $ cd /tmp
 $ wget http://www.zope.org/Products/Zope/2.11.0/Zope-2.11.0-final.tar
 $ tar xzf Zope-2.11.0-final.tar
 $ cd Zope-2.11.0-final
 $ ./configure --with-python=/path/to/python2.4 && make inplace
 ...
 $ bin/mkzopeinstance.py -d /tmp/lp245649 -u dummy:dummy
 $ cd /tmp/lp245649
 $ bin/zopectl adduser realuser realpass

with no error. I can log in with those credentials after starting Zope.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIbuNC+gerLs4ltQ4RAu8xAJ9CLjHZggxoy69WCtW45EUnygXslQCbB5kH
mIfammeA1pyArCsEC1j1YTs=
=mosq
-----END PGP SIGNATURE-----

[Andreas Jung (ajung)]

Works also for me (MacOSX)

[Emmanuel (emmanuel-viennet)]

How to reproduce the bug : (install trying to follow step by step the instructiosn from Zope-2.11.0-final/doc/INSTALL.txt),
on a Debian 4.0 (etch).

Please note that Debian ships with Python 2.4.4, so ./configure says:

  An acceptable, but non-optimal Python version (2.4.4) was found at '/usr/bin/python'.
  But consider installing version '2.4.5' before running 'make'.

(I stayed with 2.4.4).

cd /tmp
tar xzf Zope-2.11.0-final.tar.gz
cd Zope-2.11.0-final

./configure --prefix=/tmp/zozo
mkdir /tmp/zozo
make install

/tmp/zozo/bin/mkzopeinstance.py
Directory: /tmp/instance
Please choose a username and password for the initial user.
These will be the credentials you use to initially manage
your new Zope instance.

Username: root
Password:
Verify password:

cd /tmp/instance/
bin/zopectl adduser admin mysecretpassword

and the bug:

/tmp/zozo/lib/python/ZPublisher/Iterators.py:1: DeprecationWarning: The Interface package is deprecated and will be removed in Zope 2.12. Use zope.interface instead.
  from Interface import Interface
Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "/tmp/zozo/lib/python/Zope2/Startup/run.py", line 31, in configure
    opts = _setconfig(configfile)
  File "/tmp/zozo/lib/python/Zope2/Startup/run.py", line 46, in _setconfig
    opts.realize(doc="Sorry, no option docs yet.", raise_getopt_errs=0)
  File "/tmp/zozo/lib/python/zdaemon/zdoptions.py", line 279, in realize
    self.load_schema()
  File "/tmp/zozo/lib/python/zdaemon/zdoptions.py", line 327, in load_schema
    self.schema = ZConfig.loadSchema(self.schemafile)
  File "/tmp/zozo/lib/python/ZConfig/loader.py", line 32, in loadSchema
    return SchemaLoader().loadURL(url)
  File "/tmp/zozo/lib/python/ZConfig/loader.py", line 66, in loadURL
    return self.loadResource(r)
  File "/tmp/zozo/lib/python/ZConfig/loader.py", line 187, in loadResource
    schema = ZConfig.schema.parseResource(resource, self)
  File "/tmp/zozo/lib/python/ZConfig/schema.py", line 27, in parseResource
    xml.sax.parse(resource.file, parser)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py", line 31, in parse
    parser.parse(filename_or_stream)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed
    self._parser.Parse(data, isFinal)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 312, in start_element
    self._cont_handler.startElement(name, AttributesImpl(attrs))
  File "/tmp/zozo/lib/python/ZConfig/schema.py", line 99, in startElement
    getattr(self, "start_" + name)(attrs)
  File "/tmp/zozo/lib/python/ZConfig/schema.py", line 475, in start_schema
    keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
  File "/tmp/zozo/lib/python/ZConfig/schema.py", line 201, in get_sect_typeinfo
    datatype = self.get_datatype(attrs, "datatype", "null", base)
  File "/tmp/zozo/lib/python/ZConfig/schema.py", line 194, in get_datatype...

[Andreas Jung (ajung)]

I can reproduce the problem on Etch with the default Python. I can not reproduce it with a clean Python source build.
Appearently Debian's Python is again a bit different from the standard Python. Not sure if we shall count this as a Zope bug since the
inofficial recommendation is and was always: use a clean Python source build.

[Emmanuel (emmanuel-viennet)]

many thanks.

So to distribute a software (based on a Zope product we developped), we have to distribute a package with:
 - Zope
 - python
 - our software

quite large, no ?

(But Debian python works fine with many other applications, you should make sure that it's some sort of Zope bug ;-) )

[Andreas Jung (ajung)]

*Shrug* - for serious deployments you usually install your own Python. At least that's best-practice for years within the Zope world. At least this is not a big issue for my view.

[Emmanuel (emmanuel-viennet)]

I strongly disagree, as this policy implies that I should maintain (and track security updates) for a lot of third party packages that we need (a *lot* of python libraries python-xxxx) maintained by Debian (or other distro). Repackaging and updating all that stuff is simply not an option and would be overkill.

We'll try to use the Zope packaged by Debian (but they're still with Zope2.9).

[Philipp von Weitershausen (philikon)]

I think we should care about the standard Debian Python, people will
be trying out Zope with it and if they fail, they'll give it up.
Whether or not they'll do deployment with a self-compiled Python is
another question.

Regarding the issue, it seems to me that you already have a
Products.xxx package on your PYTHONPATH *before* the Zope 2 code is
executed. You see, 'Products' is a namespace package so it's actually
forbidden to put any code in there because with namespace packages you
never know which of the many Products/__init__.py will be loaded first.

Zope, however, violates that rule by having an __ac_permissions__
variable in its Products/__init__.py and the code that breaks for you
expects this variable to be there already:

         Products.__ac_permissions__=(
             Products.__ac_permissions__+((perm,(),default),))

I think we should get rid of the (rather useless) definition of
__ac_permissions__ in Products/__init__.py and change all other code
that uses __ac_permissions__ to accept it not being there (in which
case it would simply use an empty tuple).

Emmanuel, could you try to subsitute the line above with

         ac_permissions = getattr(Products, '__ac_permissions__', ())
         Products.__ac_permissions__=(ac_permissions+((perm,
(),default),))

and check if it works?

[Lennart Regebro (regebro-gmail)]

I don't see why not using a system python would mean you need to track security changes. The custom python would use the same libraries as the system python....

But I agree, Zope should work out of the box with the system Python. But serious installations should have a separate Python for the Zope instances. Trying to use system Zope has always been a recipy for problems, so I would try to avoid that.

[Philipp von Weitershausen (philikon)]

El 5 Jul 2008, a las 11:13 , Lennart Regebro escribió:
> I don't see why not using a system python would mean you need to track
> security changes.

It's because we violate the rules for namespace packages. See my reply
from 20 minutes ago.

> The custom python would use the same libraries as the system
> python....

How do you know that? The system python might have stuff installed on
site-packages.

[Emmanuel (emmanuel-viennet)]

I'm glad to see so many helpful people here: thanks !

Philipp: your fix works !

Lennart & Andreas: so you recommend to distribute my own version of python, and try to make it use other installed packages (like python-psycopg, python-reportlab, python-numeric, python-crack etc etc...) ?
I'd like to build a .deb package with our software: it would then contains: Zope, Python 2.4.x, our Zope products.

Thanks.

[Andreas Jung (ajung)]

We have zc.buildout for isolated Zope environments.

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Philipp von Weitershausen wrote:
> I think we should care about the standard Debian Python, people will
> be trying out Zope with it and if they fail, they'll give it up.
> Whether or not they'll do deployment with a self-compiled Python is
> another question.
>
> Regarding the issue, it seems to me that you already have a
> Products.xxx package on your PYTHONPATH *before* the Zope 2 code is
> executed. You see, 'Products' is a namespace package so it's actually
> forbidden to put any code in there because with namespace packages you
> never know which of the many Products/__init__.py will be loaded first.
>
> Zope, however, violates that rule by having an __ac_permissions__
> variable in its Products/__init__.py and the code that breaks for you
> expects this variable to be there already:
>
> Products.__ac_permissions__=(
> Products.__ac_permissions__+((perm,(),default),))
>
> I think we should get rid of the (rather useless) definition of
> __ac_permissions__ in Products/__init__.py and change all other code
> that uses __ac_permissions__ to accept it not being there (in which
> case it would simply use an empty tuple).
>
> Emmanuel, could you try to subsitute the line above with
>
> ac_permissions = getattr(Products, '__ac_permissions__', ())
> Products.__ac_permissions__=(ac_permissions+((perm,
> (),default),))
>
> and check if it works?

I think we should fix this on the trunk (making Products a proper
namespace package) but not worry about a backport: it is *definitely*
outside the box to run Zope from a Python which already has bits of
another Zope installed (as appears to be the case here).

I will check in the attached as soon as I have finished running the tests.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIb5PW+gerLs4ltQ4RAko9AJ9rLeNUWSEV1vFBWTBYttI053kdUQCbBWKO
SRpl/SPP2jr72eDT+YFj6u0=
=b13f
-----END PGP SIGNATURE-----

[Tres Seaver (tseaver)]

Fix now checked in on the 2.10 branch:

 http://svn.zope.org/Zope/branches/2.10/?rev=88041&view=rev

the 2.11 branch:

 http://svn.zope.org/Zope/branches/2.11/?rev=88040&view=rev

and the trunk:

 http://svn.zope.org/Zope/trunk/?rev=88039&view=rev