ZODB doesn't record attribute changes on Gutsy x64

[MFeif]

Binary package hint: python-zodb

This is a show-stopper: ZODB no longer records attribute changes. I have another system running Gutsy on i386, and have no problems there.

I'm attaching a code sample that gives the following output for me:

CODE:
====
from ZODB import FileStorage, DB
from BTrees.IOBTree import IOBTree
import transaction
from persistent import Persistent

class MyObj(Persistent):
    def __init__(self):
        self.my_attribute = "froon"

print "Making empty database"
storage = FileStorage.FileStorage('db.fs')
db = DB(storage)
conn = db.open()
root = conn.root()

print "\nstoring an object..."
root[1] = MyObj()

print "\ninspecting object..."
if hasattr(root[1], 'my_attribute') and root[1].my_attribute == 'froon':
    print "object still works..."
else:
    print "object is broken!"

print "\ncommitting transaction..."
transaction.commit()

print "\ninspecting object..."
if hasattr(root[1], 'my_attribute') and root[1].my_attribute == 'froon':
    print "object still works..."
else:
    print "object is broken!"

print "\nclosing up zodb...",
del root
conn.close()
del conn
db.close()
del db
storage.close()
del storage
print "done"

print "\nopening for another look..."
storage = FileStorage.FileStorage('db.fs')
db = DB(storage)
conn = db.open()
root = conn.root()
print "\ninspecting object..."
if hasattr(root[1], 'my_attribute') and root[1].my_attribute == 'froon':
    print "object still works..."
else:
    print "object is broken!"

transaction.abort()
del root
conn.close()
del conn
db.close()
del db
storage.close()
del storage
print "done"

#############################################

OUTPUT:
=====

~$ python test_zodb.py
Making empty database

storing an object...

inspecting object...
object still works...

committing transaction...

inspecting object...
object still works...

closing up zodb... done

opening for another look...

inspecting object...
object is broken!

I'm running an up-to-date Gutsy with "no funny stuff" on x86_64.

[Esteve Fernandez (esteve)]

This is a bug in the Persistent module, see ZODB bug #153316 (https://bugs.launchpad.net/zodb/+bug/153316) I've attached a patch to solve this issue, please apply against version 3.6.0-6 version of the python-zodb package.

[MFeif (matt-feifarek)]

I applied the patch to the debian source, which went without trouble, but I can't compile the package:

sudo dpkg-buildpackage

leads to the attached error message.

I noticed that the installed deb is also missing those utilities; thing like "runzeo" should land in /usr/bin/ but they don't... the package seems incomplete.

[Esteve Fernandez (esteve)]

My previous patch lacked some changes to MANIFEST and SETUP.cfg files, here's another one that fixes this. Please let me know if it works.

[Esteve Fernandez (esteve)]

BTW, you shouldn't do "sudo build-package", it's better to use fakeroot:

dpkg-buildpackage -rfakeroot (add -b if you only want to build binary packages)

also, if you want to capture errors, you should type "> file.txt 2>&1" after the command, e.g.:

dpkg-buildpackage -b -rfakeroot > /tmp/out.txt 2>&1

[MFeif (matt-feifarek)]

On Dec 1, 2007 7:38 AM, esteve <email address hidden> wrote:

> My previous patch lacked some changes to MANIFEST and SETUP.cfg files,
> here's another one that fixes this. Please let me know if it works.

Thanks!

It's getting closer, but still not quite. Here's the output (full output
attached; relevant portion here inline).

running install_egg_info
Writing /home/work/zodb-3.6.0
/debian/python-zodb/usr/local/lib/python2.5/site-packages/ZODB3-
3.6.0.egg-info
mv: cannot stat
`debian/python-zodb/usr/lib/python/site-packages/zope/testing/testrunner-ex':
No such file or directory
chmod: cannot access
`debian/python-zodb/usr/lib/python2.5/site-packages/ZConfig/tests/runtests.bat':
No such file or directory
chmod: cannot access
`debian/python-zodb/usr/lib/python2.5/site-packages/ZEO/zeoctl.py': No such
file or directory
chmod: cannot access
`debian/python-zodb/usr/lib/python2.5/site-packages/ZConfig/tests/runtests.py':
No such file or directory
chmod: cannot access
`debian/python-zodb/usr/lib/python2.5/site-packages/zdaemon/tests/nokill.py':
No such file or directory
mkdir: cannot create directory
`debian/python-zodb/usr/lib/python2.5/site-packages/ZODB/utilities': No such
file or directory
mv: cannot stat `debian/python-zodb/usr/bin/*': No such file or directory
rmdir: debian/python-zodb/usr/bin: No such file or directory
make: *** [install-stamp] Error 1

It's similar to what I was saying (somewhere? here?) that some of the ZODB
stuff seems to be missing in the Deb... testing stuff, the command line
utilities like runzeo... Maybe there's another source package in the
repository to supplement "python-zodb"? I did a build-deps, but maybe the
dependencies aren't correct.

Thanks!

[MFeif (matt-feifarek)]

On Dec 1, 2007 2:37 PM, Matt Feifarek <email address hidden> wrote:

> It's similar to what I was saying (somewhere? here?) that some of the ZODB
> stuff seems to be missing in the Deb... testing stuff, the command line
> utilities like runzeo... Maybe there's another source package in the
> repository to supplement "python-zodb"? I did a build-deps, but maybe the
> dependencies aren't correct.
>

One more thing: I notice when inspecting the tree after a failed compile
that
~/zodb-3.6.0/debian/python-zodb/usr/lib
contains python2.4 but not 2.5... and my /usr/bin/python -> python2.5

Why is this package creating 2.4 stuff, when I'm not (intentionally) running
2.4?

[MFeif (matt-feifarek)]

I guess the email doesn't snarf attachments.

[Esteve Fernandez (esteve)]

There's something weird, I noticed that zeoctl.py is installed in /home/work/zodb-3.6.0/debian/python-zodb/usr/local/lib/python2.5/site-packages/ZEO/zeoctl.py (notice the "local" directory), while in my computer it is installed in /tmp/zodb/zodb-3.6.0/debian/python-zodb/usr/lib/python2.5/site-packages/ZEO/zeoctl.py

What's the output of "echo $PYTHONPATH"? Are you using Divmod Combinator?

[MFeif (matt-feifarek)]

On Dec 2, 2007 4:56 AM, esteve <email address hidden> wrote:

> There's something weird, I noticed that zeoctl.py is installed in
> /home/work/zodb-3.6.0/debian/python-zodb/usr/local/lib/python2.5/site-
> packages/ZEO/zeoctl.py (notice the "local" directory), while in my
> computer it is installed in /tmp/zodb/zodb-3.6.0/debian/python-
> zodb/usr/lib/python2.5/site-packages/ZEO/zeoctl.py
>

Is it possible that changing my /usr/lib/python2.5/distultils/distutils.cfg
may be a culprit? I use lots of software that does lots of installing and
upgrading of eggs, and I didn't want easy_install messing with
Debian-supported packages, so I modified that file with this info:

[install]
install_lib = /usr/local/lib/python2.5/site-packages
install_scripts = /usr/local/bin

[easy_install]
site_dirs = /usr/local/lib/python2.5/site-packages
zip_ok = 0

I'm told by those smarter than me that this is the right thing to do on a
Debian system. This puts installed eggs in local; perhaps the ZODB compile
script is listening to that? I didn't mess with any of the source from the
repository, so it seems to me that it shouldn't.

What's the output of "echo $PYTHONPATH"? Are you using Divmod
> Combinator?
>

echo $PYTHONPATH gives me no output (well, looks like a single newline,
maybe). I'm not running Divmod Combinator.

[Christian Theune (ctheune)]

I can't reproduce this on AMD or Intel 64-bit with Python 2.5 and Python 2.4, tests are passing for me.

[Esteve Fernandez (esteve)]

Let me disagree, Christian. It's was known bug in ZODB, as I pointed (https://bugs.launchpad.net/zodb/+bug/153316) and you fixed it in revision r81758. BTW, tests don't pass on my machine:

esteve@beeblebrox:~$ uname -a
Linux beeblebrox 2.6.22-14-generic #1 SMP Sun Oct 14 21:45:15 GMT 2007 x86_64 GNU/Linux

esteve@beeblebrox:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz
stepping : 6
cpu MHz : 1000.000
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips : 3661.52
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

[Jim Fulton (jim-zope)]

On Dec 3, 2007, at 2:33 PM, Matt Feifarek wrote:

> On Dec 3, 2007 1:26 PM, Jim Fulton <email address hidden> wrote:
>
> > Well, it certainly still affects zodb 3.6 and 3.7. I filed a bug
> > against 3.6 in the Ubuntu repositories, and "esteve" is helping me
> > with that, but your patch against 3.8 doesn't fix 3.6 or 3.7.
>
>
> No, it doesn't because Python 2.5 isn't supported by those releases.
>
>
>
> Well, someone might want to tell the Debian package people then, as
> 3.6 is distributed compiled against 2.5, on both 32 and 64 bit
> platforms. Seems like 3.6 is in testing and unstable, too. I tried
> to find out who to inform, but the maintainer is called "Debian Zope
> Team". I'm hoping some of them might be listening ;-)

Me too.

>
> And also, it would be nice to update this page
> :
> http://pypi.python.org/pypi/ZODB3.
>
> As it looks to me like 3.7.2 is compiled for 2.5 (on Win) and no
> information on compatibility is supplied for the source. I suppose I
> erred in assuming inclusive compatibility.

Gaaaaah, I misspoke. 3.7 does claim to support Python 2.5

Jim

--
Jim Fulton
Zope Corporation

[Esteve Fernandez (esteve)]

On Dec 3, 2007 8:33 PM, Matt Feifarek <email address hidden> wrote:
> On Dec 3, 2007 1:26 PM, Jim Fulton <email address hidden> wrote:
>
> >
> >
> > > Well, it certainly still affects zodb 3.6 and 3.7. I filed a bug
> > > against 3.6 in the Ubuntu repositories, and "esteve" is helping me
> > > with that, but your patch against 3.8 doesn't fix 3.6 or 3.7.

Yes it does. I'm already running ZODB 3.6 in Ubuntu x86_64. The
testcase attached at bug to ZODB 3.6 works flawlessly:

esteve@beeblebrox:/tmp$ python test_int_pysize.py
Making empty database

storing an object...

inspecting object...
object still works...

committing transaction...

inspecting object...
object still works...

closing up zodb... done

opening for another look...

inspecting object...
object still works...
done

so there must be something weird in your environment, my patch (which
is just a backport fix to ZODB 3.6 from ZODB svn repo) applies cleanly
and fixes all porting issues in BTree and Persistent modules.

Cheers.

[Esteve Fernandez (esteve)]

On Dec 3, 2007 9:33 PM, Esteve Fernàndez <email address hidden> wrote:
> On Dec 3, 2007 8:33 PM, Matt Feifarek <email address hidden> wrote:
> > On Dec 3, 2007 1:26 PM, Jim Fulton <email address hidden> wrote:
> >
> > >
> > >
> > > > Well, it certainly still affects zodb 3.6 and 3.7. I filed a bug
> > > > against 3.6 in the Ubuntu repositories, and "esteve" is helping me
> > > > with that, but your patch against 3.8 doesn't fix 3.6 or 3.7.
>
> Yes it does. I'm already running ZODB 3.6 in Ubuntu x86_64. The
> testcase attached at bug to ZODB 3.6 works flawlessly:

Jeez, here's the URL to the bug report:

https://bugs.launchpad.net/ubuntu/+source/zodb/+bug/165287

Cheers.

[Esteve Fernandez (esteve)]

Christian, I don't know why you keep marking this bug as invalid. You were the one who committed the fix to ZODB 3.8 (https://bugs.edge.launchpad.net/zodb/+bug/153316), and this is exactly the same bug. This is the output of running the testcase that Matt attached against a stock python-zodb package:

Making empty database

storing an object...

inspecting object...
object still works...

committing transaction...

inspecting object...
object still works...

closing up zodb... done

opening for another look...

inspecting object...
object is broken!
done

and here's the output of the same testcase using a patched python-zodb package:

Making empty database

storing an object...

inspecting object...
object still works...

committing transaction...

inspecting object...
object still works...

closing up zodb... done

opening for another look...

inspecting object...
object still works...
done

[Esteve Fernandez (esteve)]

On Dec 3, 2007 10:09 PM, Matt Feifarek <email address hidden> wrote:
> On Dec 3, 2007 3:33 PM, Esteve Fernàndez <email address hidden> wrote:
>
> > so there must be something weird in your environment, my patch (which
> > is just a backport fix to ZODB 3.6 from ZODB svn repo) applies cleanly
> > and fixes all porting issues in BTree and Persistent modules.
> >
>
> How did you make that patch, and/or can I have it?
>
> If it's the earlier patch that you sent, yes it applies cleanly, but then I
> can't make a package.
>
> "Something weird" may indeed be up... but I don't know where to look.

It's here:

http://launchpadlibrarian.net/10681816/python-zodb-x86_64-2.patch

as I said, I'm already using it. I did nothing special to build the
python-zodb package, a pretty standard "dpkg-buildpackage -rfakeroot
-b" command should suffice.

[Esteve Fernandez (esteve)]

Matt's testcase spots this bug on my laptop (Intel Core 2 Duo, 64-bit).

[Esteve Fernandez (esteve)]

> I'm cc'ing James Troup, who is somehow involved in the packaging of ZODB for
> Ubuntu.

Fabio Tranchitella (python-zodb Debian maintainer) uploaded a new
python-zodb package just a few minutes ago, so expect it to be
available to a Debian FTP mirror near you. It fixes Debian bug #454196
(http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=454196), it's the
same bug that Matt reported in Ubuntu Launchpad.

Cheers.

[Christian Theune (ctheune)]

Am Montag, den 03.12.2007, 14:37 -0500 schrieb Jim Fulton:
> Gaaaaah, I misspoke. 3.7 does claim to support Python 2.5

Interesting. The README doesn't state that on the 3.7 branch. Where is
this claimed?

I'll port the 64-bit fixes if we really do support Python 2.5 in ZODB
3.7.

Christian

--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - <email address hidden> - phone [phone] -
fax [phone] - zope and plone consulting and development

[Jim Fulton (jim-zope)]

On Dec 5, 2007, at 2:30 AM, Christian Theune wrote:

> Am Montag, den 03.12.2007, 14:37 -0500 schrieb Jim Fulton:
>> Gaaaaah, I misspoke. 3.7 does claim to support Python 2.5
>
> Interesting. The README doesn't state that on the 3.7 branch. Where is
> this claimed?
>
> I'll port the 64-bit fixes if we really do support Python 2.5 in ZODB
> 3.7.

No. I misspoke the second time. I remember trying to get 3.7 working
with Python 2.5, but looking back at the announcement I see I gave
up. I got confused when I made the 3.7 Python 2.5 windows release at
someone's request.

3.7 doesn't support Python 2.5. Even though I don't like removing
eggs from PyPI, I'm going to remove the Python 2.5 egg for 3.7.

Jim

--
Jim Fulton
Zope Corporation

[Christian Theune (ctheune)]

Am Mittwoch, den 05.12.2007, 09:30 -0500 schrieb Jim Fulton:
> On Dec 5, 2007, at 2:30 AM, Christian Theune wrote:
>
> > Am Montag, den 03.12.2007, 14:37 -0500 schrieb Jim Fulton:
> >> Gaaaaah, I misspoke. 3.7 does claim to support Python 2.5
> >
> > Interesting. The README doesn't state that on the 3.7 branch. Where is
> > this claimed?
> >
> > I'll port the 64-bit fixes if we really do support Python 2.5 in ZODB
> > 3.7.
>
>
> No. I misspoke the second time. I remember trying to get 3.7 working
> with Python 2.5, but looking back at the announcement I see I gave
> up. I got confused when I made the 3.7 Python 2.5 windows release at
> someone's request.
>
> 3.7 doesn't support Python 2.5. Even though I don't like removing
> eggs from PyPI, I'm going to remove the Python 2.5 egg for 3.7.

Ok. I saw that the debian people released an updated package working
with Python 2.5 which is their choice but I'll invalidate this bug for
the ZODB project then.

Christian

--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - <email address hidden> - phone [phone] -
fax [phone] - zope and plone consulting and development

[Christian Theune (ctheune)]

ZODB 3.7 does not support Python 2.5 which makes this bug invalid.

[Christian Theune (ctheune)]

The general issue was solved with ZODB 3.8 and upwards which support Python 2.5

[Christian Theune (ctheune)]

Wrong status. WONTFIX is better.

[Matthias Klose (doko)]

fixed in 3.6.0 and later

[Chris Bainbridge (chris-bainbridge)]

I just ran into this bug in Jaunty which still ships python-zodb-3.6.0-4. From the Debian changelog here http://packages.debian.org/changelogs/pool/main/z/zodb/zodb_3.6.0-2/changelog it appears that the bug wasn't fixed till python-zodb-3.6.0-7 but then support for python 2.5 was dropped in -8. I'm not sure what the solution for Jaunty is or whether anyone cares, but at the moment this package http://launchpadlibrarian.net/25084058/python-zodb_3.6.0-4_amd64.deb does not work.

[Jim Fulton (jim-zope)]

On Sat, Oct 31, 2009 at 1:29 PM, Chris Bainbridge
<email address hidden> wrote:
> I just ran into this bug in Jaunty which still ships python-
> zodb-3.6.0-4. From the Debian changelog here
> http://packages.debian.org/changelogs/pool/main/z/zodb/zodb_3.6.0-2/changelog
> it appears that the bug wasn't fixed till python-zodb-3.6.0-7 but then
> support for python 2.5 was dropped in -8. I'm not sure what the solution
> for Jaunty is or whether anyone cares, but at the moment this package
> http://launchpadlibrarian.net/25084058/python-zodb_3.6.0-4_amd64.deb
> does not work.

I don't know what to say. The current supported version of ZODB is
3.9.3. I'm not doing any work on 3.6 and haven't for a very long
time.

Jim

--
Jim Fulton