python-pscycopg2 segfaults on innocent operator

[Eugene Morozov]

Binary package hint: python-psycopg2

This python program crashes immediately:

import psycopg2
conn = psycopg2.connect("dbname=jmv")
cur = conn.cursor()
cur.execute("SET TIME ZONE %s", ['America/Chicago'])

This prevents django 0.96 from working.

[Eugene Morozov (emorozov)]

GDB stacktrace:

(gdb) run /tmp/crash.py
Starting program: /usr/bin/python /tmp/crash.py
[Thread debugging using libthread_db enabled]
[New Thread 47324899014384 (LWP 4070)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47324899014384 (LWP 4070)]
call_function_tail (callable=<value optimized out>, args=0x2b0a00000000)
    at ../Objects/abstract.c:1893
1893 ../Objects/abstract.c: No such file or directory.
        in ../Objects/abstract.c
(gdb) bt
#0 call_function_tail (callable=<value optimized out>, args=0x2b0a00000000)
    at ../Objects/abstract.c:1893
#1 0x0000000000419a10 in PyObject_CallMethod (o=<value optimized out>,
    name=0x2b0ab0371e95 "getquoted", format=0x0) at ../Objects/abstract.c:1968
#2 0x00002b0ab036a7e3 in microprotocol_getquoted (obj=<value optimized out>,
    conn=0x2b0aafef2118) at psycopg/microprotocols.c:142
#3 0x00002b0ab036c29b in _mogrify (var=0x2b0aaeef18d0,
    fmt=<value optimized out>, conn=0x2b0aafef2118, new=0x7ffffbc04cd8)
    at psycopg/cursor_type.c:203
#4 0x00002b0ab036cfa8 in _psyco_curs_execute (self=0x2b0ab031f4e0,
    operation=0x2b0aaef3c570, vars=0x2b0aaeef18d0, async=0)
    at psycopg/cursor_type.c:300
#5 0x00002b0ab036d7ff in psyco_curs_execute (self=0x2b0ab031f4e0,
    args=<value optimized out>, kwargs=<value optimized out>)
    at psycopg/cursor_type.c:427
#6 0x000000000048875d in PyEval_EvalFrameEx (f=0x7b96a0,
    throwflag=<value optimized out>) at ../Python/ceval.c:3564
#7 0x000000000048973a in PyEval_EvalCodeEx (co=0x2b0aaef2c4e0,
    globals=<value optimized out>, locals=<value optimized out>, args=0x0,
    argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at ../Python/ceval.c:2831
#8 0x0000000000489782 in PyEval_EvalCode (co=0x2b0aafcb2960,
    globals=0x7cc120, locals=0x2b0ab034d170) at ../Python/ceval.c:494
#9 0x00000000004aae7e in PyRun_FileExFlags (fp=0x755010,
    filename=0x7ffffbc0594a "/tmp/crash.py", start=<value optimized out>,
    globals=0x778410, locals=0x778410, closeit=1, flags=0x7ffffbc051d0)
    at ../Python/pythonrun.c:1271
#10 0x00000000004ab110 in PyRun_SimpleFileExFlags (fp=0x755010,
    filename=0x7ffffbc0594a "/tmp/crash.py", closeit=1, flags=0x7ffffbc051d0)
    at ../Python/pythonrun.c:877
#11 0x00000000004146b5 in Py_Main (argc=<value optimized out>,
    argv=0x7ffffbc052f8) at ../Modules/main.c:523
#12 0x00002b0aaf9838e4 in __libc_start_main () from /lib/libc.so.6
#13 0x0000000000413bf9 in _start ()

[Eugene Morozov (emorozov)]

Seems that version packaged in Ubuntu is just unusable on 64 bit architectures. When I find workaround for one segfault, psycopg2 immediately segfaults in another place. There were 64 bit fixes in the psycopg2 repository recenty, probably they should be applied to Ubuntu package source immediately.

I think many widely used packages depend on psycopg2 (django, pylons and zope are most prominent examples). Please, fix the problem. Meanwhile, I'll try to compile newest version from svn.

[Lee Packham (lee-packham)]

I've experienced the same issue with one of our recently upgraded systems.

The fix I applied was to apt-get remove python-psycopg2 and compile psycopg2-2.0.6b2.tar.gz from the upstream site.

It appears this patch:

http://initd.org/tracker/psycopg/attachment/ticket/167/psycho2-64bit.patch

is the most important one (Bug 167 at psycopg's site).

Cheers.

[James Henstridge (jamesh)]

It seems that psycopg2 is more stable under Python 2.4, but may still have some problems.

The patch on that bug report does not appear to be the only size_t related change though: it is probably worth checking up on what else is in the 2.0.6 release.

[Lukas Fittl (lfittl)]

Upstream pre-release 2.0.6b2 package available for feisty at http://psycopg2.packages.fresh-ubuntu.com/ - should fix the 64 bit problems

Apparently the 2.0.6 release is mostly bug fixes, maybe this version can be uploaded to feisty-updates?

[Andy Boyko (andy-boyko)]

Problem confirmed with feisty, amd64, python 2.5, and python-psycopg 2.05: segmentation fault, using Django, traced to psycopg with gdb. Using the released version of psycopg 2.06, built from source, solves the problem.

[Eugene Morozov (emorozov)]

Sorry, this is a bit off-topic, but: I've found two serious bugs in Ubuntu related to x86-64 platform. Neither was fixed at all.
This is really discouraging. I think this Psycopg bug affects many people and they will badmouth Ubuntu or stop using it. And the fix is available upstream. Why not release an update?... Just don't understand it.

Another bug I've found was in previous release of ubuntu: popular IMAP server dovecot was loosing letters on x86-64 -- it was discussed and affirmed several times on dovecot mailing list, but the patch wasn't applied to the previous Ubuntu release...

[Ruben Fonseca (fonseka)]

that's why I moved to debian unstable :)

[Don Spaulding II (donspauldingii)]

Can we get the package uploaded to Gutsy backported into feisty please? This fixes a completely broken package for feisty amd64. Many python frameworks depend on psycopg2. Any help is appreciated.

Thanks,
Don Spaulding

[James Henstridge (jamesh)]

Note that the 2.0.5 release of psycopg2 has other serious problems, such as not reporting errors from commit().

So if an update is going to be made available for previous versions of Ubuntu, it would be better to backport the latest version rather than trying to backport just the 64-bit fix.

[John Dong (jdong)]

Backports team is not authorized to perform backports for the purpose of bugfixing.

[Ohad Lutzky (lutzky)]

This is a very serious issue for me... Psycopg2 2.0.5 is completely broken on amd64. No chance for a fix?

[Eugene Morozov (emorozov)]

I guess no. This is not a first issue of such kind that is not ever fixed. I'm currently making updates for such packages myself. Probably I should start an unofficial updates repository for Ubuntu. Or return to Debian.

[Ruben Fonseca (fonseka)]

On 16/09/2007, mediodormido <email address hidden> wrote:
> I guess no. This is not a first issue of such kind that is not ever
> fixed. I'm currently making updates for such packages myself. Probably I
> should start an unofficial updates repository for Ubuntu. Or return to
> Debian.

Believe me, that was the nicest thing I've ever done when I stared at
this bug some months ago :)

Ruben

--
Will work for bandwidth

[Ohad Lutzky (lutzky)]

ׂGood thing it's easy to compile. I trust this is not an issue in LTS versions?

On 9/16/07, Ruben Fonseca <email address hidden> wrote:
> On 16/09/2007, mediodormido <email address hidden> wrote:
> > I guess no. This is not a first issue of such kind that is not ever
> > fixed. I'm currently making updates for such packages myself. Probably I
> > should start an unofficial updates repository for Ubuntu. Or return to
> > Debian.
>
> Believe me, that was the nicest thing I've ever done when I stared at
> this bug some months ago :)
>
> Ruben
>
> --
> Will work for bandwidth
>
> --
> python-pscycopg2 segfaults on innocent operator
> https://bugs.launchpad.net/bugs/108067
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Man is the only animal that laughs and weeps, for he is the only
animal that is struck with the difference between what things are and
what they ought to be.
 - William Hazlitt

Ohad Lutzky

[Eugene Morozov (emorozov)]

> Good thing it's easy to compile. I trust this is not an issue in LTS versions?

There's at least a problem with dovecot IMAP server in the LTS version. It contains off-by-one bug or something similar that causes hangs when you save messages via IMAP (by moving email from one folder to another, for example, or when your IMAP client saves sent message on the server). The bug was closed as invalid because:
"Thanks taking the time to report this bug and helping to make Ubuntu better. However, I am closing it because the bug has been fixed in the latest development version of Ubuntu - the Gutsy Gibbon (and also Feisty as you suggested)."

It seems that Ubuntu maintainers don't care about anything. I've posted the patch from the dovecot mailing list, they only had to apply it and put a new package in updates.

What does it mean? I cannot understand. :( Bugs shouldn't be fixed in Ubuntu? Maintainers are too busy? Noone uses PostgreSQL from Python and dovecot IMAP/POP server? I must do something so that they pay attention to my bugs (make a donation or something)? Maybe I'm crazy and do not understand something about bugs, bug reports and updates?

[Mark Loeser (halcy0n)]

This is a very bad bug and really needs to be fixed. The package is completely useless on 64bit machines. Is there any possibility this will ever be fixed so I don't have to build my own version of this?

[Matthias Klose (doko)]

> The patch on that bug report does not appear to be the only size_t related change though:
> it is probably worth checking up on what else is in the 2.0.6 release.

did somebody start this?

[Eugene Morozov (emorozov)]

Oh, just a year passed and someone noticed that there's totally broken package. And even patch exists. LOL :)
Sorry, couldn't resist. You can remove my message.

[James Henstridge (jamesh)]

Matthias: I believe that the 2.0.6 release fixes the 64-bit compatibility problem this bug was about, so the bug can probably be closed. It'd be nice if we had 2.0.7 for Hardy, but I haven't managed to get Federico to make the release yet :(

[James Henstridge (jamesh)]

Sorry, I misread this bug. I believe it is still an issue on Feisty. A backport or update to 2.0.6 is still needed to fix this bug in that release.

[LumpyCustard (orangelumpycustard)]

Please could someone mark this as Won't Fix for Feisty?

[Hew (hew)]

Ubuntu Feisty Fawn is no longer supported, so a SRU will not be issued for this release. Marking Feisty as Won't Fix.