Axiom should use stdlib sqlite3 on PyPy 2.0 and beyond

[Laurens Van Houtven]

PyPy2.0 now contains cffi as a standard library addition, and also has a sqlite3 module that uses cffi. cffi sqlite3 is better than pysqlite2, but that code has not made it back into pysqlite2.

It seems that pysqlite2 changes so rarely these days it might be a good idea to use stdlib sqlite3 preferentially over pysqlite2; it'll be an easier patch, at least :)

[Tristan Seligmann (mithrandi)]

I don't think this matters on PyPy; does pysqlite2 even work there at all? On the other hand, on CPython, if pysqlite2 is installed, it seems likely that you want to use it in preference to the stdlib version.

[Laurens Van Houtven (lvh)]

Pretty sure it does:

======== SNIP SNIP ========
tmp ❯ virtualenv -p pypy pypy-pysqlite2 && source pypy-pysqlite2/bin/activate && pip install pysqlite
Running virtualenv with interpreter /usr/local/bin/pypy
New pypy executable in pypy-pysqlite2/bin/pypy
Installing setuptools............done.
Installing pip...............done.
Downloading/unpacking pysqlite
  Using download cache from /Users/lvh/.pip/cache/https%3A%2F%2Fpysqlite.googlecode.com%2Ffiles%2Fpysqlite-2.6.3.tar.gz
  Running setup.py egg_info for package pysqlite

Installing collected packages: pysqlite
  Running setup.py install for pysqlite
    building 'pysqlite2._sqlite' extension
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/module.c -o build/temp.macosx-10.8-x86_64-2.7/src/module.o
    src/module.c:132:10: warning: 'sqlite3_enable_shared_cache' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
        rc = sqlite3_enable_shared_cache(do_enable);
             ^
    /usr/include/sqlite3.h:4671:16: note: 'sqlite3_enable_shared_cache' declared here
    SQLITE_API int sqlite3_enable_shared_cache(int) __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0, __MAC_10_7, __IPHONE_2_0, __IPHONE_5_0);
                   ^
    1 warning generated.
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/connection.c -o build/temp.macosx-10.8-x86_64-2.7/src/connection.o
    src/connection.c:597:43: warning: passing 'const char **' to parameter of type 'char **' discards qualifiers in nested pointer types [-Wincompatible-pointer-types]
            if (PyObject_AsCharBuffer(py_val, &buffer, &buflen) != 0) {
                                              ^~~~~~~
    /Users/lvh/tmp/pypy-pysqlite2/include/pypy_decl.h:278:62: note: passing argument to parameter 'arg1' here
    PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *arg0, char **arg1, Py_ssize_t *arg2);
                                                                 ^
    1 warning generated.
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/cursor.c -o build/temp.macosx-10.8-x86_64-2.7/src/cursor.o
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/cache.c -o build/temp.macosx-10.8-x86_64-2.7/src/cache.o
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/microprotocols.c -o build/temp.macosx-10.8-x86_64-2.7/src/microprotocols.o
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/prepare_protocol.c -o build/temp.macosx-10.8-x86_64-2.7/src/prepare_protocol.o
    cc -arch x86_64 -fPIC -Wimplicit -DMODULE_NAME="pysqlite2.dbapi2" -DSQLITE_OMIT_LOAD_EXTENSION=1 -I/Users/lvh/tmp/pypy-pysqlite2/include -c src/statement.c -o build/temp.macosx-10.8-x86_64-2.7/src/sta...

[Jean-Paul Calderone (exarkun)]

I don't understand the issue. If you want to use stdlib sqlite3, then don't install pysqlite2?

[Laurens Van Houtven (lvh)]

That doesn't appear to work, at least not when I'm pip-installing PyPI-released version of axiom. Redacted output follows, unredacted output in gist: https://gist.github.com/lvh/5999347

tmp ❯ pypy --version && virtualenv pypyvenv -p pypy && source pypyvenv/bin/activate
Python 2.7.3 (5acfe049a5b0, May 21 2013, 13:47:22)
[PyPy 2.0.2 with GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]
Running virtualenv with interpreter /usr/local/bin/pypy
...
(pypyvenv)tmp ❯ pip install Twisted && pip install Epsilon && pip install Axiom
Downloading/unpacking Twisted
...
Downloading/unpacking Epsilon
...
Downloading/unpacking Axiom
(pypyvenv)tmp ❯ python
Python 2.7.3 (5acfe049a5b0, May 21 2013, 13:47:22)
[PyPy 2.0.2 with GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``it's Sunday, again''
>>>> from axiom import store
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/lvh/tmp/pypyvenv/site-packages/axiom/store.py", line 28, in <module>
    from axiom._pysqlite2 import Connection
  File "/Users/lvh/tmp/pypyvenv/site-packages/axiom/_pysqlite2.py", line 11, in <module>
    from pysqlite2 import dbapi2
ImportError: No module named pysqlite2
>>>> from axiom import version; version
Version('axiom', 0, 6, 0)

[Jean-Paul Calderone (exarkun)]

trunk@HEAD tries pysqlite2 and falls back to sqlite3. However this appears to be unreleased behavior, so when `pip install` finds the older version, the above failure is encountered.

Please try with trunk@HEAD!

[Laurens Van Houtven (lvh)]

So, apparently, the released version of Axiom (0.6.0) doesn't have this behavior, trunk does. This makes this ticket less important: people using PyPy are probably less likely to have pysqlite2 installed, plus, not installing pysqlite2 is easier than installing it :)