unit test failures: "Cursor isolation with 2 cursors, different connections, trans" "Interface exports the connect()-function"

[GuilhemBichot]

I branched it, installed it (like in README, went ok; this is Ubuntu 8.10 64-bit). I have this revision as latest:
  102 Geert Vanderkelen 2009-10-07
      revision-id:<email address hidden>
      lp:Bug#445002 - unit tests fail when run with python 2.4

Started a MySQL 5.1 server with:

cd mysql-test
./mtr --mem alias --start &

Back in the directory of connector/python, did
python unittests.py -H localhost -S /home/mysql_src/bzrrepos/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock -u root -D test
(and "root" has all privileges according to the setup done by mtr, and "test" is created by mtr),

and it failed like this:
<cut irrelevant things>
Cursor isolation with 2 cursors, different connections, trans. ... FAIL
<cut>
Interface exports the connect()-function ... ERROR
<cut>
======================================================================
ERROR: Interface exports the connect()-function
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_pep249.py", line 83, in test_connect
    db = myconn.connect()
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/__init__.py", line 42, in Connect
    return MySQL(*args, **kwargs)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/mysql.py", line 270, in __init__
    self.connect(*args, **kwargs)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/mysql.py", line 351, in connect
    self._open_connection()
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/mysql.py", line 92, in _open_connection
    database=self.database)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/protocol.py", line 92, in do_auth
    buf = self.conn.recv()[0]
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/connection.py", line 70, in recv
    self.protocol.is_error(buf)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/protocol.py", line 150, in is_error
    self._handle_error(buf)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/protocol.py", line 140, in _handle_error
    raise InterfaceError(err)
InterfaceError: 1045 (28000): Access denied for user ''@'localhost' (using password: NO)

======================================================================
FAIL: Cursor isolation with 2 cursors, different connections, trans.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_pep249.py", line 377, in test_isolation2
    self._isolation_test(self.db,db2,'InnoDB')
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_pep249.py", line 362, in _isolation_test
    self.assertEqual(None, result)
AssertionError: None != (1L, u'myconnpy')

----------------------------------------------------------------------
Ran 138 tests in 1.790s

FAILED (failures=1, errors=1)

The "Access denied" made me think that the test was wrongly trying to connect as user '' (instead of using what is passed to -u); to see if this was the case I restarted mysqld with:
./mtr --mem alias --start --mysqld=--skip-grant-tables &
(to eliminate any privilege checks in mysqld), but the errors were the same.

I see that some successful pieces (before the failure) were able to connect, I see them in mysqld's query log.

If there is something I can do to help, let me know.

[Geert JM Vanderkelen (geertjmvdk)]

I can't repeat this problem using:
 ./unittests.py -H loclahost -S /tmp/mysql.sock -u root -D test -t pep249

Tested using MacOSX and Linux, just to make sure the getpass() works owkay.

[GuilhemBichot (guilhem-bichot)]

Hello Geert. I retried and got the same errors. I started mysqld by hand with a simple command line, errors too.
Steps
cd mysql-5.1/sql # using debug build, Linux x64
 ./mysqld --datadir=../data --language=./share/english/ --skip-grant-tables --skip-innodb --port=3307 --socket=/tmp/mysql.sock --pid-file=/tmp/a.pid
and then:
python unittests.py -H localhost -S /home/mysql_src/bzrrepos/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock -u root -D test -p pep249
and then errors.
I added the same "-p" as you:
python unittests.py -H localhost -S /home/mysql_src/bzrrepos/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock -u root -D test -p pep249
and same errors.

[Geert JM Vanderkelen (geertjmvdk)]

Hello Guilhem,

Please retest with revision 145 (internally available, but soon on Launchpad).

I fixed a couple of things in the tests and added a -P/--port option to change the default port of MySQL.

[GuilhemBichot (guilhem-bichot)]

I pulled 145, ran "python setup.py install", and then ran the tests:
======================================================================
ERROR: test_timeout (tests.test_bugs.Bug328998Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_bugs.py", line 43, in test_timeout
    c.execute, "SELECT SLEEP(%d)" % (config['connection_timeout']+4))
  File "/usr/lib/python2.5/unittest.py", line 320, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/cursor.py", line 314, in execute
    res = self.protocol.cmd_query(stmt)
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/protocol.py", line 262, in cmd_query
    buf = self.conn.recv()[0]
  File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/connection.py", line 72, in recv
    header = self.sock.recv(4, self.socket_flags)
timeout: timed out

======================================================================
FAIL: Cursor isolation with 2 cursors, different connections, trans.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_pep249.py", line 370, in test_isolation2
    self._isolation_test(self.db,db2,'InnoDB')
  File "/home/mysql_src/bzrrepos/connector_python/tests/test_pep249.py", line 355, in _isolation_test
    self.assertEqual(None, result)
AssertionError: None != (1L, u'myconnpy')

----------------------------------------------------------------------

[Geert JM Vanderkelen (geertjmvdk)]

No good still.. timeout and innodb..

Any other configuration other that the parameters given to mysqld? Like transaction levels of InnoDB for example? There is no --defaults-file or such to prevent system wide options to be read, is why I ask.

The timeout is still odd..

[GuilhemBichot (guilhem-bichot)]

Geert: did you try running exactly like me:
Started a MySQL 5.1 server with:

cd mysql-test
./mtr --mem alias --start &

Back in the directory of connector/python, did
python unittests.py -H localhost -S /home/mysql_src/bzrrepos/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock -u root -D test

(5.1 server is debug build).

[Geert JM Vanderkelen (geertjmvdk)]

AssertionError: None != (1L, u'myconnpy')

This happens because the InnoDB is not active. Start mtr with --mysqld="--innodb" to have it available.

I've pushed change to our tests which needs a specific storage engine. They can now check for the availability of the engine. This is part of revision 146.

The timeout-test failing is still not reproducible.

[GuilhemBichot (guilhem-bichot)]

--mysqld="--innodb" does fix the "AssertionError: None != (1L, u'myconnpy')".
I still get timeout-test failing. Debugged it a bit: in

File "/home/mysql_src/bzrrepos/connector_python/mysql/connector/connection.py", line 72, in recv
    header = self.sock.recv(4, self.socket_flags)
 the MySQLBaseConnectionr.recv fails with exception of type:
 <class 'socket.timeout'> (according to sys.exc_info())

[GuilhemBichot (guilhem-bichot)]

It may be a bug in my Python installation or some module (though it's a clean install of Python 2.5.2 from my Ubuntu).
1) If I run this:
 python unittests.py -t bugs -H localhost -S /home/mysql_src/bzrrepos/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock -u root -D test
I see the failure:
test_set_connection_timetout (tests.test_bugs.Bug328998Tests) ... ok
test_timeout (tests.test_bugs.Bug328998Tests) ... ERROR
2) I I patch connection.py like this:
=== modified file 'mysql/connector/connection.py'
--- mysql/connector/connection.py 2009-11-07 15:00:45 +0000
+++ mysql/connector/connection.py 2009-11-25 15:53:12 +0000
@@ -74,6 +74,8 @@
             buf = header + self.sock.recv(pktsize, self.socket_flags)
             self.protocol.is_error(buf)
         except:
+ fd=open("/tmp/aa","w")
+# close(fd)
             raise

         return (buf, pktsize, pktnr)

it still fails.
3) If I uncomment the "close(fd)", test passes.

[Geert JM Vanderkelen (geertjmvdk)]

Yes, Ubuntu, Python 2.5, the timeout test fails as well. Time to indeed review the problems around the sockets.

======================================================================
ERROR: test_timeout (tests.test_bugs.Bug328998Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/geert/mysql-connector-python-0.1.0-devel/tests/test_bugs.py", line 50, in test_timeout
    c.execute, "SELECT SLEEP(%d)" % (config['connection_timeout']+4))
  File "unittest.py", line 320, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "/home/geert/mysql-connector-python-0.1.0-devel/mysql/connector/cursor.py", line 320, in execute
    res = self.protocol.cmd_query(stmt)
  File "/home/geert/mysql-connector-python-0.1.0-devel/mysql/connector/protocol.py", line 268, in cmd_query
    buf = self.conn.recv()[0]
  File "/home/geert/mysql-connector-python-0.1.0-devel/mysql/connector/connection.py", line 78, in recv
    header = self.sock.recv(4, self.socket_flags)

[GuilhemBichot (guilhem-bichot)]

I looked at this in winpdb, and it is weird.
sock.rev() raises socket.timeout (normal, it seems). This is caught by the try/except of cmd_query(), which then raises it again. So we go up one level, in cursor.py:
        except StandardError, e:
            raise errors.InterfaceError(
                "Failed executing the operation; %s" % e)
which raises InterfaceError, as the unit test expects.
This exception is then caught inside failUnlessRaises(), as expected, so we come to the "return"

    def failUnlessRaises(self, excClass, callableObj, *args, **kwargs):
        <cut>
        try:
            callableObj(*args, **kwargs)
        except excClass:
            return # <<< we come here

but, somehow it seems that the exception still comes out of this, as the debugger says that
        self.assertRaises(errors.InterfaceError,
            c.execute, "SELECT SLEEP(%d)" % (config['connection_timeout']+4))
produces an exception.
Weird: the exception was apparently caught but it seems to come out.

[Geert JM Vanderkelen (geertjmvdk)]

Code has been refactored and this bug has to be rechecked with later release (no date yet).