Fix 400 error in test_Client.test_import()

[Jason Gerard DeRose]

I'm not sure what's going on here. This test passes on Maverick, fails on Natty. Could be issue in dmedia, desktopcouch, or CouchDB. Run `./setup.py test` in the dmedia tree to see failure. Here is trace:

======================================================================
ERROR: test_import (dmedia.tests.test_client.test_Client)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/tests/test_client.py", line 212, in test_import
    self.assertEqual(inst.list_imports(), [])
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/client.py", line 238, in list_imports
    return self._call('ListImports')
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/client.py", line 141, in _call
    return method(*args)
  File "/usr/lib/pymodules/python2.7/dbus/proxies.py", line 140, in __call__
    **keywords)
  File "/usr/lib/pymodules/python2.7/dbus/connection.py", line 630, in call_blocking
    message, timeout)
DBusException: org.freedesktop.DBus.Python.couchdb.http.ServerError: Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/dbus/service.py", line 702, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/service.py", line 291, in ListImports
    return self.manager.list_imports()
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/service.py", line 126, in manager
    self._manager = ImportManager(self._on_signal, self._dbname)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/importer.py", line 323, in __init__
    self.metastore = MetaStore(dbname=dbname)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/metastore.py", line 177, in __init__
    self.desktop = CouchDatabase(self.dbname, create=True)
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 101, in __init__
    oauth_tokens=oauth_tokens, ctx=ctx)
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 75, in __init__
    oauth_tokens=oauth_tokens, ctx=ctx, views_factory=views_factory)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/database.py", line 123, in __init__
    self._reconnect()
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 85, in _reconnect
    super(DesktopDatabase, self)._reconnect(uri=uri)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/database.py", line 137, in _reconnect
    if self._database_name not in self._server:
  File "/usr/lib/pymodules/python2.7/couchdb/client.py", line 93, in __contains__
    self.resource.head(validate_dbname(name))
  File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 377, in head
    return self._request('HEAD', path, headers=headers, **params)
  File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 419, in _request
    credentials=self.credentials)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/http.py", line 255, in request
    raise ServerError((status, error))
ServerError: (400, '')

----------------------------------------------------------------------
Ran 128 tests in 26.349s

FAILED (errors=1)

[Jason Gerard DeRose (jderose)]

Raised this to critical. This seems to be related to the D-Bus timeout problem I've been having:

DEBUG dispatch in process 25389: worker='ImportWorker', key=u'/home/jderose/Videos', args=('ISDGZ4OU3OUC3VZORCXZGXBX', u'/home/jderose/Videos', dbus.Boolean(True), None)
ERROR Introspect error on :1.311:/: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
DEBUG Executing introspect queue due to error

[Jason Gerard DeRose (jderose)]

I believe this is related to lp:714406

https://bugs.launchpad.net/ubuntu/+source/desktopcouch/+bug/714406

[Jason Gerard DeRose (jderose)]

Okay, this bug is officially driving me crazy. But here's what I can tell so far:

1) As far as I know, it only occurs in the unit tests (so not a big concern). Why it occurs where it does, and why there wasn't a problem under Maverick, I have no idea.

2) My work on lp:722035 has not been able to fix this bug, however, as far as I can tell, it has fixed lp:714406 - which is far more important as lp:714406 was occurring frequently when actually using dmedia

[Jason Gerard DeRose (jderose)]

Wow, after over 2 months, I finally have this fixed. Something weird is going on with desktopcouch get_oauth_tokens() in that it's returning a different value when called from a DBus service launched with Popen. This was all happing in unit tests only, not something that comes up during normal use. Steps:

1. In `setup.py` process, call get_oauth_tokens(), delete 'test_dmedia' database if it exists

2. Start `dmedia-service` or `dmedia-importer-service` with Popen, which will themselves call get_oauth_tokens(), but the value will be different than in (1)

3. Kill (2) and again start `dmedia-service` or `dmedia-importer-service` with Popen, they call get_oauth_tokens(), and the value will yet again be different... different than (2), still not equal to (1)

I'll try to cook up a minimal script to demo this behaviour outside of dmedia, against desktopcouch only.

I stumbled across the solution during the great refactor of 2011 [1]. As both `dmedia-service` and `dmedia-importer-service` can now take an --env arg with the JSON serialized environment, I pass them the env that was created in the `setup.py` process... the services then use the oauth tokens retrieved in the main process, don't themselves call get_oauth_tokens()

[1] Great refactor of 2011 - https://bugs.launchpad.net/dmedia/+bug/753260

[Chad Miller (cmiller)]

Hrm, it could be a race in starting up desktopcouch. Two processes try to access couchdb, but detect none running, and each start one. I'll investigate.

[Jason Gerard DeRose (jderose)]

For what it's worth, I tried putting a delay (like 1 or 2 seconds) between when the test started and when I launched the service with Popen, and it didn't seem to make a difference.

Anyway, thanks for looking into this!