Fix 400 error in test_Client.test_import()

Bug #710405 reported by Jason Gerard DeRose on 2011-01-31
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Dmedia
Critical
Jason Gerard DeRose

Bug Description

I'm not sure what's going on here. This test passes on Maverick, fails on Natty. Could be issue in dmedia, desktopcouch, or CouchDB. Run `./setup.py test` in the dmedia tree to see failure. Here is trace:

======================================================================
ERROR: test_import (dmedia.tests.test_client.test_Client)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/tests/test_client.py", line 212, in test_import
    self.assertEqual(inst.list_imports(), [])
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/client.py", line 238, in list_imports
    return self._call('ListImports')
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/client.py", line 141, in _call
    return method(*args)
  File "/usr/lib/pymodules/python2.7/dbus/proxies.py", line 140, in __call__
    **keywords)
  File "/usr/lib/pymodules/python2.7/dbus/connection.py", line 630, in call_blocking
    message, timeout)
DBusException: org.freedesktop.DBus.Python.couchdb.http.ServerError: Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/dbus/service.py", line 702, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/service.py", line 291, in ListImports
    return self.manager.list_imports()
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/service.py", line 126, in manager
    self._manager = ImportManager(self._on_signal, self._dbname)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/importer.py", line 323, in __init__
    self.metastore = MetaStore(dbname=dbname)
  File "/home/jderose/bzr/dmedia/natty-port-man/dmedia/metastore.py", line 177, in __init__
    self.desktop = CouchDatabase(self.dbname, create=True)
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 101, in __init__
    oauth_tokens=oauth_tokens, ctx=ctx)
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 75, in __init__
    oauth_tokens=oauth_tokens, ctx=ctx, views_factory=views_factory)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/database.py", line 123, in __init__
    self._reconnect()
  File "/usr/lib/pymodules/python2.7/desktopcouch/application/server.py", line 85, in _reconnect
    super(DesktopDatabase, self)._reconnect(uri=uri)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/database.py", line 137, in _reconnect
    if self._database_name not in self._server:
  File "/usr/lib/pymodules/python2.7/couchdb/client.py", line 93, in __contains__
    self.resource.head(validate_dbname(name))
  File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 377, in head
    return self._request('HEAD', path, headers=headers, **params)
  File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 419, in _request
    credentials=self.credentials)
  File "/usr/lib/pymodules/python2.7/desktopcouch/records/http.py", line 255, in request
    raise ServerError((status, error))
ServerError: (400, '')

----------------------------------------------------------------------
Ran 128 tests in 26.349s

FAILED (errors=1)

Related branches

lp:~jderose/dmedia/core-api
James Raymond: Approve on 2011-04-11
Jason Gerard DeRose (jderose) wrote :

Raised this to critical. This seems to be related to the D-Bus timeout problem I've been having:

DEBUG dispatch in process 25389: worker='ImportWorker', key=u'/home/jderose/Videos', args=('ISDGZ4OU3OUC3VZORCXZGXBX', u'/home/jderose/Videos', dbus.Boolean(True), None)
ERROR Introspect error on :1.311:/: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
DEBUG Executing introspect queue due to error

Changed in dmedia:
importance: High → Critical
Jason Gerard DeRose (jderose) wrote :

Okay, this bug is officially driving me crazy. But here's what I can tell so far:

1) As far as I know, it only occurs in the unit tests (so not a big concern). Why it occurs where it does, and why there wasn't a problem under Maverick, I have no idea.

2) My work on lp:722035 has not been able to fix this bug, however, as far as I can tell, it has fixed lp:714406 - which is far more important as lp:714406 was occurring frequently when actually using dmedia

Changed in dmedia:
milestone: 0.4 → 0.5
Changed in dmedia:
milestone: 0.5 → 0.6
Changed in dmedia:
status: Triaged → In Progress
assignee: nobody → Jason Gerard DeRose (jderose)
Jason Gerard DeRose (jderose) wrote :

Wow, after over 2 months, I finally have this fixed. Something weird is going on with desktopcouch get_oauth_tokens() in that it's returning a different value when called from a DBus service launched with Popen. This was all happing in unit tests only, not something that comes up during normal use. Steps:

1. In `setup.py` process, call get_oauth_tokens(), delete 'test_dmedia' database if it exists

2. Start `dmedia-service` or `dmedia-importer-service` with Popen, which will themselves call get_oauth_tokens(), but the value will be different than in (1)

3. Kill (2) and again start `dmedia-service` or `dmedia-importer-service` with Popen, they call get_oauth_tokens(), and the value will yet again be different... different than (2), still not equal to (1)

I'll try to cook up a minimal script to demo this behaviour outside of dmedia, against desktopcouch only.

I stumbled across the solution during the great refactor of 2011 [1]. As both `dmedia-service` and `dmedia-importer-service` can now take an --env arg with the JSON serialized environment, I pass them the env that was created in the `setup.py` process... the services then use the oauth tokens retrieved in the main process, don't themselves call get_oauth_tokens()

[1] Great refactor of 2011 - https://bugs.launchpad.net/dmedia/+bug/753260

Changed in dmedia:
status: In Progress → Fix Committed
Chad Miller (cmiller) wrote :

Hrm, it could be a race in starting up desktopcouch. Two processes try to access couchdb, but detect none running, and each start one. I'll investigate.

Jason Gerard DeRose (jderose) wrote :

For what it's worth, I tried putting a delay (like 1 or 2 seconds) between when the test started and when I launched the service with Popen, and it didn't seem to make a difference.

Anyway, thanks for looking into this!

Changed in dmedia:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers