When possible, use python-couchdb directly rather than desktopcouch

[Jason Gerard DeRose]

dmedia is obviously very focused on desktop use, for which desktopcouch is a godsend.

But dmedia also needs to run on headless, minimal servers. In this case, we should start dmedia using upstart (rather than as a per-user D-Bus service), and talk to the system wide CouchDB (rather than to a per user instance started with desktopcouch).

Long term, I'd love to see some abstractions in desktopcouch to make it easy to run an application against either desktopcouch or system wide CouchDB without the application knowing the difference. I expect a lot of the really cool use cases for desktopcouch will also involve a server component. And that basically the same component will run on the desktop as well (so that functionality is available whether or not you have network connectivity). That's the case with dmedia anyway.

By taking a stab at this in dmedia, I hope to gain some insights so that this can be further discussed with the desktopcouch people at UDS-O.

My basic plan is to add a get_couchdb_server() or somesuch that returns an appropriate couchdb.Server instance. When running on a desktop with desktopcouch, the couchdb.Server instance will be created with an appropriate desktopcouch.records.http.OAuthSession instance. When running on a headless server, the couchdb.Server will be created using a static port from a config file in /etc. Or something like that.

I also want to talk to CouchDB using a couchdb.Database rather than a desktopcouch.application.server.DesktopDatabase. I want less wrapper, more CouchDB REST API.

Actually, for my tastes python-couchdb itself has far too much wrapper, far too little CouchDB REST API, although that's another story. But a small rant: I'm personally appalled that python-couchdb wraps every dict in its basically useless Row. The Python builtin data types are wicked fast and quite memory efficient... but if a library (like python-couchdb) gets a bad case of wrapper-itis, there's a big performance and memory hit. Rant rant...

[Jason Gerard DeRose (jderose)]

Ah, something important I forgot to note: in case anyone is wondering why I'm making this seemingly aggressive change so near the 0.4 release, it's because this should fix lp:710405 "Fix 400 error in test_Client.test_import()":

https://bugs.launchpad.net/dmedia/+bug/710405

With this change, I'll only call desktopcouch.find_port() in the main process, and then just pass a dict with the CouchDB info to subprocesses creating with `multiprocessing`. Longterm, this is the way dmedia needs to work anyway, and making this change will probably take less time than resolving the mysterious D-Bus + Python multiprocessing issue that sprung up under Natty.

[Jason Gerard DeRose (jderose)]

Okay, this is getting close to ready for a merge.

1) Appears to have fixed lp:714406 - DBusException: org.freedesktop.DBus.Error.NoReply:

https://bugs.launchpad.net/ubuntu/+source/desktopcouch/+bug/714406

2) No luck on lp:710405 - Fix 400 error in test_Client.test_import():

https://bugs.launchpad.net/dmedia/+bug/710405

But as far as I can tell, lp:710405 is only occurring in the test_Client.test_import() unit test, is not occurring when actually using dmedia (so low priority). Right now it seems that lp:710405 and lp:714406 aren't related after all.

I'll work on this a bit more and then make a decision on whether to include in 0.4. Fixing lp:714406 is fairly high priority as it makes dmedia quite unreliable.