Part 1: unified HTTPS server

Bug #1053909 reported by Jason Gerard DeRose on 2012-09-21
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Jason Gerard DeRose

Bug Description

Requiring an open port accepting outside connections is a bit scary, but there's no other way to do device to device file transfer and CouchDB replication on the local network. Use the cloud (and Internet) when it helps solve the problem, but never depend on it.

And worse, right now we have two open ports (file transfer, and CouchDB). So we want to condense this into a single HTTPS server that handles file transfers, and proxies requests to CouchDB. Many things about CouchDB's design make me not thrilled about having it directly accepting outside connections, so CouchDB will only listen on

This server will be launched via it's own executable so we can use a tightly contained AppArmor profile. It needs to talk to CouchDB, and it needs read access to files in FileStore (.dmedia/files/*/*). A few restrictions I think are prudent:

 * No DBus access whatsoever... the needed CouchDB env will be passed via stdin when the server is launched

 * Server will use SSL and will only accept connections from clients with certs signed by user CA or otherwise in a cert whitelist

 * Server should block access to all the "fun" CouchDB paths like /_config, /_replicate, etc... we probably want to block all the /_foo paths, and some of the /dbname/_foo paths too

 * Server should be smart about using limited credentials when forwarding to CouchDB... this is especially important once we can do project sharing on the local network

 * Server should be smart about limiting what CouchDB paths can be accessed based on the cert the client is connecting with... again, this is especially important for localnet project sharing. For example, the user's other devices need to replicate dmedia-0, but "outside" users should never have access to dmedia-0 (they should only be able to access the dmedia-0-foo and novacut-0-foo project databases).

I also want this server to use IPv6. As this is only indented for localnet use, it doesn't matter if the ISP offers IPv6 support. When there isn't an IPv6 DHCP server, Ubuntu automatically configures all networking interfaces with IPv6 link-local addresses (derived from the MAC address). So that means you could plug two devices together with an Ethernet cable, and it would just work, with no config, no DHCP server.

Related branches

description: updated
description: updated
description: updated
Changed in dmedia:
status: Triaged → In Progress
summary: - Unified web server for file transfer and CouchDB proxy
+ Unified web server for file transfer and CouchDB proxy [Part 1]
summary: - Unified web server for file transfer and CouchDB proxy [Part 1]
+ Unified web server for file transfer & CouchDB proxy [Part 1]

Okay, this is getting split into several pieces. The first piece here is big and sprawling, but that's how it goes sometimes.

I ended up writing a very small WSGI server tailored to our needs. When it comes to Python3 WSGI servers that are suitable for a desktop HTTP server, support IPv6, have SSL support, and are packaged in Debian/Ubuntu.... well, that list includes zero candidates, unless I missed something.

SSL support is especially a critical issue. As we're using client-side certificates in a very specific way, we need direct control over how SSL is being used.

`ssl.SSLContext` is the future... it's getting lots of development, even in Python 3.2 it already can do everything that we really need it to. But that also means we can only look for Python3-only WSGI servers, as `ssl.SSLContext` is new in Python 3.2.

Another reason why I wrote our own WSGI server is I want something that's easy to spin-up for unit tests... something that can be started as a Python library. That's the biggest reason why I ruled out uwsgi-plugin-python3. Plus, uwsgi is just way more complicated than what we need. And Apache + mod_wsgi is a very poor fit for a per-user, desktop HTTP server (obviously).

Anyway, still to do:

1) Finished designing the new SSL-based machine and user identity stuff... it's close, but needs a few more tweaks

2) Build the new WSGI app that will proxy requests to CouchDB

3) Finished the secure peering protocol (also very close)

4) Turn it all on!

Changed in dmedia:
status: In Progress → Fix Committed
summary: - Unified web server for file transfer & CouchDB proxy [Part 1]
+ Part 1: unified HTTPS server
Changed in dmedia:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers