Sorry, we have to split up :P

[Jason Gerard DeRose]

Hehe, my funny new years eve bug title :^)

So we're going to split Dmedia into multiple databases...

The main Dmedia database is for driving all the Dmedia automated behaviours.... the "making file management go away" part of Dmedia. The schema needed for this part is quite mature now, and is also surprisingly minimal considering the complex behaviours in drives. But it needs to be simple because this is the *scary* part of Dmedia... if we F-up, there could be data loss. This main database is really only manipulated by the Dmedia service, and it's not particularly important to snapshot... the idea is not to trust the metadata that much, but instead to constantly verify the reality... are these files still on hard drive X as expected, are the modification timestamps what we expect, do the content hashes match up?

The 2nd type of database is the project databases, and there will be one per project. This is all about content creation, and the metadata needed to support interesting applications built on Dmedia (eg, ingest with Dmedia for editing in Novacut). This is also the data the user will manipulate (say tagging, rating, deleting, comments, etc)... so these are databases we want to snapshot with version control. A project also represents the unit of sharing... you share/collaborate/publish at the project level. The metadata here is comparatively complex and also less refined thus far... but also a bigger domain, and we want things very free form because we want applications to be able to extend this metadata in the way they need.

Oh, and we're also going to start versioning the database names to make it easier to upgrade between schema versions, and so that running an older version of say Dmedia wont F-up your database that was created by a newer version. So when we're at schema version 0 (which we currently are), the database will be `dmedia-0`, and then `dmedia-1`, `dmedia-2` etc for subsequent schema versions. So after this lands, main automation-focused DB will be:

dmedia-0

Now not every file needs to be in a project and it will be valid for a file to be in multiple projects. Also, projects only make sense for 'origin': 'user' files... well, or projects that were shared with you, you might not be the origin. Anyway, so like always, a project with have a random, 120-bit, 24-character base32 encoded ID, something like this:

26TSXJ7T33KBQJUP725QLWSH

Damn, I love the way those look! Sorry... anyway, CouchDB database names must be lowercase, plus it's handy to see/filter what DBs are for Dmedia, what are Novacut. So the above project will have the database name:

dmedia-0-26tsxj7t33kbqjup725qlwsh

Notice we still version with the "0". I just landed this in Novacut as a little experiment... there's way less complexity in Novacut right now, so easier to try this out and get a feel for it.

I'm going to try and do this migration automatically for those who already have some files in Dmedia. There wont be any file-data loss for sure, and hopefully things will go smoothly for the metadata also. As Dmedia hasn't yet had usable UI for tagging or anything like that, we should be pretty safe here.

So two problem domains, two types of databases.