Add import record, make file's record reference import record

[Jason Gerard DeRose]

There are two reasons for this: 1) provide a richer audit trail for these all important files; 2) provide meta data for user interfaces to show things like all the files added during the last import operation... or actually during any import operation that ever occurred on any of your dmedia peers, which is pretty awesome.

It will go something like this:

When an import operation is triggered, a scan is made to see if any suitable files exists (based on file extension). We want to know how many files are being considered up front so that we can give the user a reasonable progress bar.

If at least some files are being considered for an import, a new import record will be created, something like this:

{
  "_id": UUID,
  "identity": UUID of machine/device doing import (same as self_identity used by dc?),
  "time_start": time.time(),
  "path": "/media/EOS_DIGITAL", # path importing from
  "considered": 324, # Number of files considered
}

Then dmedia proceeds with the import. When new files are found, their record contains a pointer to the above import record:

{
  "_id": CONTENT_HASH,
  "import": _id of import record created above,
}

When the import completes, the above import record is updated, adding these attributes:

{
  "time_end": time.time(),
  "actions": {
    "imported": 73,
    "duplicates": 251,
  }
}

Or something like that. There is probably more useful information that should go in the import record. Also, as the import records wont be created that often, they can be pretty verbose... and at this stage, erring on the side of too verbose is probably good because that's just more useful debugging/logging information.

So it might be good if the "actions" above was something more like this:

{
  "actions": {
    "imported": [list of CHASH of imported files],
    "duplicates": [list of QUICKID of skipped duplicates],
  }
}

The import record is the first dmedia record where we need to use a unique ID for a specific machine/device (dmedia peer). My hunch is it would make sense to use the same unique ID that desktopcouch uses. (Which I think is the "self_identity" attribute in the server_identity record, right?)

Stuart, what do you think?