Select protocol version based on ID length

Bug #1094801 reported by Jason Gerard DeRose on 2012-12-31
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
FileStore
High
Jason Gerard DeRose

Bug Description

We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.

And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).

For background, see the current proposed v1 protocol:

http://docs.novacut.com/filestore/protocol2.html

And this excellent paper on the challenges and limitations of content-based addressing:

http://valerieaurora.org/monkey.html

Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:

1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)

2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patterns ready to embrace a protocol migration

The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.

Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them.

Related branches

description: updated
description: updated
description: updated
Changed in filestore:
status: In Progress → Fix Committed
Changed in filestore:
milestone: 13.01 → 13.02
milestone: 13.02 → 13.01
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers