| 2012-12-31 02:28:04 |
Jason Gerard DeRose |
bug |
|
|
added bug |
| 2012-12-31 02:28:27 |
Jason Gerard DeRose |
branch linked |
|
lp:~jderose/filestore/protocols |
|
| 2012-12-31 03:24:21 |
Jason Gerard DeRose |
description |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addresses:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach approves simply too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patters ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addresses:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patters ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
|
| 2012-12-31 03:27:28 |
Jason Gerard DeRose |
description |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addresses:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patters ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addresses:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patterns ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
|
| 2012-12-31 03:59:12 |
Jason Gerard DeRose |
description |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addresses:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patterns ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
We're building Dmedia with the assumption that any given version of the hashing protocol will have a limited useful lifetime (likely in the 10-20 year range). And so we already have a migration strategy: each newer version of the protocol will use a different (probably longer) digest size, which means the protocol version can be identified via the digest length. This allows multiple versions of the protocol to be supported simultaneously during a lengthy transition period.
And we always want to be able to verify a file based on an older protocol version, even once newly added files are never hashed via such an old protocol. The reason for this is so we can maintain references made to these files via their ID under an earlier protocol (in Novacut edits, federated public systems, etc).
For background, see the current proposed v1 protocol:
http://docs.novacut.com/filestore/protocol2.html
And this excellent paper on the challenges and limitations of content-based addressing:
http://valerieaurora.org/monkey.html
Anyway, with this is mind, I want to update FileStore to select the protocol version via the ID length even when there is only a single protocol version. There are two reasons to do this:
1) It's a sanity check on how viable this migration strategy is. It's better to figure this out now rather than waiting till the migration is actually needed. So if this approach proves too complex, lets embrace reality now and figure out something different. (Note that currently the approach seems entirely viable.)
2) We want to start practicing and learning about how best to implement this functionality now, so that our reference implementation has well refined design patterns ready to embrace a protocol migration
The other issue is we're considering using a 280-bit digest for the official version one protocol, which will allow us to do a real practice migration from the current 240-bit "interim" version zero protocol. That is still up for debate, but right now I'm working from the assumption that the version one protocol will be 280-bit.
Regardless, we will provide a migration path from the current interim protocol, even if it's a somewhat hacky one-time migration that the user must manually run. The interim protocol has been stable for over a year, and a number of folks have built fairly large Dmedia libraries, so we need to provide a reasonable upgrade path for them. |
|
| 2012-12-31 16:48:01 |
Launchpad Janitor |
branch linked |
|
lp:filestore |
|
| 2012-12-31 16:48:08 |
Jason Gerard DeRose |
filestore: status |
In Progress |
Fix Committed |
|
| 2013-01-28 05:26:06 |
Jason Gerard DeRose |
filestore: milestone |
13.01 |
13.02 |
|
| 2013-01-28 05:26:52 |
Jason Gerard DeRose |
filestore: milestone |
13.02 |
13.01 |
|
| 2013-01-28 05:26:58 |
Jason Gerard DeRose |
filestore: status |
Fix Committed |
Fix Released |
|
