Add quick id function to speed up import dedup
Bug #671949 reported by
Jason Gerard DeRose
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Dmedia |
Fix Released
|
High
|
Akshat Jain |
Bug Description
We need a quick dedup to run when importing files that may have already been imported. This is especially important when importing large files (say 4GB MOV) over slow USB2. A good algorithm would probably be something like:
hash(utf-8 repr of mtime + 1024k + last 1024k)
Corner cases are when total file is less than 1MiB and when less than 2MiB.
This function should go in dmedialib/
quick_
When you can pass either a filename or an open file handle.
Related branches
lp:~akshatj/dmedia/quick-id
- Jason Gerard DeRose: Pending requested
-
Diff: 59 lines (+15/-2)2 files modifieddmedialib/filestore.py (+9/-0)
dmedialib/tests/test_filestore.py (+6/-2)
Changed in dmedia: | |
milestone: | none → 0.1 |
importance: | Undecided → High |
status: | New → Triaged |
Changed in dmedia: | |
assignee: | nobody → Akshat Jain (ssj6akshat1234) |
status: | Triaged → In Progress |
Changed in dmedia: | |
status: | In Progress → Fix Committed |
Changed in dmedia: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.