debdelta integration / zsync support

Bug #21837 reported by andrewpmk on 2005-09-17
148
This bug affects 23 people
Affects Status Importance Assigned to Milestone
apt (Baltix)
Undecided
Unassigned
apt (Ubuntu)
Wishlist
Unassigned
Declined for Intrepid by Brian Murray

Bug Description

Ubuntu should, like Mandrake 10.2, Firefox 1.5 and Windows, update packages
using binary patches instead of by re-downloading entire packages. This would be
especially helpful when running beta versions of software which change daily.

Matt Zimmerman (mdz) on 2006-01-19
Changed in apt:
assignee: mdz → mvo
Michael Vogt (mvo) wrote :

Thanks for your bugreport.

There are various way to do this discussed currently. The most promissing approaches currently are using https://wiki.ubuntu.com/APTPackageDeltas and zsync. I personally think zsync is the way forward, but it needs to be able to deal with deb packages first.

Daniel Hahler (blueyed) on 2007-12-13
Changed in apt:
status: New → Triaged
Stéphane Maniaci (stephh) wrote :

This "bug" is still present in Hardy. Made a fresh install of Hardy yesterday : had to download 180Mo of update. How much would binary update decrease size ? Lowering the update size to ~10 Mo ? This is a __very__ interesting feature.

Stéphane Maniaci (stephh) wrote :

Apt-sync should do the trick : https://wiki.ubuntu.com/apt-sync

kojot (kojot350-tlen) wrote :

Can anyone tell me why this is still unimplemented? It's 2 years now... :F

Adam Porter (alphapapa) wrote :

It's really a shame that Ubuntu still doesn't have this feature. Debian at least has apt indexes, which saves a lot of time updating the package lists. Just think about the poor users all over the world on low-bandwidth connections. Please make this a higher priority and get it implemented soon.

Jean-Paul (jeanpaul145) wrote :

It would fit well with the "goals" already set for Jaunty (that is, fast boot times, faster system performance, blurring of the line between off- and online apps).

Only, I don't think that a binary diff will work (especially not for people like me who don't have all the hard disk space in the world, and consequently remove all the cached packages. I'd rather redownload 100 MB than filling up my hdd).

Jan Niklas Hasse (jhasse) wrote :

I thought that the binary patches are for the actually installed files in /usr, so that the cache isn't needed at all.

Jairaj (ursjai) wrote :

fedora is working on a similar project https://hosted.fedoraproject.org/presto
This doesn't require you to have a cache of the downloaded rpms but just the installed files.

Jean-Paul (jeanpaul145) wrote :

If the diffs are for the installed files, then I don't see why that wouldn't work.
However, it seems that Fedora's Presto utilizes an rpm utility, which is making me doubt if it is flexible enough to adapt it to deb without rewriting major parts of the code.
If it isn't flexible enough, there might as well be a new project, one that perhaps *is* flexible enough to work with multiple package management systems.

Michael Vogt (mvo) on 2009-06-09
Changed in apt (Ubuntu):
assignee: Michael Vogt (mvo) → nobody
status: Triaged → Confirmed
LoonyPhoenix (loonyphoenix) wrote :

Could Google's Courgette be used here?

http://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html

As far as I understand, it would be applicable here, no?

Doesn't debdelta[1] just do that?

[1] http://packages.ubuntu.com/lucid/debdelta

Julian Andres Klode (juliank) wrote :

Retitled, so it can be found when searching for the common keywords.

summary: - Binary patch updating
+ debdelta integration / zsync support
Gabe Gorelick (gabegorelick) wrote :

https://wiki.ubuntu.com/SmallerUpdates gives a lot of (outdated?) information on this.

This problem is even more serious now that Ubuntu is pushing paid apps like games through the Software Store. Downloading a 4 GB game for a simple bug fix is not pleasant.

Changed in apt:
status: Unknown → Fix Released
Changed in apt (Baltix):
status: New → Invalid
Changed in apt (Baltix):
status: Invalid → Confirmed
Julian Andres Klode (juliank) wrote :

Work on delta debs was being done in https://wiki.debian.org/Teams/Dpkg/Spec/DeltaDebs amongst other places, but it kind of stopped a bit. Deltas are getting harder and harder to justify with increasing speeds and less and less data caps, but it's certainly something I'm interested in developing.

I believe my format is fairly solid. The problem with debdelta was that it was incredibly slow, to the point where using it outside of dialup (well, I did not measure it, it might be a hyperbole) would basically be pointless.

Changed in apt (Ubuntu):
status: Confirmed → Triaged
no longer affects: apt
Julian Andres Klode (juliank) wrote :

Blocking points raised are:

- We need to be able to validate that the delta is correct before using it. We could

+ provide delta indices like we have packages - this does require pdiff support so we don't have to download the entire index everytime, as that would make deltas pointless for small updates
+ sign the deltas directly - but that's too slow, signatures are slow
+ provide one delta index per source package - might be useful compromise

- we need to uniquely identify package. I proposed we introduce a package id, based on the hash of the mtree (+ control fields maybe), but this does need mtrees first (or hashing the md5sum file)

- space usage on mirrors, how to keep this in check

Julian Andres Klode (juliank) wrote :

Oh, also building the deltas (which is fairly expensive), and which deltas to build are important points to consider.

Clinton H (49studebaker) wrote :

Windows and android have delta updates.

https://en.wikipedia.org/wiki/Delta_update

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.