Progress reporting during "Preparing the upgrade" wrong

[Mary Gardiner]

Binary package hint: update-manager

When running update-manager -c -d (feisty to gutsy, but I pre-installed the gutsy version of update-manager in order to get the fix for bug 732463) the stage "Preparing the upgrade" does not produce consistent, or even comprehensible, progress output. At various times the progress summary stated "fetching file 1 of 29", "fetching file 1 of 2", "fetching file 6 of 25" and "fetching file 20 of 35". It eventually settled on 35 files in total, but only after guessing 29, 2 and 25 in that order. In addition, the progress bar was full very early in a process that took several minutes.

This makes it almost impossible to guess how long any stage of the process might take, and in fact encourages the user to expect it to be considerably faster than it actually is... several times.

[Michael Vogt (mvo)]

2007-09-24 14:10:46,438 ERROR not handled expection:
Traceback (most recent call last):

  File "/tmp/distupgrade.oH6457/gutsy", line 59, in <module>
    app.run()

  File "/tmp/distupgrade.oH6457/DistUpgradeControler.py", line 1318, in run
    self.fullUpgrade()

  File "/tmp/distupgrade.oH6457/DistUpgradeControler.py", line 1307, in fullUpgrade
    self.doPostUpgrade()

  File "/tmp/distupgrade.oH6457/DistUpgradeControler.py", line 903, in doPostUpgrade
    if not self.cache._tryMarkObsoleteForRemoval(pkgname, remove_candidates, self.foreign_pkgs):

  File "/tmp/distupgrade.oH6457/DistUpgradeCache.py", line 519, in _tryMarkObsoleteForRemoval
    if pkgname.endswith(self.uname):

AttributeError: 'Package' object has no attribute 'endswith'

2007-09-24 14:10:48,915 ERROR failed to import apport python module, can't report bug: No module named python_hook

[Michael Vogt (mvo)]

Sorry, this last coment was meant for https://bugs.edge.launchpad.net/ubuntu/+source/update-manager/+bug/144417

[Matthew Paul Thomas (mpt)]

The underlying cause of this bug is that each repository file can refer to one or more other files that also have to be read. So we don't know, ahead of time, the total number of files that will need to be read (much less the size of each). An analogous problem occurs in displaying page load progress for Web browsers, where an HTML resource may refer to other resources such as images, frames, scripts, and style sheets, and many of those may in turn refer to still other resources that need to be loaded.

A simple solution, where
    p = the value of the progress meter (between 0 and 1)
    t = now
    t' = the previous time the progress meter was rendered
    F = the total number of files needing to be read that are known so far
    f = the proportion of those files that have been read so far (including the fraction(s) of the current file(s))
would be to render the progress meter as
    p(t) = p(t') + (f(t)-f(t'))/F.

This would at least ensure that the progress meter went from 0% to 100% only once, and never went backwards, so implementing it by itself it would be an improvement on the current behavior.

The next problem would be that the progress bar started off quickly (when F was small) and then slowed down depressingly as F became larger. (The progress meter in Internet Explorer for Windows has this problem, probably for the same reason.) The solution to this would be to scale the above function using a curve that incorporates a hard-coded constant for the *average* number of extra repository files (r) a typical repository file refers to. In other words, adjust the progress to be slower near the beginning and faster near the end, exactly counterbalancing the average gradual increase in F. Unfortunately I don't know enough maths to know what that curve would be (though I'll go out on a limb and guess that it involves a logarithm to the base of r).