pipelining on archive.u.c aborts after 101 packages
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
apt (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Xenial |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Impact]
Downloading many packages on archive.ubuntu.com or some other mirrors seems to close the connection after every 100 or so packages. APT prior to 1.7.0~rc1 (commit df696650b7a8c58
With high speed connections at higher latency, this can cause a severe reduction in usable bandwidth. For example, I saw speeds drop from 40 MB/s to 15 MB/s due to this.
The fix ensures that we continue pipelining if the previous connection to the server successfully retrieved at least 3 files with pipelining enabled.
[Test case]
Pick a package that would cause a large number (200/300 packages of packages to be installed). I used plasma-desktop and xubuntu-desktop, for example. Run apt install -d $package. Ensure that after the first 101 packages the progress does not slow down - you should not see a lot "working" in the progress output. The speed should be substantially higher.
Don't run in a container setup by cloud-init, as cloud-init disables pipelining; or remove /etc/apt/
Requirements:
* High speed, medium-high latency connection (e.g. 400 Mbit/s at 30 ms RTT is enough); or just increase latency, e.g.
sudo tc qdisc add dev wlp61s0 root netem delay 300ms
until you see the slow down
* Not a terribly slow CPU, as we'd get slowed down by hashing otherwise
[Regression potential]
This fix is isolated to code enabling/disabling pipelining on subsequent connections. It could cause more pipelining to be tried on servers who are not particularly good at it, but can deal with 3 items correctly.
Changed in apt (Ubuntu): | |
status: | New → Fix Released |
Changed in apt (Ubuntu Xenial): | |
status: | New → Triaged |
Changed in apt (Ubuntu Bionic): | |
status: | New → Triaged |
description: | updated |
description: | updated |
Hello Julian, or anyone else affected,
Accepted apt into bionic-proposed. The package will build now and be available at https:/ /launchpad. net/ubuntu/ +source/ apt/1.6. 5 in a few hours, and then in the -proposed repository.
Please help us by testing this new package. See https:/ /wiki.ubuntu. com/Testing/ EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification- needed- bionic to verification- done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed- bionic. In either case, without details of your testing we will not be able to proceed.
Further information regarding the verification process can be found at https:/ /wiki.ubuntu. com/QATeam/ PerformingSRUVe rification . Thank you in advance for helping!
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.