Backport "libdpkg: Only preallocate disk size for files bigger than 16 KiB".

Bug #1663749 reported by Vinson Lee on 2017-02-10
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
dpkg
Fix Released
Unknown
dpkg (Ubuntu)
High
Unassigned
Xenial
Undecided
Unassigned
Yakkety
Undecided
Unassigned

Bug Description

Please backport upstream dpkg patch "libdpkg: Only preallocate disk size for files bigger than 16 KiB" to xenial, yakkety, and zesty versions of dpkg.

dpkg 1.17.11 commit "dpkg: Try to preallocate the disk size for extracted files" causes installs of packages with many small files to be slow. https://anonscm.debian.org/git/dpkg/dpkg.git/commit/?id=87b0b20b86407baf1deb4e91b3fd839e01228ac8

This was fixed in dpkg 1.18.15 with "libdpkg: Only preallocate disk size for files bigger than 16 KiB". https://anonscm.debian.org/git/dpkg/dpkg.git/commit/?id=a971ad91437af8880cad4703695dcf12ee45959b

trusty dpkg is based off 1.17.5 and is not affected. xenial dpkg based off 1.18.4, yakkety dpkg based off 1.18.10, and zesty dpkg based off 1.18.10 need the fix.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in dpkg (Ubuntu):
status: New → Confirmed
Changed in dpkg:
status: Unknown → Fix Released
Vinson Lee (vlee) wrote :

Here is a test case than be used to benchmark the performance with and without the fix.

$ wget https://packages.chef.io/files/stable/chefdk/1.1.16/ubuntu/16.04/chefdk_1.1.16-1_amd64.deb
$ time sudo dpkg --install chefdk_1.1.16-1_amd64.deb

Luke Faraone (lfaraone) wrote :

Triaging as high, because this has a moderate negative performance impact/regression on *all* Ubuntu users, and is adversely impacting at least one substantially large deployment's Xenial rollout. (albeit with a hacky workaround)

Changed in dpkg (Ubuntu):
importance: Undecided → High
status: Confirmed → Triaged
tags: added: yakkety zesty
Vinson Lee (vlee) on 2017-04-01
tags: added: xenial

Hello Vinson, or anyone else affected,

Accepted dpkg into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpkg/1.18.4ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in dpkg (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed
Changed in dpkg (Ubuntu Yakkety):
status: New → Fix Committed
Łukasz Zemczak (sil2100) wrote :

Hello Vinson, or anyone else affected,

Accepted dpkg into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpkg/1.18.10ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpkg - 1.18.10ubuntu2

---------------
dpkg (1.18.10ubuntu2) zesty; urgency=medium

  * Cherry-pick patch from git to limit preallocation to files over
    16KiB, fixing installation performance regression (LP: #1663749)

 -- Adam Conrad <email address hidden> Mon, 10 Apr 2017 04:08:19 -0600

Changed in dpkg (Ubuntu):
status: Triaged → Fix Released
Łukasz Zemczak (sil2100) wrote :

I was asked to test this bug on a slower non-SSD disk. Checked zesty, yakkety and xenial. The detailed test result:

zesty:
- 1.18.10ubuntu1:
real 0m39.559s
user 0m7.176s
sys 0m8.780s
- 1.18.10ubuntu2:
real 0m20.273s
user 0m6.096s
sys 0m7.164s

yakkety:
- 1.18.9ubuntu2
real 0m38.154s
user 0m7.008s
sys 0m9.112s
- 1.18.10ubuntu1.1
real 0m24.107s
user 0m6.028s
sys 0m7.116s

xenial:
- 1.18.4ubuntu1
real 0m41.030s
user 0m8.164s
sys 0m8.632s
- 1.18.4ubuntu1.2
real 0m19.263s
user 0m6.792s
sys 0m5.644s

Retried a few times afterwards to make sure the results are reproducible. With this, I consider the bug as verified for both stable releases.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpkg - 1.18.4ubuntu1.2

---------------
dpkg (1.18.4ubuntu1.2) xenial; urgency=medium

  * Cherry-pick patch from git to limit preallocation to files over
    16KiB, fixing installation performance regression (LP: #1663749)

 -- Adam Conrad <email address hidden> Mon, 10 Apr 2017 04:08:19 -0600

Changed in dpkg (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for dpkg has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpkg - 1.18.10ubuntu1.1

---------------
dpkg (1.18.10ubuntu1.1) yakkety; urgency=medium

  * Cherry-pick patch from git to limit preallocation to files over
    16KiB, fixing installation performance regression (LP: #1663749)

 -- Adam Conrad <email address hidden> Mon, 10 Apr 2017 04:08:19 -0600

Changed in dpkg (Ubuntu Yakkety):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.