support running apt with eatmydata

Bug #1236531 reported by Scott Moser on 2013-10-07
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)

Bug Description

During instance boot, cloud-init installs packages (if instructed by the user).
During instance boot, there is no state of the system that the user cares about.

I'd like to use eatmydata for apt-get install at that point, as it is dramatically faster than apt-get even with --force-unsafe-io.

From an ubuntu perspective, this change would mean:
 a.) depending on (or recommending) eatmydata
 b.) MIR for eatmydata
 c.) fixing eatmydata's relationship with sysvinit scripts. The thing to fix there is that if you 'eatmydata apt-get install some-service' and some-service starts a daemon, the daemon will inherit the LD_PRELOAD which is most likely not desired. (bug 1257036)

Related bugs:
 * bug 1257036: services started with eatmydata should remove eatmydata by default

Related branches

Scott Moser (smoser) on 2013-10-07
Changed in cloud-init (Ubuntu):
milestone: none → ubuntu-14.02
status: New → Confirmed
Changed in cloud-init:
status: New → Confirmed
Scott Moser (smoser) on 2013-10-29
description: updated
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in libeatmydata (Ubuntu):
status: New → Confirmed
Scott Moser (smoser) on 2013-12-17
description: updated
no longer affects: libeatmydata (Ubuntu)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.5~bzr902-0ubuntu1

cloud-init (0.7.5~bzr902-0ubuntu1) trusty; urgency=medium

  * debian/control: Build-Depend on python-jsonpatch as #717916 is now fixed.
  * debian/control: Recommend eatmydata (LP: #1236531)
  * New upstream snapshot.
    * support invoking apt with 'eatmydata' (LP: #1236531)
    * add a message in log about dynamic import failures
  * New in '0.7.4' release.
    * fix reading of mount information on kernels < 2.6.26 (LP: #1248625)
    * SmartOS: change 'region' to 'datacenter_name' to address change
      in data provided to instance (LP: #1249124)
    * support calling 'add-apt-repository' for 'cloud-archive:' entries
      (LP: #1244355)
    * DataSourceAzure: fix incompatibility with python 2.6 (LP: #1232175)
    * fix bug mounting first partition of a alias'd name. (LP: #1236594)
    * SmartOS: fix bug with hostname due to trailing whitespace (LP: #1236445)
    * fix creation of partitions on Azure (LP: #1233698)
    * cc_growpart: respect /etc/growroot-disabled (LP: #1234331)
    * ubuntu config: add default user to 'sudo' group (LP: #1228228)
    * Fix usage of libselinux-python when selinux is disabled
    * add OpenNebula datasource
 -- Scott Moser <email address hidden> Tue, 17 Dec 2013 16:51:30 -0500

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Scott Moser (smoser) wrote :

So, very unscientific. But 2 systems launched with eatmydata enabled on trusty on azure. 2 systems without. Systems here were 'extrasmall' (670M memory).
cloud-config to do this is:
| system_info:
| apt_get_wrapper:
| enabled: False

The /var/log/cloud-init.log will have something like:
Jan 9 15:18:27 sm-eatdata0 [CLOUDINIT][DEBUG]: apt-install [eatmydata apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install bzr pastebinit ubuntu-dev-tools ccache bzr-builddeb vim-nox git-core lftp] took 105.916 seconds

Jan 9 15:26:20 sm-realsave0 [CLOUDINIT][DEBUG]: apt-install [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install bzr pastebinit ubuntu-dev-tools ccache bzr-builddeb vim-nox git-core lftp] took 88.697 seconds

eatmydata: 145.542 105.916
no-eatmydata: 88.697, 78.097

So... some interesting data there. I suspect that is a result of memory pressure on the vfs or something. Something to look at though.

Scott Moser (smoser) wrote :

ok. so on a 'small' instance type (1678M memory), I see:
eatmydata: 94.597, 103.672
no-eatmydata: 137.641, 115.919

All of the above were done on
3.12.0-7-generic (linux-image-3.12.0-7-generic 3.12.0-7.15).

Scott Moser (smoser) wrote :

bah. i think my data is garbage. i'll try to get better data. The issue I see is that my tests had run my 'apt-go-fast' ( as a bootcmd which makes /usr/bin/apt a wraper around eatmydata /usr/bin/apt.real anyway.

so really, the above should have all been being run with eatmydata anyway.

better numbers coming.

Scott Moser (smoser) wrote :

ok. better tests and results using the provided launcher script here.
I launched 3 small , 3 extrasmall both with and without eatmydata involved. sm-save-extrasmall-2 failed to launch (api/service failure). Below are the results, obtained with with the shell below.

sm-eat-extrasmall-0 104.547 16 88
sm-eat-extrasmall-1 106.096 12 94
sm-eat-extrasmall-2 127.666 35 92
sm-eat-small-0 103.512 10 93
sm-eat-small-1 128.128 24 104
sm-eat-small-2 101.121 5 96
sm-save-extrasmall-0 342.229 6 336
sm-save-extrasmall-1 332.766 5 327
sm-save-small-0 307.896 36 271
sm-save-small-1 290.326 28 262
sm-save-small-2 276.366 26 250

Scott Moser (smoser) wrote :

fixed in 0.7.5

Changed in cloud-init:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers