server minimal virtual installations are reported to be bloated by utah

[Para Siva]

Utah appear to be adding significant content for minimal virtual server installations for raring smoke tests. Whilst a fresh local minimal virtual installation of amd64 server takes up roughly 611MB the reported installation size using utah is 964MB which is nearly 50% offset to the correct value.

Please see comment #10 for further information as to what could add to the contents in preseeded installations required for utah.

Originally it was with i386 but from 20121209 the amd64 installations are also reporting larger than the expected maximum for the daily smoke tests.

i386 installations are reporting 885560 against the expected maximum of 870400

amd64 installations are showing 964812 against the expected maximum of 912656

i386 server (20121207) minimal virtual installations for raring with utah installation are reporting an installation size of 885560 against the expected maximum of 870400. The size was below the expected max until 20121206 for i386 server images and the bloating only happened with 20121207 i386 server images. For amd64 images the size was below the expected until 20121208 and the bloating only happened with 20121209 images

The corresponding test code is located,
http://bazaar.launchpad.net/~ubuntu-server-dev/utah/server-tests-quantal/view/head:/testsuites/minimal-virtual/test_install_size/test.py
The failure is

======================================================================
    FAIL: testInstallSize (__main__.MinimalVirtualTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test.py", line 48, in testInstallSize
        self.assertTrue( used < max_install_size, "Used: %s" % (used))
    AssertionError: Used: 885560

The impacted job is,
https://jenkins.qa.ubuntu.com/view/Raring/view/Smoke%20Testing/job/raring-server-i386-smoke-minimal-virtual/20/

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1087630

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Para Siva (psivaa)]

the logs from /var/log of a failing VM is attached. The size after removing utah related content is 888924 in this particular VM. Hence do not suspect utah for the increase in the size.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1087630

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Para Siva (psivaa)]

The logs are attached above.

[Joseph Salisbury (jsalisbury)]

This doesn't appear to be a kernel bug. Maybe this should be filed against the ubiquity package?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Gema Gomez (gema)]

This bug is making smoke testing fail, it should be dealt with asap.

[James Page (james-page)]

I installed a new, fresh minimal-virtual install from todays raring server iso (amd64) and the total install size is 611MB (which is less than the 715MB threshold in the test).

Looks like the testing overlay is bloating things out more than it did two days ago - the test allows 150MB for the overlay.

[Max Brustkern (nuclearbob)]

As part of the preseed, we install openssh-server and gdebi-core. APT reports that 497K get fetched for that, 2024K are used afterwards. After the installation, we install a custom version of python-jsonschema, utah-common, and utah-client. The packages for those three add up to 112K. The whole utah tree checked out from bzr including all the provisioning is 2.4M. Some of that won't be installed, but even if we assume everything is, and double the size to account for .pyc files, that still gives us under 5 MB for those packages, and under 8 total.

I ran minimal-virtual.run on a server, and checked the utah directories afterwards. /var/cache/utah contained 4K, and /var/lib/utah had 120K. There may be some other issue with the preseeding we're doing, but the parts in direct service of utah don't seem to be large enough to cause the problem. Do you still have that minimal-virtual install so we can compare it to a utah minimal-virtual install? If not, I can create one.

[Jean-Baptiste Lallement (jibel)]

The problem is that when you preseed:
apt-get -q -y install -- openssh-server python-yaml bzr git gdebi-core python3-software-properties software-properties-common

it installs:
Dec 6 08:40:54 in-target: The following NEW packages will be installed:
Dec 6 08:40:54 in-target: bzr ca-certificates gdebi-core gir1.2-glib-2.0 git git-man krb5-locales
Dec 6 08:40:54 in-target: libasn1-8-heimdal libcurl3-gnutls libdbus-glib-1-2 libedit2 liberror-perl
Dec 6 08:40:54 in-target: libgcrypt11 libgirepository-1.0-1 libgmp10 libgnutls26 libgpg-error0
Dec 6 08:40:54 in-target: libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
Dec 6 08:40:54 in-target: libheimntlm0-heimdal libhx509-5-heimdal libidn11 libjs-jquery libk5crypto3
Dec 6 08:40:54 in-target: libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2
Dec 6 08:40:54 in-target: libp11-kit0 libpython3.3 libroken18-heimdal librtmp0 libsasl2-2
Dec 6 08:40:54 in-target: libsasl2-modules libtasn1-3 libwind0-heimdal libwrap0 libx11-6 libx11-data
Dec 6 08:40:54 in-target: libxau6 libxcb1 libxdmcp6 libxext6 libxmuu1 libyaml-0-2 ncurses-term
Dec 6 08:40:54 in-target: openssh-client openssh-server openssl patch python-bzrlib python-configobj
Dec 6 08:40:54 in-target: python-crypto python-httplib2 python-keyring python-launchpadlib
Dec 6 08:40:54 in-target: python-lazr.restfulclient python-lazr.uri python-oauth python-paramiko
Dec 6 08:40:54 in-target: python-pkg-resources python-simplejson python-wadllib python-yaml
Dec 6 08:40:54 in-target: python-zope.interface python3-apt python3-dbus python3-gi
Dec 6 08:40:54 in-target: python3-software-properties rsync software-properties-common ssh-import-id
Dec 6 08:40:54 in-target: tcpd unattended-upgrades wget xauth

[Para Siva (psivaa)]

Marking fix released since this is no longer an issue