Support Ubuntu 14.04 as a host OS

[Alexei Sheplyakov]

- Make sure Fuel components (nailgun, astute) work properly with Ubuntu 14.04 (or rather find out what gets broken)
- Rebuild packages (find out the exact set) in Ubuntu 14.04 environment
- Modify ISO build system so it's possible to create ISOs with Ubuntu 14.04 packages

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/127445
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=84541ff9db89158ca7c499385b096dd62a7074ea
Submitter: Jenkins
Branch: master

commit 84541ff9db89158ca7c499385b096dd62a7074ea
Author: Alexei Sheplyakov <email address hidden>
Date: Tue Oct 7 18:25:10 2014 +0400

    requirements-deb.txt: omit redundant packages

     * don't list packages pulled by ubuntu-minimal/ubuntu-standard metapackages
     * don't list the dependencies of build-essential metapackage
     * don't list nova, neutron, cinder, and glance dependencies

    Skip the prerequisites of the following packages:

     * apache2
     * aptitude
     * cloud-init
     * corosync
     * cpu-checker
     * crda
     * discover
     * gdisk
     * git
     * grub-pc
     * guestmount
     * libgtk2.0 (not sure why we need libgtk itself)
     * libparse-debianchangelog-perl
     * libvte9 (why do we need this one?)
     * lksctp-tools
     * mysql
     * nodejs
     * novnc
     * pacemaker
     * python-utidylib
     * python-zmq
     * qemu-kvm
     * ruby-ohai
     * rabbitmq-server
     * tasksel
     * xauth
     * xinetd

    Also added a script which helps to reduce dependencies.

    Note: the obtained requirements-deb.txt still contains redundant and
    extraneous entries, this issue will be handled later on.

    Change-Id: Ic0d143a6f6e2eae01b721bae4fd3b6b748558c3b
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/124031
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=01e9f8bd671ed8c99c6108df8cad6318fc4fbdf0
Submitter: Jenkins
Branch: master

commit 01e9f8bd671ed8c99c6108df8cad6318fc4fbdf0
Author: Alexei Sheplyakov <email address hidden>
Date: Wed Sep 24 19:15:19 2014 +0400

    apt-ftparchive*.conf: don't hard code Ubuntu version

    Use the UBUNTU_RELEASE and (the newly introduced) UBUNTU_RELEASE_NUMBER
    variables instead.

    The idea is to have a single place (config.mk) where the Ubuntu (kernel,
    whatever) versions are defined (instead of spreading that data over 10+ files)

    Change-Id: If6b298aa51cdffde6c633c3b7128173148fe95aa
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/127835
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=41d9e60a6c347021ed8db18a637902ca524d020d
Submitter: Jenkins
Branch: master

commit 41d9e60a6c347021ed8db18a637902ca524d020d
Author: Alexei Sheplyakov <email address hidden>
Date: Thu Oct 2 16:56:18 2014 +0400

    automatically guess the Debian-installer's kernel version

    Change-Id: Id64197ce49bc7704e091533c85e83e6ae9d19875
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/133061

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/133286

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/133341

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/133640

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/133648

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133640
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=3eb1af76bd52ed65c2cdd6737a9209cf38af9b3f
Submitter: Jenkins
Branch: master

commit 3eb1af76bd52ed65c2cdd6737a9209cf38af9b3f
Author: Alexei Sheplyakov <email address hidden>
Date: Mon Nov 10 18:41:27 2014 +0300

    mirror-ubuntu: use the OSCI repo for bootstrapping the chroot

    If some of the base packages (like python-minimal) has been rebuilt
    we should use them when bootstrapping the temporary chroot. Otherwise
    the newly created apt repo will contain either only upstream version of
    the package in question, or both the upstream and customized versions.

    Assume that the base package X has been customized. The customized version
    of X is located on the OSCI mirror. Multistrap does not use that mirror, so
    the upstream version of X gets pulled into the apt cache (typically
    /var/apt/cache/archives) during bootstrapping of the chroot. Packages
    depending on X pull its customized version (since the OSCI repo is added
    to the sources.list.d after bootstrapping the chroot). If X is an essential
    package (so other packages are not obliged to declare dependence on X)
    only the upstream version of X gets into the apt repo being created.

    Use the OSCI mirror when bootstrapping the chroot to avoid the problem.

    While at it reduce copy-paste noise in createchroot.mk and make it a little
    bit less obscure.

    Related-bug: #1390416
    Related-bug: #1390044
    Related-bug: #1383641

    Change-Id: Ic418c760a1635d5c72cc108658d412228fb83a62

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133061
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=4009440abce43c25ab0b0e93c4b2386ef1965a98
Submitter: Jenkins
Branch: master

commit 4009440abce43c25ab0b0e93c4b2386ef1965a98
Author: Alexei Sheplyakov <email address hidden>
Date: Thu Nov 6 13:33:18 2014 +0300

    ubuntu: simplify the kernel updates

    At the moment the kernel version to be installed on Ubuntu nodes is hard
    coded in config.mk (and several other files, such as requirements-deb.txt).
    This version must coninside with the version of the kernel used by Debian
    installer (otherwise Debian installer won't be able to load any kernel
    modules which in general breaks the provisioning). As a result updating
    the kernel (say, to incorporate the upstream security fixes) requires
    a lot of manual intervention: downloading the netboot image, finding out
    the exact kernel version it uses, updating config.mk, requirements-deb.txt,
    and so on.

    This patch makes the process much simpler: updating the kernel (to a version
    provided by Ubuntu) boils down to changing the UBUNTU_KERNEL_FLAVOR
    variable (and possilby the UBUNTU_NETBOOT_FLAVOR variable).

     * config.mk: introduce UBUNTU_KERNEL_FLAVOR and UBUNTU_NETBOOT_FLAVOR
       variables. The former specifies which kernel should be installed on
       nodes, and the latter tells which netboot variant should be used.

     * image/ubuntu/module.mk: install the kernel specified by
       the UBUNTU_KERNEL_FLAVOR variable. There's no need to use exactly
       the same version of the kernel as Debian installer does, as image
       based provisioning does not use Debian installer at all.

     * mirror/ubuntu/boot.mk: guess and write the installer kernel version
       into a file which can be include'd by other modules. While at it
       reduce the copy-paste noise.

     * mkrepo.sh: make sure that the kernel (and the correspodning headers)
       (meta-)packages specified by UBUNTU_KERNEL_FLAVOR are included into
       the mirror.

    Change-Id: Ia8db615284644b0e2bcdb70c1c7bb1f4388d5e46
    Related-bug: #1390044
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133286
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=1a1c37fb4b191c1bc61cac214544fd28b7a71e6e
Submitter: Jenkins
Branch: master

commit 1a1c37fb4b191c1bc61cac214544fd28b7a71e6e
Author: Alexei Sheplyakov <email address hidden>
Date: Fri Sep 5 20:34:15 2014 +0400

    ubuntu-1204.preseed: avoid bashisms

    Provisioning fails with kernel 3.13 because of preseed/late_command errors.
    These errors are caused by bashisms (echo -e and similar). The default shell
    in Ubuntu is dash, also bash is not available in the installer environment
    (which uses busybox shell).

    Change-Id: Ia47ffe0584424e67f173ca9467334679071053c7
    Related-bug: #1390044
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/134520

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133648
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=311effec3dcdf878f46e26ce326cc8e71863db02
Submitter: Jenkins
Branch: master

commit 311effec3dcdf878f46e26ce326cc8e71863db02
Author: Alexei Sheplyakov <email address hidden>
Date: Mon Nov 10 17:17:07 2014 +0300

    mirror/ubuntu: fix typo in apt-ftparchive-release.conf

    Related-bug: #1383641
    Change-Id: Ica50a15f63678c43648a16c2c0800aba3a536d4e

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/135677

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133341
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=64177cb8020bec42a6121b5eca2744152f58bd03
Submitter: Jenkins
Branch: master

commit 64177cb8020bec42a6121b5eca2744152f58bd03
Author: Alexei Sheplyakov <email address hidden>
Date: Sat Nov 15 16:09:54 2014 +0300

    Use in-tree openvswitch datapath module if the kernel is new enough

    The major part of the openvswitch datapath kernel module has been merged into
    Linux 3.13. In particular both GRE and VXLAN tunnels are supported (which are
    used by neutron). The out-of-tree module is no longer actively developed.
    In particular, version 1.10 and 1.11 do not support kernel newer than 3.8
    (although Debian maintainers made them work with kernels <= 3.11).

    Therefore use the out-of-tree module only with old kernels (the ones older
    than 3.13)

    Change-Id: Ic0f19bb451cb6a27ba4f17431810301414e711c3
    Related-bug: #1390044
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/134520
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=2102c176152be482e04d072f2af0afc48741ec9d
Submitter: Jenkins
Branch: master

commit 2102c176152be482e04d072f2af0afc48741ec9d
Author: Alexei Sheplyakov <email address hidden>
Date: Thu Nov 6 19:32:41 2014 +0300

    ubuntu: use kernel 3.13 (lts-trusty)

    * packages/deb/debian-boot/01_mount_target_proc.sh: the linux-image
      pre-install script uses /proc/cpuinfo to query CPU features without
      checking if /proc has been mounted. This script mounts proc in /target
      before installing the kernel itself.

    * packages/deb/debian-boot/initrd.mk: include the helper script for
      mounting /proc into initramfs.

    * config.mk: use lts-trusty (Linux 3.13) kernel and the corresponding
      netboot variant (netboot-trusty) by default.

    Change-Id: Id6a2578e36f5f303ffa05ed4d702b119964f62a4
    Related-bug: #1390044
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135677
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=f56012302149b5d69b5cb67a3276acafa30390ae
Submitter: Jenkins
Branch: master

commit f56012302149b5d69b5cb67a3276acafa30390ae
Author: Alexei Sheplyakov <email address hidden>
Date: Sat Nov 15 16:09:54 2014 +0300

    ubuntu: use linux-image-lts-trusy (Linux 3.13)

    Change-Id: If33458231f355b27b36a31185075a686c2513f17
    Related-bug: #1390044
    Related-bug: #1383641

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/137770

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/137770
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=ce3bb61d91e7dafa72dd4281698f343961750d5f
Submitter: Jenkins
Branch: master

commit ce3bb61d91e7dafa72dd4281698f343961750d5f
Author: Vladimir Kozhukalov <email address hidden>
Date: Fri Nov 28 14:38:07 2014 +0300

    Ubuntu: using 3.11 kernel for MOS versions < 6.0

    * Added 'if' condition into ubuntu preseed so as to
      install 3.11 kernel if fuel_version is not equal to 6.0
      trusty kernel if it is.

    Change-Id: I495b1da9d4ed5effcaea32b27cd0cc0b5f527075
    Related-bug: 1390044
    Related-bug: 1383641
    Related-bug: 1348331

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/138298

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/142408

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/142409

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/138298
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=857ba823d5115c20d4d6d31df50698b71a1a5ed1
Submitter: Jenkins
Branch: master

commit 857ba823d5115c20d4d6d31df50698b71a1a5ed1
Author: Alexei Sheplyakov <email address hidden>
Date: Tue Oct 14 14:37:33 2014 +0400

    requirements-deb: skip the obosolete software

    Skip the dependencies of the very old qemu (pre 1.0) versions. Same for
    the pacemaker <= 1.1.6. The download script (mkrepo.sh) picks the
    correct libraries on its own (in fact it has been downloading for a long
    time so none bothered to update the dependencies).

    While at it remove Linux 3.2 packages too. These haven't been actually
    used for a long time (starting from Fuel 4.x?), however it has been
    installed due to a bug in cobbler configuration. That bug has been fixed
    now (see https://review.openstack.org/#/c/135677), so it's time to purge
    the unused Linux 3.2 packages.

    Last but not least this patch helps to update Ubuntu to 14.04 (the packages
    excluded by this patch are not available in the newer versions of Ubuntu).

    Related-bug: #1383641
    blueprint support-ubuntu-trusty
    Change-Id: I4f278d08ff643c7bb90d431c7cee4f79a72d32a1

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/142408
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=1efbeeb69df3f508be3c881b6fbb01ae4e1a56ef
Submitter: Jenkins
Branch: master

commit 1efbeeb69df3f508be3c881b6fbb01ae4e1a56ef
Author: Alexei Sheplyakov <email address hidden>
Date: Wed Dec 17 11:41:57 2014 +0300

    image/ubuntu: suppress the packages' configuration dialogs

    make iso stops during creation of the Ubuntu chroot (intended for image
    based provisioning) and displays packages' configuration dialogs
    (in particular postfix). Force a non-interactive packages installation.
    While at it suppress the warnings regarding an unsupported locale.

    Closes-bug: #1403390
    Related-bug: #1383641
    blueprint support-ubuntu-trusty
    Change-Id: I8c104a41d267e30feea72619ae4ad27b6d59a562

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/142409
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=2c7395dae943e3098660653ecf7c0fb1dfb40d07
Submitter: Jenkins
Branch: master

commit 2c7395dae943e3098660653ecf7c0fb1dfb40d07
Author: Alexei Sheplyakov <email address hidden>
Date: Wed Dec 17 11:47:54 2014 +0300

    image/ubuntu: fix packages installation failures due to missing /proc

    The configuration scripts of some packages (in particular linux-image)
    fail if /proc is not mounted. Mount /proc in the chroot before installing
    packages (and umount it afterwards).

    Related-bug: #1383641
    Closes-bug: #1403435
    blueprint support-ubuntu-trusty
    Change-Id: I6846b7d7107260125291efee27f6205b8e48969f

[Anastasia Palkina (apalkina)]

Verified on ISO #304

"build_id": "2015-04-10_22-54-31", "ostf_sha": "c2a76a60ec4ebbd78e508216c2e12787bf25e423", "build_number": "304", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-04-10_22-54-31", "ostf_sha": "c2a76a60ec4ebbd78e508216c2e12787bf25e423", "build_number": "304", "api": "1.0", "nailgun_sha": "69547a71abb4696df7e6f44b1f7864b0535f2df7", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "9208ff4a08dcb674ce2df132399a5aa3ddfac21c", "astute_sha": "d96a80b63198a578b2c159edbd76048819039eb0", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "8daac234aea6ac0a98f27871deec039f74f6fdab", "fuellib_sha": "867028fe78837dc2e4635a2cbb976782856964d0"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "69547a71abb4696df7e6f44b1f7864b0535f2df7", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "9208ff4a08dcb674ce2df132399a5aa3ddfac21c", "astute_sha": "d96a80b63198a578b2c159edbd76048819039eb0", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "8daac234aea6ac0a98f27871deec039f74f6fdab", "fuellib_sha": "867028fe78837dc2e4635a2cbb976782856964d0"