ISO build fails: problem downloading RPM packages

[Alexei Sheplyakov]

The error message is:

http://osci-obs.vm.mirantis.net:82/centos-fuel-6.1-stable/centos/noarch/python-mako-0.9.1-5.el6.noarch.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=fuel clean metadata
Trying other mirror.
Could not download/verify pkg python-mako-0.9.1-5.el6.noarch: failure: noarch/python-mako-0.9.1-5.el6.noarch.rpm from fuel: [Errno 256] No more mirrors to try.

Apparently the problem is caused by yumdownloader bug [1]: downloading
a package fails if that package is already present in the destination
directory if the package (i.e. it has been downloaded as a dependency.
The yumdownloader bug has been fixed recently, however, we can't require
the user to install the bleeding edge version of yum, and need a work
around.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1004089

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/147817

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/147817
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=0d6abfd4cdfdb71aa2005c050d93177a8d77dca8
Submitter: Jenkins
Branch: master

commit 0d6abfd4cdfdb71aa2005c050d93177a8d77dca8
Author: Alexei Sheplyakov <email address hidden>
Date: Thu Jan 15 22:19:59 2015 +0300

    mirror/centos: improve handling of the conflicting packages

    current version of mirror/centos/repo.mk has a couple of serious bugs:

    - error handling is flaky and incomplete, that is, the exit status of
      yumdownloader is ignored (due to misuse of shell pipeline). Instead
      the errors are detected by parsing the yumdownloader log which is
      unreliable (and adds a lot of extra code for no gain at all).

    - Inconsistent dependencies/conflicts are handled in an extremely flaky
      manner: the error code of yumdownloader is ignored, instead the total
      number of conflicts (or rather the number of "conflicts with" strings in
      the yumdownloader log) is compared with some magic number. No attempts
      are made to check which packages were conflicting, also the magic number
      should be adjusted whenever a new package is added into the repository
      (requirements-rpm.txt).

    - Current code uses the default RPM database and thus is unusable on RPM
      based systems.

    To solve these problems

    - process packages one by one, ask yumdownloader to print the URLs
    - use a separate RPM database to not interfere with the system one
    - use the error code returned by yumdownloader to check for errors
    - download the actual packages with wget
    - make sure the independent tasks can run in parallel

    As a side effect yum bug https://bugzilla.redhat.com/show_bug.cgi?id=1004089
    no longer affects building the ISO

    Closes-Bug: #1411526
    Partial-Bug: #1396618

    Change-Id: I9e7bd1461ef0a28a28397c1ba3cb56e8409f5e49