make iso error because of missing range support at mirror.fuel-infra.org

[Daniele Pizzolli]

Sometimes make iso fails because there are partially downloaded files
by yumdownloader.

The error message is quite obscure and there is no easy fix.

Here is an snippet from the log:

    # Yumdownloader workaround number four:
    # yumdownloader should fail if some errors appears
    test `grep "Errno" /vagrant/fuel-main-dev/build/mirror/centos/yumdownloader.log | wc -l` = 0
    make: *** [/vagrant/fuel-main-dev/build/mirror/centos/yum.done] Error 1

A detailed check reveals:

    $ grep "Errno" /vagrant/fuel-main-dev/build/mirror/ceyntos/yumdownloader.log
    http://mirror.fuel-infra.org/fwm/5.1.1/centos/os/x86_64/Packages/logrotate-3.8.7-1.el6.x86_64.rpm: [Errno 14] HTTP Error 416 : http://mirror.fuel-infra.org/fwm/5.1.1/centos/os/x86_64/Packages/logrotate-3.8.7-1.el6.x86_64.rpm
    Could not download/verify pkg logrotate-3.8.7-1.el6.x86_64: failure: Packages/logrotate-3.8.7-1.el6.x86_64.rpm from proprietary: [Errno 256] No more mirrors to try.

The 'HTTP Error 416' is caused by the server not supporting the
requests with the Range header made by yumdownloader.

I do not know if there is a configuration option to force a full
download for yumdownloader as a workaround on the client.

A proper solution is to support range headers on the server, as
usually the mirrors do for saving bandwidth.

A really naive workaround: find and delete the partial downloaded files:

    grep "Errno" /vagrant/fuel-main-dev/build/mirror/centos/yumdownloader.log | cut -d : -f 4- | grep http | awk -F / '{print $NF}' | xargs -n1 find . -iname | xargs rm

The problem is simple to reproduce, truncate a random rpm already
downloaded and restart the make iso:

    rm ./build/mirror/centos/repo.done
    truncate -s 100k ./local_mirror/centos/os/x86_64/Packages/logrotate-3.8.7-1.el6.x86_64.rpm
    make iso

[Dmitry Burmistrov (dburmistrov)]

Hello, Daniele.
Please use "make deep_clean" before each run "make iso".

[Daniele Pizzolli (daniele-pizzolli)]

> Please use "make deep_clean" before each run "make iso".

Hello Dmitry,

this is a workaround for the issue, but it will force the download of
around 2.5GB of data and takes more than 1h with an high-speed
connection. It is really sub-optimal.

[Daniele Pizzolli (daniele-pizzolli)]

Dmitry Burmistrov <email address hidden> writes:

> *** This bug is a duplicate of bug 1376694 ***
> https://bugs.launchpad.net/bugs/1376694
>
> ** This bug has been marked a duplicate of bug 1376694
> OBS rebuilds a package, but don't increase its version number

Hello Dmitry,
I do not think that the two bugs are related.

Why do you think so?

Best,
Daniele

[Dmitry Burmistrov (dburmistrov)]

This error appears when we update package at yum repository without changing its name+version+revision tags.
Wokraround is "make deep_clean" which performs cleaning operations including "yum clean all"