Retries for getting packages via http are not working in fuel-mirror

[Alexandr Kostrikov]

Expected:
When fuel-mirror faces 404, it should retry it

Actual:
Only such stacktrace is in logs:
return self.opener.open(request)\n', ' File "/usr/lib64/python2.7/urllib2.py", line 410, in open\n', ' response = meth(req, response)\n', ' File "/usr/lib64/python2.7/urllib2.py", line 523, in http_response\n', " 'http', request, response, code, msg, hdrs)\n", ' File "/usr/lib64/python2.7/urllib2.py", line 448, in error\n', ' return self._call_chain(*args)\n', ' File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain\n', ' result = func(*args)\n', ' File "/usr/lib64/python2.7/urllib2.py", line 531, in http_error_default\n', ' raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)\n', 'HTTPError: HTTP Error 404: Not Found\n',

[Ivan Ponomarev (ivanzipfer)]

Please provide fuel version

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/273972

[Ihor Kalnytskyi (ikalnytskyi)]

We have a similar issue here - bug 1524855.

I believe we should fix it not by adding retries, but by fixing mirrors. Moreover, apt-get/yum doesn't make retries for 404. So I insist to do not introduce workarounds, but by fixing the original problem.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/273972
Committed: https://git.openstack.org/cgit/openstack/fuel-mirror/commit/?id=3677dc62e40d87811a4ed217958ef848a65634eb
Submitter: Jenkins
Branch: master

commit 3677dc62e40d87811a4ed217958ef848a65634eb
Author: Bulat Gaifullin <email address hidden>
Date: Fri Jan 29 13:06:59 2016 +0300

    Fixed retries for server errors

    The code does not properly handle situation when server returns http error.
    Also added option 'retries_delay' to specify timeout between retries
    Also disabled retries for 404 code

    Change-Id: Ie5b2e720d453ef6e398400d27e631607b56e2937
    Closes-Bug: #1539025

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/275254

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/275254
Committed: https://git.openstack.org/cgit/openstack/fuel-mirror/commit/?id=6f8e1a005446eb49981fe93bd9a67ca944c0a763
Submitter: Jenkins
Branch: stable/8.0

commit 6f8e1a005446eb49981fe93bd9a67ca944c0a763
Author: Bulat Gaifullin <email address hidden>
Date: Fri Jan 29 13:06:59 2016 +0300

    Fixed retries for server errors

    The code does not properly handle situation when server returns http error.
    Also added option 'retries_delay' to specify timeout between retries
    Also disabled retries for 404 code

    Change-Id: Ie5b2e720d453ef6e398400d27e631607b56e2937
    Closes-Bug: #1539025
    (cherry picked from commit 3677dc62e40d87811a4ed217958ef848a65634eb)

[Alexandr Kostrikov (akostrikov-mirantis)]

There were no reproduces in swarm job.
Fix has not been broken since 8.0