fuel-createmirror broken due old Docker

[Roman Sokolkov]

Script "fuel-createmirror" fails to prepare Ubuntu mirror.
It uses "docker pull ubuntu:latest" which is fails because
Docker hub officially stopped support of Docker<=1.5.
https://blog.docker.com/2015/10/docker-hub-deprecation-1-5/
Currently Fuel 7.0 has docker 1.4.1 which is affected by this restriction.

Environment: MOS 7.0

How to reproduce:
1) Deploy Fuel 7.0 master node
2) Execute "docker pull ubuntu:latest"

Expected result:
Successfull download of Ubuntu image

Actual result:
Pulling repository ubuntu
FATA[0002] Could not reach any registry endpoint

[Bulat Gaifullin (bulat.gaifullin)]

this bug is deprecated, because the fuel-createmirror was replaced by fuel-mirror and fuel-mirror does not use docker.

[Roman Sokolkov (rsokolkov)]

Why you decided to not fix for 6.1/7.0?

What is the workaround for these versions?

[Victor Denisov (vdenisov)]

Maybe this bug is not in fuel itself, but in the documentation. If fuel-createmirror is deprecated then this page https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#setting-up-local-mirrors
should not recommend using it.

[Roman Rufanov (rrufanov)]

Customer found on 7.0 - need workaround or fix asap.

[Alex Schultz (alex-schultz)]

We need a workaround or documentation on how to pull in fuel-mirror for 6.1 or 7.0 if that is way to handle it. Since we shipped fuel-createmirror for 6.1 and 7.0 we need to continue to support it or provide an alternate way to pull down the required docker image.

[Miroslav Anashkin (manashkin)]

Raised to Critical.
All 6.1 and 7.0 users, who just deployed the new master node are unable to deploy new Ubuntu environments since Fuel downloads Ubuntu image with Docker.

[Alex Schultz (alex-schultz)]

We cannot upgrade docker on previous versions of Fuel. So as a solution we will need to store an ubuntu docker image on a mirror and update the mirror scripts to pull down the image and load it into the Fuel master to allow for the mirror process to complete.

[Alex Schultz (alex-schultz)]

Currently as a work around until the fuel-createmirror script can be updated, you can load the ubuntu image into docker by pulling it down:
1) wget http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz -O /tmp/ubuntu.trusty.tar.xz
2) docker load -i /tmp/ubuntu.trusty.tar.xz

Once it's loaded, you can run fuel-createmirror normally and it should proceed.

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: 6.1
Change author: Alex Schultz <email address hidden>
Review: https://review.fuel-infra.org/15980

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/7.0
Review: https://review.openstack.org/264457

[Dmitry Borodaenko (angdraug)]

Now that the workaround for the bug is available and documented (see comment #8) this bug can be downgraded to High severity.

[Roman Rufanov (rrufanov)]

Added 8.0 milestone to check if the same issue exists there.

[Denis Meltsaykin (dmeltsaykin)]

Reassigned to the Fuel Library team for 8.0 as 8.0 is not released yet and Maintenance Team doesn't triage bugs in non-released products.

[Alex Schultz (alex-schultz)]

Doesn't affect 8.0 because it has a newer version of docker

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/264457
Committed: https://git.openstack.org/cgit/openstack/fuel-mirror/commit/?id=20f88c67c4e83433bc0e539572aa3f888a292177
Submitter: Jenkins
Branch: stable/7.0

commit 20f88c67c4e83433bc0e539572aa3f888a292177
Author: Alex Schultz <email address hidden>
Date: Wed Jan 6 16:18:28 2016 -0700

    Update deb-mirror to download ubuntu docker image

    This change updates the deb-mirror script download the ubuntu docker
    image from mirror.fuel-infra.org because the version of docker provided
    by Fuel 7.0 is no longer able to pull it down from the docker hub.

    Change-Id: I5f7d95f4288520db4dc49557540ff1e8b569c2bb
    Related-Bug: #1528498

[meher (meher4india)]

Executing the downloaded tar file is failing

[root@fuel tmp]# wget http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz
--2016-02-17 03:08:38-- http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz
...
HTTP request sent, awaiting response... 200 OK
Length: 1969 (1.9K) [text/html]
Saving to: “ubuntu.trusty.tar.xz”

100%[==================================================================================================================================================================================================================================================================================================================================>] 1,969 --.-K/s in 0s

2016-02-17 03:08:40 (454 MB/s) - “ubuntu.trusty.tar.xz” saved [1969/1969]

[root@fuel tmp]# docker load -i /tmp/ubuntu.trusty.tar.xz
FATA[0000] Error: Untar exit status 1 archive/tar: invalid tar header
[root@fuel tmp]# ls /tmp/
atop.d ks-script-2_ckpa ks-script-6Y7kHn ks-script-6Y7kHn.log logrotate ubuntu.trusty.tar.xz yum.log
[root@fuel tmp]# tar tvf /tmp/ubuntu.trusty.tar.xz
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
[root@fuel tmp]#

[Alex Schultz (alex-schultz)]

@meher, check the contents of that file. It looks like it didn't download correctly as it says it was text/html and only 1.9k. I tested it and the url is still good so check to see what was downloaded. It should be ~41MB

[meher (meher4india)]

Company security tricked me into downloading that file and it didn't. Used a different way to download the file. Thank you Alex!

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 6.1
Change author: Rodion Tikunov <email address hidden>
Review: https://review.fuel-infra.org/18960

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Rodion Tikunov <email address hidden> on branch: 6.1
Review: https://review.fuel-infra.org/18960
Reason: This fix already proposed there https://review.fuel-infra.org/#/c/15980/

[Dmitry Belyaninov (dbelyaninov)]

Reproduced on SWARMS:

https://mirantis.testrail.com/index.php?/tests/view/4044652

[Alex Schultz (alex-schultz)]

@dbelyaniov, I checked the logs and there is no indication that the issue is related to this bug. From the mirror-sync log:

Tue Apr 5 18:37:20 UTC 2016 * INFO: Fetching component 'multiverse' binary package lists
Tue Apr 5 18:37:22 UTC 2016 * FATAL: Failed to find Packages file at dists/trusty/multiverse/binary-amd64

Please indicate why you believe that failure to be this bug.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/15980
Submitter: Vitaly Sedelnik <email address hidden>
Branch: 6.1

Commit: 952ba2772d627789c28fcefa1eb2a3e5e1ef46fe
Author: Alex Schultz <email address hidden>
Date: Wed Jan 6 22:54:19 2016

Update deb-mirror to download ubuntu docker image

This change updates the deb-mirror script download the ubuntu docker
image from mirror.fuel-infra.org because the version of docker provided
by Fuel 6.1 is no longer able to pull it down from the docker hub.

Change-Id: I51393cf1a2cc98b5a06d3e6d551336b8b51aaa5b
Related-Bug: #1528498

[Alexander Gromov (agromov)]

Verified on MOS 6.1 + mu6.

without updates:
http://paste.openstack.org/show/497099/

with updates:
http://paste.openstack.org/show/497100/

Note:
fuel-createmirror command is fixed. But command "docker pull ubuntu:latest" from description continues to fail (it is expected behavior).

[Alexander Gromov (agromov)]

Verified on 7.0 + mu4.