fuel-createmirror broken due old Docker

Bug #1528498 reported by Roman Sokolkov on 2015-12-22
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Undecided
Unassigned
6.1.x
High
Rodion Tikunov
7.0.x
High
MOS Maintenance
8.0.x
Undecided
Fuel Library (Deprecated)

Bug Description

Script "fuel-createmirror" fails to prepare Ubuntu mirror.
It uses "docker pull ubuntu:latest" which is fails because
Docker hub officially stopped support of Docker<=1.5.
https://blog.docker.com/2015/10/docker-hub-deprecation-1-5/
Currently Fuel 7.0 has docker 1.4.1 which is affected by this restriction.

Environment: MOS 7.0

How to reproduce:
1) Deploy Fuel 7.0 master node
2) Execute "docker pull ubuntu:latest"

Expected result:
Successfull download of Ubuntu image

Actual result:
Pulling repository ubuntu
FATA[0002] Could not reach any registry endpoint

Dmitry Pyzhov (dpyzhov) on 2015-12-23
tags: added: area-python

this bug is deprecated, because the fuel-createmirror was replaced by fuel-mirror and fuel-mirror does not use docker.

Dmitry Pyzhov (dpyzhov) on 2015-12-23
no longer affects: fuel/8.0.x
Roman Sokolkov (rsokolkov) wrote :

Why you decided to not fix for 6.1/7.0?

What is the workaround for these versions?

Victor Denisov (vdenisov) wrote :

Maybe this bug is not in fuel itself, but in the documentation. If fuel-createmirror is deprecated then this page https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#setting-up-local-mirrors
should not recommend using it.

Roman Rufanov (rrufanov) wrote :

Customer found on 7.0 - need workaround or fix asap.

tags: added: customer-found support
Alex Schultz (alex-schultz) wrote :

We need a workaround or documentation on how to pull in fuel-mirror for 6.1 or 7.0 if that is way to handle it. Since we shipped fuel-createmirror for 6.1 and 7.0 we need to continue to support it or provide an alternate way to pull down the required docker image.

Miroslav Anashkin (manashkin) wrote :

Raised to Critical.
All 6.1 and 7.0 users, who just deployed the new master node are unable to deploy new Ubuntu environments since Fuel downloads Ubuntu image with Docker.

Alex Schultz (alex-schultz) wrote :

We cannot upgrade docker on previous versions of Fuel. So as a solution we will need to store an ubuntu docker image on a mirror and update the mirror scripts to pull down the image and load it into the Fuel master to allow for the mirror process to complete.

Alex Schultz (alex-schultz) wrote :

Currently as a work around until the fuel-createmirror script can be updated, you can load the ubuntu image into docker by pulling it down:
1) wget http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz -O /tmp/ubuntu.trusty.tar.xz
2) docker load -i /tmp/ubuntu.trusty.tar.xz

Once it's loaded, you can run fuel-createmirror normally and it should proceed.

Related fix proposed to branch: 6.1
Change author: Alex Schultz <email address hidden>
Review: https://review.fuel-infra.org/15980

Dmitry Borodaenko (angdraug) wrote :

Now that the workaround for the bug is available and documented (see comment #8) this bug can be downgraded to High severity.

Roman Rufanov (rrufanov) wrote :

Added 8.0 milestone to check if the same issue exists there.

Denis Meltsaykin (dmeltsaykin) wrote :

Reassigned to the Fuel Library team for 8.0 as 8.0 is not released yet and Maintenance Team doesn't triage bugs in non-released products.

Alex Schultz (alex-schultz) wrote :

Doesn't affect 8.0 because it has a newer version of docker

Reviewed: https://review.openstack.org/264457
Committed: https://git.openstack.org/cgit/openstack/fuel-mirror/commit/?id=20f88c67c4e83433bc0e539572aa3f888a292177
Submitter: Jenkins
Branch: stable/7.0

commit 20f88c67c4e83433bc0e539572aa3f888a292177
Author: Alex Schultz <email address hidden>
Date: Wed Jan 6 16:18:28 2016 -0700

    Update deb-mirror to download ubuntu docker image

    This change updates the deb-mirror script download the ubuntu docker
    image from mirror.fuel-infra.org because the version of docker provided
    by Fuel 7.0 is no longer able to pull it down from the docker hub.

    Change-Id: I5f7d95f4288520db4dc49557540ff1e8b569c2bb
    Related-Bug: #1528498

meher (meher4india) wrote :

Executing the downloaded tar file is failing

[root@fuel tmp]# wget http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz
--2016-02-17 03:08:38-- http://mirror.fuel-infra.org/docker/ubuntu.trusty.tar.xz
...
HTTP request sent, awaiting response... 200 OK
Length: 1969 (1.9K) [text/html]
Saving to: “ubuntu.trusty.tar.xz”

100%[==================================================================================================================================================================================================================================================================================================================================>] 1,969 --.-K/s in 0s

2016-02-17 03:08:40 (454 MB/s) - “ubuntu.trusty.tar.xz” saved [1969/1969]

[root@fuel tmp]# docker load -i /tmp/ubuntu.trusty.tar.xz
FATA[0000] Error: Untar exit status 1 archive/tar: invalid tar header
[root@fuel tmp]# ls /tmp/
atop.d ks-script-2_ckpa ks-script-6Y7kHn ks-script-6Y7kHn.log logrotate ubuntu.trusty.tar.xz yum.log
[root@fuel tmp]# tar tvf /tmp/ubuntu.trusty.tar.xz
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
[root@fuel tmp]#

Alex Schultz (alex-schultz) wrote :

@meher, check the contents of that file. It looks like it didn't download correctly as it says it was text/html and only 1.9k. I tested it and the url is still good so check to see what was downloaded. It should be ~41MB

meher (meher4india) wrote :

Company security tricked me into downloading that file and it didn't. Used a different way to download the file. Thank you Alex!

Fix proposed to branch: 6.1
Change author: Rodion Tikunov <email address hidden>
Review: https://review.fuel-infra.org/18960

Change abandoned by Rodion Tikunov <email address hidden> on branch: 6.1
Review: https://review.fuel-infra.org/18960
Reason: This fix already proposed there https://review.fuel-infra.org/#/c/15980/

Alex Schultz (alex-schultz) wrote :

@dbelyaniov, I checked the logs and there is no indication that the issue is related to this bug. From the mirror-sync log:

Tue Apr 5 18:37:20 UTC 2016 * INFO: Fetching component 'multiverse' binary package lists
Tue Apr 5 18:37:22 UTC 2016 * FATAL: Failed to find Packages file at dists/trusty/multiverse/binary-amd64

Please indicate why you believe that failure to be this bug.

Reviewed: https://review.fuel-infra.org/15980
Submitter: Vitaly Sedelnik <email address hidden>
Branch: 6.1

Commit: 952ba2772d627789c28fcefa1eb2a3e5e1ef46fe
Author: Alex Schultz <email address hidden>
Date: Wed Jan 6 22:54:19 2016

Update deb-mirror to download ubuntu docker image

This change updates the deb-mirror script download the ubuntu docker
image from mirror.fuel-infra.org because the version of docker provided
by Fuel 6.1 is no longer able to pull it down from the docker hub.

Change-Id: I51393cf1a2cc98b5a06d3e6d551336b8b51aaa5b
Related-Bug: #1528498

tags: added: on-verification
Alexander Gromov (agromov) wrote :

Verified on MOS 6.1 + mu6.

without updates:
http://paste.openstack.org/show/497099/

with updates:
http://paste.openstack.org/show/497100/

Note:
fuel-createmirror command is fixed. But command "docker pull ubuntu:latest" from description continues to fail (it is expected behavior).

tags: removed: on-verification
tags: added: on-verification
Alexander Gromov (agromov) wrote :

Verified on 7.0 + mu4.

tags: removed: on-verification
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers