Upgrades from Ocata to Pike (containerized) missing container upload step

Bug #1710938 reported by Emilien Macchi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Emilien Macchi

Bug Description

By testing upgrades from Ocata (baremetal) to Pike (containerized) in https://review.openstack.org/#/c/461000/ - we found out that the upgrade process wasn't working well, the local registry is missing containers.

We think it could be a regression in https://github.com/openstack/tripleo-quickstart-extras/commit/90e703768326622eab1cf8dfa80daddccc3f88c8 and Wes is currently investigating https://review.openstack.org/#/c/493972 to see if uploading the containers actually help.

I'm setting Critical + alert on this one because it should be our first priority to test upgrades at this stage of the cycle.

Tags: alert ci upgrade
tags: added: alert upgrade
Changed in tripleo:
assignee: nobody → wes hayutin (weshayutin)
status: Triaged → In Progress
Revision history for this message
Steve Baker (steve-stevebaker) wrote :
Revision history for this message
Emilien Macchi (emilienm) wrote :

so I confirm https://review.openstack.org/#/c/493972 will help though the upgrade process is not working yet, hitting timeout in infra, because it pulls *all* containers into the local registry (it takes 30 minutes).

We need to pull only what we need. I'll open a new bug for that.

Changed in tripleo:
assignee: wes hayutin (weshayutin) → Emilien Macchi (emilienm)
Revision history for this message
Steven Hardy (shardy) wrote :

> because it pulls *all* containers into the local registry (it takes 30 minutes).

If we have an infra cache available, would it be quicker to *not* pull the images to the undercloud registry, and just pull them (once) to the subnode we're testing?

For single node tests I'm not sure we gain anything by using the undercloud cache, as we end up copying the same images to two nodes, and these copies are serialized (container upload, and the pull we do during the deploy?)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-quickstart (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/494525

Revision history for this message
Jiří Stránský (jistr) wrote :

The slowness could be caused partially by downloading images off dockerhub instead of using the local mirror. I've proposed a fix at https://review.openstack.org/494525

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-quickstart-extras (master)

Reviewed: https://review.openstack.org/493972
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=3e24d07c2a50efb4597e896334f0ba3343e1d7c8
Submitter: Jenkins
Branch: master

commit 3e24d07c2a50efb4597e896334f0ba3343e1d7c8
Author: Wes Hayutin <email address hidden>
Date: Tue Aug 15 14:12:49 2017 -0400

    upload containers to undercloud in upgrade scenario

    The upgrade container scenarios are not working atm.
    At first glance it appears the upload command is missing after the
    containers are downloaded to the undercloud.

    Closes-Bug: #1710938
    Change-Id: I0deb57083aeddc2e72e35b0662f972473d99f31c

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-quickstart (master)

Change abandoned by Jiri Stransky (<email address hidden>) on branch: master
Review: https://review.openstack.org/494525
Reason: Moved elsewhere at https://review.openstack.org/#/c/494644

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-quickstart-extras 2.1.1

This issue was fixed in the openstack/tripleo-quickstart-extras 2.1.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.