job that test ceph in periodic pipeline for stable branches are getting the wrong configuration

Bug #1817688 reported by Gabriele Cerami on 2019-02-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
Gabriele Cerami

Bug Description

https://review.openstack.org/637196 Introduced a bug for the stable periodic jobs.

Affected jobs:

- periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-rocky
- periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-rocky
- periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-queens
- periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-queens

An example at

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-rocky/4b9ccbc/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-26_07_58_32

error:

ceph_disk.main.Error: Error: Device /dev/loop3 is in use by a device-mapper mapping (dm-crypt?): dm-0,dm-1,dm-2

Master jobs scenario001 and 004 are affected currently waiting for the migration patches to bluestore

As yatin commented in periodic jobs the branch is always master, and stable branches get a configuration that is not compatible with the branch.

The solution would be to pass the release variable, but it's not available at that stage in the playbook.

We need to make the release variable available earlier in the process.

description: updated
wes hayutin (weshayutin) on 2019-02-26
tags: removed: ci promotion-blocker quickstart
description: updated
wes hayutin (weshayutin) on 2019-02-26
tags: added: promotion-blocker
tags: added: ci
John Fulton (jfulton-org) wrote :

https://review.openstack.org/#/c/547682 can be backported to queens and rocky and they'll be green again. both use ceph-ansible 3.2 which supports bluestore.

pike EOLs in 5 days. i'll start backports and we can vote on this way forward or others.

https://releases.openstack.org/

John Fulton (jfulton-org) wrote :

Plan:
- master 001/004 are green and should vote; merge [0]
- confirm rocky/queens 001/004 turn green from [1][2]
- enable voting 001/004 in stable rocky/queens/pike
- stable will be voting again but pike jobs will be broken as it cannot use bluestore
- pike EOLs in five days [3] so a separate thread will cover disabling it

[0] https://review.openstack.org/#/c/638364
[1] https://review.openstack.org/#/c/639351
[2] https://review.openstack.org/#/c/639352
[3] https://releases.openstack.org

Reviewed: https://review.openstack.org/639351
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=61911edb5deafd2d28a09a540400fb524fd46ab9
Submitter: Zuul
Branch: stable/rocky

commit 61911edb5deafd2d28a09a540400fb524fd46ab9
Author: John Fulton <email address hidden>
Date: Fri Feb 23 17:10:08 2018 -0500

    Switch scenario00{1,4}-multinode-containers to Ceph bluestore

    Modify scenario00{1,4}-multinode to use Ceph's bluestore in place
    filestore. Bluestore is the default deployment method as of version
    3.2 of ceph-ansible so we should test it in CI. Use pre-created
    lvm_volumes parameter to avoid issue with 'ceph-volume batch'
    mode which does not work on loopback devices.

    blueprint: bluestore
    Depends-On: I747ac3dca5afdc91538da40b9ed45591ac8d1662
    Fixes-Bug: #1817688
    (cherry-picked from commit e3f697df6e1ff57b6f9e2cf3d455984dbf1faf84)

    Change-Id: Id2658ae814b580971d559af616b8ba034dff681b

tags: added: in-stable-rocky
tags: added: in-stable-queens

Reviewed: https://review.openstack.org/639352
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a89cd6b19b7bd4178e052aa4cf0ef7f59145b708
Submitter: Zuul
Branch: stable/queens

commit a89cd6b19b7bd4178e052aa4cf0ef7f59145b708
Author: John Fulton <email address hidden>
Date: Fri Feb 23 17:10:08 2018 -0500

    Switch scenario00{1,4}-multinode-containers to Ceph bluestore

    Modify scenario00{1,4}-multinode to use Ceph's bluestore in place
    filestore. Bluestore is the default deployment method as of version
    3.2 of ceph-ansible so we should test it in CI. Use pre-created
    lvm_volumes parameter to avoid issue with 'ceph-volume batch'
    mode which does not work on loopback devices.

    blueprint: bluestore
    Depends-On: I747ac3dca5afdc91538da40b9ed45591ac8d1662
    Fixes-Bug: #1817688
    (cherry-picked from commit e3f697df6e1ff57b6f9e2cf3d455984dbf1faf84)

    Change-Id: Id2658ae814b580971d559af616b8ba034dff681b

wes hayutin (weshayutin) wrote :

waiting to verify w/ queens and rocky jobs.

Changed in tripleo:
status: Triaged → Fix Released
status: Fix Released → Triaged

Change abandoned by John Fulton (<email address hidden>) on branch: master
Review: https://review.openstack.org/641016
Reason: https://review.openstack.org/#/c/641479 merged so multinode master will work with filestore. We can move it to bluestore in the future if we want

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers