scenario012-standalone.yaml failed to create vms for ironic to introspect

Bug #1875681 reported by Sandeep Yadav
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Medium
Sandeep Yadav

Bug Description

Description
===========

scenario012-standalone.yaml failed to create vms for ironic to introspect

scenario012-standalone.yaml is a scenario where we test something close to ironic in overcloud. In this scenario we create two qemu vms and introspect with ironic service running on standalone node.

Related blueprint: https://blueprints.launchpad.net/tripleo/+spec/ironic-overcloud-ci.

In the past, we got that ironic overcloud job working with multinode job and then it broke around the time it was switched to a standalone job.

tripleo-ci-centos-7-scenario012-multinode-oooq-container -> tripleo-ci-centos-7-scenario012-standalone -> tripleo-ci-centos-8-scenario012-standalone

Error Snippet:

~~~
+ /tmp/ironic/devstack/tools/ironic/scripts/create-node.sh:main:L116: virsh vol-create-as default node1.qcow2 11G --format qcow2
/tmp/ironic/devstack/tools/ironic/scripts/create-node.sh: line 116: virsh: command not found
~~~

https://39256e99c33aa5dd1262-97c7700d8ad654a1c78ac6d1f50b424d.ssl.cf2.rackcdn.com/720787/2/check/tripleo-ci-centos-8-scenario012-standalone/c13ab20/logs/undercloud/home/zuul/create-node-1.log

This was working previously in centos7 job:-

~~~
https://532a20ff867a14f7a3d5-67e15cc69bd7500d8d48cf199d18d0c8.ssl.cf1.rackcdn.com/716850/2/check/tripleo-ci-centos-7-scenario012-standalone/80919de/logs/undercloud/home/zuul/create-node-1.log
~~~

This is because a shell script in vbmc_setup.yaml have many issues faced after migration from centos7 to centos8
https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/vbmc_setup.yaml#L19-L70

Changed in tripleo:
assignee: nobody → Sandeep Yadav (sandeepyadav93)
Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

Related fix posted in ironic

https://review.opendev.org/#/c/724071/

Fix configure-vm.py and xml file for centos8

Fixing configure-vm script and vm.xml with this patch because for
latest OS like centos8 - older python version and older qemu machine
type is causing issue.

* For configure-vm.py - Replacing python with python3 in environment
path as python3 is the default in many latest OS.

* For vm.xml - Replacing Machine type from pc-1.0 to pc.
- pc-1.0 is old and unsupported machine type for Centos8
- Moving machine type from pc-1.0 to pc will help as pc have alias to
supported qemu machine type

Related-Bug: #1875681
Change-Id: Icd7e98ceb8859f730a32981a150f2730bde9836b

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/724103

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/724103
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=84e016edb39f352c3f36d07f94831fe3a0874587
Submitter: Zuul
Branch: master

commit 84e016edb39f352c3f36d07f94831fe3a0874587
Author: Sandeep Yadav <email address hidden>
Date: Tue Apr 28 13:35:19 2020 -0400

    Fix vbmc_setup.yaml for c8 standalone

    Fixing vbmc_setup.yaml for c8 scenario012-standalone job with this
    patch.

    vbmc_setup.yaml was originally written for c7 multinode[0] to install
    virtual BMC and a libvirt domain on the controller for ironic to
    manage. In scenario012 we are trying to test something close to ironic
    in overcloud(Blueprint[1]).

    In the past, We have moved scenario012 mutinode to standalone job and
    reused vbmc_setup.yaml.

    The shell script in vbmc_setup.yaml need following modification as its
    now a C8 standalone job:-

    * Added 'set -e' so that script exits immediately if a command exits
    with a non-zero status so it would result in deployment failure.
    Currently deployment is passing even if some commands in the bash
    script are failing. Addition of 'set -e' will correct this behavior.
    * Replaced yum with dnf as yum not present in Centos8.
    * No need to override resolv.conf entry as standalone job already
    configure necessary nameserver entries.
    * No need to install additional repos as as standalone job already
    configure necessary repos.
    * When vbmc_setup.yaml was introduced intention was to do non-kvm
    emulation[2] and that was the reason qemu-system-x86 and qemu-kvm-ev
    was pulled from epel, but for c8 these packages are no longer available
    in repos [3] including epel. We can use qemu-kvm package instead and
    we will no longer need epel repos.
    * create-node.sh command option to use qemu-system have to be removed
    as qemu-system-x86 package is missing and we are using qemu-kvm
    instead.
    * Temporary workaround[4] to upgrade ansible in the neutron container
    can be removed as ansible version in container is now v2.5.8+.
    Test patch for tripleo-ci-centos-8-scenario012-standalone is here[5].

    [0] https://review.opendev.org/#/c/485261/
    [1]
    https://blueprints.launchpad.net/tripleo/+spec/ironic-overcloud-ci
    [2]
    https://review.opendev.org/#/c/485261/29/ci/common/vbmc_setup.yaml
    [3] https://bugzilla.redhat.com/show_bug.cgi?id=1715806
    [4] https://review.opendev.org/#/c/579603/
    [5] https://review.opendev.org/#/c/724119/

    Change-Id: I9e9d63a7ef2ef538f3c072e3f4a96ec25d7dc5f7
    Closes-Bug: #1875681

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/741589

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/741589
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=69d63e0bc13efbfaa990408afee984673bde49bb
Submitter: Zuul
Branch: stable/train

commit 69d63e0bc13efbfaa990408afee984673bde49bb
Author: Sandeep Yadav <email address hidden>
Date: Tue Apr 28 13:35:19 2020 -0400

    Fix vbmc_setup.yaml for c8 standalone

    Fixing vbmc_setup.yaml for c8 scenario012-standalone job with this
    patch.

    vbmc_setup.yaml was originally written for c7 multinode[0] to install
    virtual BMC and a libvirt domain on the controller for ironic to
    manage. In scenario012 we are trying to test something close to ironic
    in overcloud(Blueprint[1]).

    In the past, We have moved scenario012 mutinode to standalone job and
    reused vbmc_setup.yaml.

    The shell script in vbmc_setup.yaml need following modification as its
    now a C8 standalone job:-

    * Added 'set -e' so that script exits immediately if a command exits
    with a non-zero status so it would result in deployment failure.
    Currently deployment is passing even if some commands in the bash
    script are failing. Addition of 'set -e' will correct this behavior.
    * Replaced yum with dnf as yum not present in Centos8.
    * No need to override resolv.conf entry as standalone job already
    configure necessary nameserver entries.
    * No need to install additional repos as as standalone job already
    configure necessary repos.
    * When vbmc_setup.yaml was introduced intention was to do non-kvm
    emulation[2] and that was the reason qemu-system-x86 and qemu-kvm-ev
    was pulled from epel, but for c8 these packages are no longer available
    in repos [3] including epel. We can use qemu-kvm package instead and
    we will no longer need epel repos.
    * create-node.sh command option to use qemu-system have to be removed
    as qemu-system-x86 package is missing and we are using qemu-kvm
    instead.
    * Temporary workaround[4] to upgrade ansible in the neutron container
    can be removed as ansible version in container is now v2.5.8+.
    Test patch for tripleo-ci-centos-8-scenario012-standalone is here[5].

    [0] https://review.opendev.org/#/c/485261/
    [1]
    https://blueprints.launchpad.net/tripleo/+spec/ironic-overcloud-ci
    [2]
    https://review.opendev.org/#/c/485261/29/ci/common/vbmc_setup.yaml
    [3] https://bugzilla.redhat.com/show_bug.cgi?id=1715806
    [4] https://review.opendev.org/#/c/579603/
    [5] https://review.opendev.org/#/c/724119/

    Closes-Bug: #1875681
    (cherry picked from commit 84e016edb39f352c3f36d07f94831fe3a0874587)

     Conflicts:
            ci/common/vbmc_setup.yaml

    Change-Id: I9e9d63a7ef2ef538f3c072e3f4a96ec25d7dc5f7

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.4.0

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.