stable/mitaka to newton undercloud upgrade fails with various errors related to o-p-m packaging

Bug #1593736 reported by Marios Andreou
30
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Marios Andreou

Bug Description

[UPDATE 05 July 2016]:

Updating to reflect that this will track a number of errors that were seen here and that were all related to the openstack-puppet-modules packaging (and fixed by reinstalling it as commented below). The bugs and errors are:

this.bug, was previously for tracking "Error: Could not find class ::tripleo::selinux for instack on node instack"
https://bugs.launchpad.net/tripleo/+bug/1594890 "Error: Could not find class ::ironic::drivers::deploy for instack on node instack "
https://bugs.launchpad.net/tripleo/+bug/1594893 "Error: Invalid parameter pxe_bootfile_name on Class[Ironic::Drivers::Pxe] at /etc/puppet/manifests/puppet-stack-config.pp:437 on node instack"
https://bugs.launchpad.net/tripleo/+bug/1594895 "Error: Invalid parameter dport on Tripleo::Firewall::Rule[101 mongodb_config] on node instack"
https://bugs.launchpad.net/tripleo/+bug/1594896 "Error: Invalid parameter destination on Tripleo::Firewall::Rule[140 network cidr nat] on node instack"

Those bugs will all be closed as duplicated pointing here.

Original bug description, for the 'selinux' issue follows.
****************************************

tl;dr do we need to update puppet-tripleo and release openstack-puppet-modules because '::tripleo::selinux' ? The bug @ https://bugs.launchpad.net/tripleo/+bug/1593182 also needs o-p-m update so this may get fixed if that happens there.

During upgrade of a stable/mitaka environment to latest newton (delorean repos), and after applying a fix for bug https://bugs.launchpad.net/tripleo/+bug/1593182, the "openstack undercloud upgrade" fails with:

    2016-06-17 11:09:51 - Error: Could not find class ::tripleo::selinux for instack on node instack
    2016-06-17 11:09:51 - Error: Could not find class ::tripleo::selinux for instack on node instack
    + rc=1
    + set -e
    + echo 'puppet apply exited with exit code 1'
    puppet apply exited with exit code 1
    + '[' 1 '!=' 2 -a 1 '!=' 0 ']'
    + exit 1
    [2016-06-17 11:09:51,792] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run- parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]

    [2016-06-17 11:09:51,792] (os-refresh-config) [ERROR] Aborting...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1039, in install
        _run_orc(instack_env)
    File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 917, in _run_orc
      _run_live_command(args, instack_env, 'os-refresh-config')
      File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 450, in _run_live_command
        raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
    Command 'instack-install-undercloud' returned non-zero exit status 1

some combination of grep and git blame told me that '::tripleo::selinux' was introduced by

"Add tripleo::selinux " https://review.openstack.org/#/c/309408/ If214005df733d41c2fa4e197df247d8a14baaa14 puppet-tripleo

so I _think_ we need an update of puppet-tripleo in the openstack-puppet-modules and a new release of o-p-m? Note that https://bugs.launchpad.net/tripleo/+bug/1593182 also requires an update of openstack-puppet-modules (but for a puppet-nova fix).

This env has

    Version : 8.1.1
    Release : 0.20160609150428.ab63b38.el7.centos
    Size : 14 M
    Repo : installed
    From repo : delorean
    Summary : Puppet modules used to deploy OpenStack

For now I worked around this by manually patching tripleo in the openstack-puppet-modules

    https://review.openstack.org/changes/309408/revisions/8e533aaf447022c62865130f2ffc88690f06aef1/patch?zip

    sudo su
    yum install unzip
    # backup the templates incase:
    cp -r /usr/share/openstack-puppet/modules/tripleo /usr/share/openstack-puppet/modules/tripleo.BACKUP
    pushd /usr/share/openstack-puppet/modules/tripleo

    # Add tripleo::selinux https://review.openstack.org/#/c/309408/
    curl -o "patch1.diff.zip" https://review.openstack.org/changes/309408/revisions/8e533aaf447022c62865130f2ffc88690f06aef1/patch?zip
    unzip patch1.diff.zip
    patch -p1 < 8e533aaf.diff

    popd
    exit # the sudo su

and with that the openstack undercloud upgrade got further (but hitting another issue, will file another bug for that \o/ ... )

description: updated
description: updated
Revision history for this message
Marios Andreou (marios-b) wrote :
Download full text (3.5 KiB)

I am having trouble understanding how rdo openstack-puppet-modules is pulling in the puppet-tripleo module (more below) so a summary here I can point at for help; I have openstack-puppet-modules-8.1.1-0.20160609150428.ab63b38.el7.centos.noarch after the yum update with latest repos, but still don't have the selinux class in /usr/share/openstack-puppet/modules/tripleo/manifests/. That is, the change at https://review.openstack.org/#/c/309408/ isn't included in that package build it seems/afaics.

To reiterate, I deployed stable/mitaka. Then to upgrade, I disable stable/mitaka, enable latest repos using tripleo.sh, stop all services (thanks bnemec) like https://review.openstack.org/#/c/331804 and then yum update:

        sudo systemctl stop openstack-*
        sudo systemctl stop neutron-*

        # Depending on when the base image was created, the pinned delorean repo may
        # actually be older, which causes yum to ignore its metadata.
        sudo rm -f /etc/yum.repos.d/delorean*
        sudo yum clean all && sudo yum clean metadata && sudo yum clean dbcache && sudo yum makecache
        if ! [[ -d ~/tripleo-ci ]]; then git clone http://github.com/openstack-infra/tripleo-ci.git ~/tripleo-ci ;fi;
        ./tripleo-ci/scripts/tripleo.sh --repo-setup
        nice -n -10 sudo yum update -y

        openstack undercloud upgrade

        2016-06-22 10:48:01 - Error: Could not find class ::tripleo::selinux for instack on node instack
        2016-06-22 10:48:01 - Error: Could not find class ::tripleo::selinux for instack on node instack

I assume the openstack-puppet-modules package I get from the latest delorean repos (from ./tripleo-ci/scripts/tripleo.sh --repo-setup) is made via https://github.com/rdo-packages/openstack-puppet-modules-distgit

The version I have is openstack-puppet-modules-8.1.1-0.20160609150428.ab63b38.el7.centos.noarch - I can confirm that there is no selinux.pp in the installed code:

        [stack@instack ~]$ ll /usr/share/openstack-puppet/modules/tripleo/manifests/
        total 96
        drwxr-xr-x. 2 root root 4096 Jun 14 13:15 cluster
        -rw-r--r--. 1 root root 1559 Jun 9 15:05 fencing.pp
        drwxr-xr-x. 2 root root 4096 Jun 14 13:15 firewall
        -rw-r--r--. 1 root root 2418 Jun 9 15:05 firewall.pp
        -rw-r--r--. 1 root root 688 Jun 9 15:05 init.pp
        -rw-r--r--. 1 root root 51401 Jun 9 15:05 loadbalancer.pp
        drwxr-xr-x. 3 root root 4096 Jun 14 13:15 network
        -rw-r--r--. 1 root root 1632 Jun 9 15:05 noop.pp
        -rw-r--r--. 1 root root 2477 Jun 9 15:05 packages.pp
        drwxr-xr-x. 4 root root 4096 Jun 14 13:15 profile
        -rw-r--r--. 1 root root 1133 Jun 9 15:05 redis_notification.pp
        drwxr-xr-x. 2 root root 4096 Jun 14 13:15 ssl

My question is, what version of https://github.com/openstack/puppet-tripleo is pulled into that openstack-puppet-modules package and how is that governed? Do we need to add a version constraint to https://github.com/rdo-packages/openstack-puppet-modules-distgit/blob/9d3bcf2d35c4b816fb310ec74c2aa1ad008ba767/openstack-puppet-modules.spec#L74 ?

One other thing I noticed - does it matter that there is a difference b...

Read more...

Revision history for this message
Marios Andreou (marios-b) wrote :

so... it looks like you have to remove and re-install the openstack-puppet-modules package alltogether in order to pull in the dependencies properly, as listed in the https://github.com/rdo-packages/openstack-puppet-modules-distgit/blob/9d3bcf2d35c4b816fb310ec74c2aa1ad008ba767/openstack-puppet-modules.spec#L74

At least when I removed and re-installed openstack-puppet-modules, it pulled in all the puppet-* things, like puppet-tripleo @ v 5.0.0 - am trying to see how far I can now get in the undercloud upgrade... but at least it looks like it has pulled in the selinux class this time:
        [stack@instack ~]$ ll /usr/share/openstack-puppet/modules/tripleo/manifests/selinux.pp
-rw-r--r--. 1 root root 2466 May 10 14:15 /usr/share/openstack-puppet/modules/tripleo/manifests/selinux.pp

        [stack@instack ~]$ sudo yum remove openstack-puppet-modules
        [stack@instack ~]$ sudo yum install -y openstack-puppet-modules python-tripleoclient instack-undercloud

         puppet-tripleo noarch 5.0.0-0.20160610144302.3ea4259.el7.centos delorean 59 k

        [stack@instack ~]$ yum info openstack-puppet-modules
        Loaded plugins: fastestmirror, priorities
        Loading mirror speeds from cached hostfile
         * base: mirrors.coreix.net
         * centos7: mirrors.coreix.net
         * epel: nl.mirror.babylon.network
         * extras: mirrors.coreix.net
         * updates: mirrors.coreix.net
        618 packages excluded due to repository priority protections
        Installed Packages
        Name : openstack-puppet-modules
        Arch : noarch
        Epoch : 1
        Version : 8.0.0
        Release : 0.20160520142355.6a3e8bf.el7.centos
        Size : 0.0
        Repo : installed
        From repo : delorean
        Summary : Puppet modules to deploy OpenStack
        URL : https://github.com/redhat-openstack/openstack-puppet-modules
        License : Apache-2.0
        Description : Metapackage for OpenStack Puppet Modules

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/332889

Changed in tripleo:
status: Triaged → In Progress
Changed in tripleo:
milestone: none → newton-2
summary: - stable/mitaka to newton undercloud upgrade fails with 'Could not find
- class ::tripleo::selinux'
+ table/mitaka to newton undercloud upgrade fails with various errors
+ related to o-p-m packaging
summary: - table/mitaka to newton undercloud upgrade fails with various errors
+ stable/mitaka to newton undercloud upgrade fails with various errors
related to o-p-m packaging
description: updated
Steven Hardy (shardy)
Changed in tripleo:
milestone: newton-2 → newton-3
Revision history for this message
Emilien Macchi (emilienm) wrote :

I confirm undercloud upgrade now works.

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-tripleoclient (master)

Change abandoned by Marios Andreou (<email address hidden>) on branch: master
Review: https://review.openstack.org/332889
Reason: this was fixed in packaging

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.