[instack-undercloud] package upgrade should happen before service configuration

[John Trowbridge]

We are currently seeing an issue where if you start with a CentOS 7.2 based undercloud, and run `openstack undercloud install`, at the end of the install ironic conductor is in a broken (though not failed) state. This is because the CentOS base repos are now 7.3 so many packages get upgraded. Specifically when trying to register nodes the following trace is seen in the conductor logs:

2016-12-15 18:57:58.927 15490 ERROR paramiko.transport [-] Unknown exception: 'module' object has no attribute 'Cryptography_EVP_MD_CTX_new'
2016-12-15 18:57:58.930 15490 ERROR paramiko.transport [-] Traceback (most recent call last):
2016-12-15 18:57:58.931 15490 ERROR paramiko.transport [-] File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 1757, in run
2016-12-15 18:57:58.931 15490 ERROR paramiko.transport [-] self.kex_engine.parse_next(ptype, m)
2016-12-15 18:57:58.932 15490 ERROR paramiko.transport [-] File "/usr/lib/python2.7/site-packages/paramiko/kex_group1.py", line 75, in parse_next
2016-12-15 18:57:58.932 15490 ERROR paramiko.transport [-] return self._parse_kexdh_reply(m)
2016-12-15 18:57:58.932 15490 ERROR paramiko.transport [-] File "/usr/lib/python2.7/site-packages/paramiko/kex_group1.py", line 111, in _parse_kexdh_reply
2016-12-15 18:57:58.933 15490 ERROR paramiko.transport [-] self.transport._verify_key(host_key, sig)
2016-12-15 18:57:58.933 15490 ERROR paramiko.transport [-] File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 1605, in _verify_key
2016-12-15 18:57:58.934 15490 ERROR paramiko.transport [-] if not key.verify_ssh_sig(self.H, Message(sig)):
2016-12-15 18:57:58.934 15490 ERROR paramiko.transport [-] File "/usr/lib/python2.7/site-packages/paramiko/rsakey.py", line 119, in verify_ssh_sig
2016-12-15 18:57:58.934 15490 ERROR paramiko.transport [-] algorithm=hashes.SHA1(),
2016-12-15 18:57:58.935 15490 ERROR paramiko.transport [-] File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/rsa.py", line 589, in verifier
2016-12-15 18:57:58.935 15490 ERROR paramiko.transport [-] self._backend, self, signature, padding, algorithm
2016-12-15 18:57:58.935 15490 ERROR paramiko.transport [-] File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/rsa.py", line 400, in __init__
2016-12-15 18:57:58.936 15490 ERROR paramiko.transport [-] self._hash_ctx = hashes.Hash(self._algorithm, self._backend)
2016-12-15 18:57:58.936 15490 ERROR paramiko.transport [-] File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/primitives/hashes.py", line 82, in __init__
2016-12-15 18:57:58.936 15490 ERROR paramiko.transport [-] self._ctx = self._backend.create_hash_ctx(self.algorithm)
2016-12-15 18:57:58.937 15490 ERROR paramiko.transport [-] File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 195, in create_hash_ctx
2016-12-15 18:57:58.937 15490 ERROR paramiko.transport [-] return _HashContext(self, algorithm)
2016-12-15 18:57:58.937 15490 ERROR paramiko.transport [-] File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/hashes.py", line 21, in __init__
2016-12-15 18:57:58.938 15490 ERROR paramiko.transport [-] ctx = self._backend._lib.Cryptography_EVP_MD_CTX_new()
2016-12-15 18:57:58.938 15490 ERROR paramiko.transport [-] AttributeError: 'module' object has no attribute 'Cryptography_EVP_MD_CTX_new'
2016-12-15 18:57:58.939 15490 ERROR paramiko.transport [-]

Restarting ironic-conductor service allows the nodes to be registered without fail.

This seems like a bug in our puppet manifest that the service configuration (and therefore starting) should always happen after the package upgrade.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/411957
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=853b4bfbadda67cf659a8a744b82d128a4bc3247
Submitter: Jenkins
Branch: master

commit 853b4bfbadda67cf659a8a744b82d128a4bc3247
Author: Emilien Macchi <email address hidden>
Date: Fri Dec 16 13:55:58 2016 -0500

    Run `yum update -y` before Puppet run

    This patch run `yum update -y` right before running os-refresh-config
    (Puppet).

    1) Running `yum` from an Exec in Puppet is a bad idea because Puppet has
    no idea which packages resources are touched, so we're loosing the
    orchestration control.

    2) We need to ensure that `yum update -y` runs before any attempt to
    deploy OpenStack services, so we have latest packages from RDO and
    CentOS.

    Change-Id: Ifbbbdbf1f0c569791abc712913c7e0f6bbe533d9
    Closes-Bug: #1650374

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/413035

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/413035
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=aab2dadd199a57042274a7f8771dc1774be85399
Submitter: Jenkins
Branch: stable/newton

commit aab2dadd199a57042274a7f8771dc1774be85399
Author: Emilien Macchi <email address hidden>
Date: Sat Dec 17 09:15:42 2016 -0500

    Align the stars to unblock instack-undercloud gate on stable/newton

    We need to squash to patches if we want to unblock stable/newton CI.

    First patch:

    Fix bashate errors and warnings (lint)

    Fix 1 warning and 1 error in bash syntax:

    1) E043: Arithmetic compound has inconsistent return semantics.
       Fix the bash syntax to be compliant.

    2) ERROR: Split export and assignments in 2 files. Ignore the files.
       Fix the bash syntax to be compliant.
       Note: this test has been introducted by
       https://review.openstack.org/#/c/400542/

    (cherry picked from commit 9031dcf62016c5ca198a39109425b9f063e131fc)

    Second Patch:

    Run `yum update -y` before Puppet run

    This patch run `yum update -y` right before running os-refresh-config
    (Puppet).

    1) Running `yum` from an Exec in Puppet is a bad idea because Puppet has
    no idea which packages resources are touched, so we're loosing the
    orchestration control.

    2) We need to ensure that `yum update -y` runs before any attempt to
    deploy OpenStack services, so we have latest packages from RDO and
    CentOS.

    Closes-Bug: #1650374
    (cherry picked from commit 853b4bfbadda67cf659a8a744b82d128a4bc3247)

    Change-Id: Ifbbbdbf1f0c569791abc712913c7e0f6bbe533d9

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/413792

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/instack-undercloud 5.2.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/413792
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=8ba565cb21fb76bd6fdc8f3d3a032e5011bd9a74
Submitter: Jenkins
Branch: stable/mitaka

commit 8ba565cb21fb76bd6fdc8f3d3a032e5011bd9a74
Author: Emilien Macchi <email address hidden>
Date: Sat Dec 17 09:15:42 2016 -0500

    Align the stars to unblock instack-undercloud gate on stable/mitaka

    We need to squash to patches if we want to unblock stable/mitaka CI.

    First patch:

    Fix bashate errors and warnings (lint)

    Fix 1 warning and 1 error in bash syntax:

    1) E043: Arithmetic compound has inconsistent return semantics.
       Fix the bash syntax to be compliant.

    2) ERROR: Split export and assignments in 2 files. Ignore the files.
       Fix the bash syntax to be compliant.
       Note: this test has been introducted by
       https://review.openstack.org/#/c/400542/

    (cherry picked from commit 9031dcf62016c5ca198a39109425b9f063e131fc)

    Second Patch:

    Run `yum update -y` before Puppet run

    This patch run `yum update -y` right before running os-refresh-config
    (Puppet).

    1) Running `yum` from an Exec in Puppet is a bad idea because Puppet has
    no idea which packages resources are touched, so we're loosing the
    orchestration control.

    2) We need to ensure that `yum update -y` runs before any attempt to
    deploy OpenStack services, so we have latest packages from RDO and
    CentOS.

    Closes-Bug: #1650374
    (cherry picked from commit 853b4bfbadda67cf659a8a744b82d128a4bc3247)

    Change-Id: Ifbbbdbf1f0c569791abc712913c7e0f6bbe533d9
    (cherry picked from commit aab2dadd199a57042274a7f8771dc1774be85399)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/instack-undercloud 6.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/instack-undercloud 4.2.1 release.