openstack undercloud install fails on upgrade during restart of systemd-journald on a file assertion error

[Marios Andreou]

When upgrading the undercloud to stable/liberty and running

<setup repos>
sudo yum -y update
openstack undercloud install

^^ This fails and needs to be run a second time to succeed. The issue is originally discussed at https://bugzilla.redhat.com/show_bug.cgi?id=1315442

The undercloud install fails running:

2016-03-07 09:55:38,538 INFO: dib-run-parts Mon Mar 7 09:55:38 EST 2016 Running /tmp/tmp5t46Kc/pre-install.d/01-persistent-journal

which is at https://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-install/pre-install.d/01-persistent-journal

A trace from journalctl looks like:

Mar 31 09:31:07 instack.localdomain systemd-journald[23175]: Failed to create new runtime journal: No such file or directory
Mar 31 09:31:07 instack.localdomain systemd-journald[23175]: Assertion 'f' failed at src/journal/journal-file.c:132, function journal_file_close(). Aborting.
Mar 31 09:31:07 instack.localdomain systemd-journal[23252]: Journal started

seems the assert failure is causing the install to fail. Weirdly, the journal is started fine. As above, running the undercloud install a second time succeeds. I am contemplating adding set +e for the restart waiting a second and checking that the journal is running fine.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/300051

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300051
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=cc41b7a1f9009a1dd24193b3a7e98a13496cbcc0
Submitter: Jenkins
Branch: master

commit cc41b7a1f9009a1dd24193b3a7e98a13496cbcc0
Author: marios <email address hidden>
Date: Thu Mar 31 18:55:56 2016 +0300

    Temporarily set +e on systemd-journald restart for +bug/1564471

    Though the restart of the systemd-journald service succeeds fine, an
    assertion error during the restart of the systemd-journald service
    causes the instack-undercloud install to fail (this is during an
    undercloud upgrade in fact, details in the bug).

    This review sets +e to allow the script to continue and then also
    checks that the service was indeed started OK, exiting otherwise.

    Related-Bug: 1564471
    Change-Id: I38c14a6b143b51dec6f6f88f2e1cb591e07a08b3

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/liberty
Review: https://review.openstack.org/300360

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/300363

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300363
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=a508ad7218cbc2064fc9f1316ab5afca85b25eda
Submitter: Jenkins
Branch: stable/mitaka

commit a508ad7218cbc2064fc9f1316ab5afca85b25eda
Author: marios <email address hidden>
Date: Thu Mar 31 18:55:56 2016 +0300

    Temporarily set +e on systemd-journald restart for +bug/1564471

    Though the restart of the systemd-journald service succeeds fine, an
    assertion error during the restart of the systemd-journald service
    causes the instack-undercloud install to fail (this is during an
    undercloud upgrade in fact, details in the bug).

    This review sets +e to allow the script to continue and then also
    checks that the service was indeed started OK, exiting otherwise.

    Related-Bug: 1564471
    Change-Id: I38c14a6b143b51dec6f6f88f2e1cb591e07a08b3
    (cherry picked from commit cc41b7a1f9009a1dd24193b3a7e98a13496cbcc0)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300360
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=9f6e87809a892c4aae4f936ec46d2bbf828dfc8c
Submitter: Jenkins
Branch: stable/liberty

commit 9f6e87809a892c4aae4f936ec46d2bbf828dfc8c
Author: marios <email address hidden>
Date: Thu Mar 31 18:55:56 2016 +0300

    Temporarily set +e on systemd-journald restart for +bug/1564471

    Though the restart of the systemd-journald service succeeds fine, an
    assertion error during the restart of the systemd-journald service
    causes the instack-undercloud install to fail (this is during an
    undercloud upgrade in fact, details in the bug).

    This review sets +e to allow the script to continue and then also
    checks that the service was indeed started OK, exiting otherwise.

    Related-Bug: 1564471
    Change-Id: I38c14a6b143b51dec6f6f88f2e1cb591e07a08b3
    (cherry picked from commit cc41b7a1f9009a1dd24193b3a7e98a13496cbcc0)

[Emilien Macchi (emilienm)]

This bug is > 365 days without activity. We are unsetting assignee and milestone and setting status to Incomplete in order to allow its expiry in 60 days.

If the bug is still valid, then update the bug status.

[Launchpad Janitor (janitor)]

[Expired for tripleo because there has been no activity for 60 days.]