Comment 12 for bug 1506112

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/318544
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=6e49910fcadf52cdeff235b0795f0dde2f63f60a
Submitter: Jenkins
Branch: master

commit 6e49910fcadf52cdeff235b0795f0dde2f63f60a
Author: Dmitry Guryanov <email address hidden>
Date: Thu May 19 12:14:56 2016 +0300

    fix race with syslog while renaming remote log dir

    When a node gets new name nailgun renmaes log directoty
    which had name in for mof <IP> to <node name> and then
    creates symlink <IP> -> <node name>. The problem is that
    node is still active and this moment and sends some logs
    to syslog on master node.

    So when we renamed <IP> to <node name> and haven't created
    symlink name syslog may find missing log dir and it creates
    it again in this case. Then nailgun tries to create symlink
    and fails.

    To avoid this situation we can suspend rsyslog temporarity.
    It's better to do it with SIGSTOP/SIGCONT, because process
    exist in this case and remote logs will be held in kernel
    tcp or odp buffers until we wake up rsyslog and it process
    new messages. Fortunately centos-7 uses journald together
    with rsyslog, so local logs (send by syslog function) are
    handled by journald and rsyslog just get it from journald
    files. So it's safe to stop rsyslog temporarily, nothing
    will be droppped.

    Change-Id: Icb1ed6669be2f7b448c465de33de37e0b945fb79
    Closes-Bug: #1506112