use_syslog=True does not log to syslog via /dev/log anymore

[Bogdan Dobrelya]

python-oslo.log SRU:
[Impact]

 * Nova services not able to write log to syslog

[Test Case]

 * 1. Set use_syslog to True in nova.conf/cinder.conf
   2. stop rsyslog service
   3. restart nova/cinder services
   4. restart rsyslog service
   5. Log is not written to syslog after rsyslog is brought up.

[Regression Potential]

 * none

Reproduced on:
https://github.com/openstack-dev/devstack 514c82030cf04da742d16582a23cc64962fdbda1
/opt/stack/keystone/keystone.egg-info/PKG-INFO:Version: 2015.1.dev95.g20173b1
/opt/stack/heat/heat.egg-info/PKG-INFO:Version: 2015.1.dev213.g8354c98
/opt/stack/glance/glance.egg-info/PKG-INFO:Version: 2015.1.dev88.g6bedcea
/opt/stack/cinder/cinder.egg-info/PKG-INFO:Version: 2015.1.dev110.gc105259

How to reproduce:
Set
 use_syslog=True
 syslog_log_facility=LOG_SYSLOG
for Openstack config files and restart processes inside their screens

Expected:
Openstack logs logged to syslog as well

Actual:
Nothing goes to syslog

[Ben Nemec (bnemec)]

use_use=True <-- typo?

[Bogdan Dobrelya (bogdando)]

yes, thanks.

[Bogdan Dobrelya (bogdando)]

Which info is missing, please elaborate?

[Bogdan Dobrelya (bogdando)]

Thanks Dmitry Mescheryakov for help, here is the root cause: https://github.com/openstack/oslo-incubator/commit/1188d88a73e75da5c143d0dfad34e10dad0daee0
/dev/log was missed by that change

[Bogdan Dobrelya (bogdando)]

Please return missed /dev/log https://review.openstack.org/#/c/110753/1/openstack/common/log.py,cm

[Bogdan Dobrelya (bogdando)]

Looks like /dev/log/ was removed by intentionally and this bug is invalid.
But I believe the commit 1188d88a73e75da5c143d0dfad34e10dad0daee0 with removed /dev/log should had had a DocImpact tag and the change should be documented anyway.
E.g. something like "/dev/log/ will not be used by Openstack services anymore due to portability reasons, please make sure syslog listens UDP:514 port in order to recieve logs from Openstack."

[Anne Gentle (annegentle)]

I think the best doc to place this in is the Cloud Administrator Guide.

[Daniel Gauthier (gaud)]

I seem to have the same problem with cinder and nova . when those service have use_syslog set to True, nothing is sent to syslog even if rsyslog UDP:514 is accepting connection.

[J-PMethot (jpmethot)]

I understand that this "bug" is actually by design, for portability. However, it's not clear to me how openstack interacts with syslog instead of through /dev/log. Actually, from our tests, it seems to completely ignore the use_syslog=true now, just as the original report said.

I understand that this will be updated in the doc, but is there actually anything replacing the old way openstack used to interact with syslog? Or was there a conscious decision to drop syslog altogether?

[Bogdan Dobrelya (bogdando)]

I vote for return /dev/log and use another explicit ways to ensure portability as well. If ... else should do the job as well.
That 'drop' of /dev/log was not backwards compatible, for sure. Such changes first should go through a deprecation at least

[Sergey Vasilenko (xenolog)]

I vote for return /dev/log.

For portability will be better use if .. else statement.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/133912

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/136842

[Tom Fifield (fifieldt)]

Docs Triage: Looks like was a bug that is being fixed in oslo, so removing manuals

[Doug Hellmann (doug-hellmann)]

This was resolved in the incubated version of the logging code by https://review.openstack.org/#/c/133638/

[Bogdan Dobrelya (bogdando)]

@Doug, could you please elaborate, should the patches be contributed both to the main and incubated versions of Oslo.log?

[OpenStack Infra (hudson-openstack)]

Change abandoned by Roman Podoliaka (<email address hidden>) on branch: master
Review: https://review.openstack.org/133912

[Doug Hellmann (doug-hellmann)]

@Bogdan, at this point the logging code has fully graduated to oslo.log. Following our process for dealing with graduated libraries, changes should happen first in openstack/oslo.log and then be backported to relevant stable branches in openstack/oslo-incubator if needed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/159606

[John Stanford (jxstanford)]

I just resubmitted Roman's change since the related portability ticket is still a work in progress, and it seems reasonable to treat portability as a separate issue from a commit that broke a working, if not portable, feature.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/159606
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=2bf8164a908b51c13d4709e63e7cb2905b5548c7
Submitter: Jenkins
Branch: master

commit 2bf8164a908b51c13d4709e63e7cb2905b5548c7
Author: John Stanford <email address hidden>
Date: Thu Feb 26 12:47:16 2015 -0800

    Make use_syslog=True log to syslog via /dev/log

    After upgrade from Icehouse to Juno OpenStack services don't log to
    syslog via /dev/log anymore. This behaviour was introduced in commit
    1188d88a73e75da5c143d0dfad34e10dad0daee0. While /dev/log is arguably
    not a very portable default, breaking the backwards compatibility is
    even worse.
    This commit restores the original behaviour. /dev/log portability
    issues will be addressed separately.
    Closes-Bug: #1385295
    Related-Bug: #1391428

    Change-Id: I5c450595fde7451a5bf9372430a53e75dd5773a3

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/136842
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=33f5c6f94d94d5464fea38229d880f177b05aff7
Submitter: Jenkins
Branch: master

commit 33f5c6f94d94d5464fea38229d880f177b05aff7
Author: Julien Danjou <email address hidden>
Date: Mon Nov 24 18:35:36 2014 +0100

    Replace RFCSysLogHandler by a syslog() based one

    Rather than using the Python provided logging module that reimplents the
    whole syslog protocol and pointing it to /dev/log (which is not
    portable), we use the system calls from the syslog module, which are
    portable and default to the behaviour we actually want.

    Closes-Bug: 1385295
    Closes-Bug: 1391428

    Change-Id: I39a36316bd8778831b1bd5e7c5e2e12ede062a37

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/193633

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/193633
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=9a6c22663b57b7f2d9071767dadf28e59075a75e
Submitter: Jenkins
Branch: stable/kilo

commit 9a6c22663b57b7f2d9071767dadf28e59075a75e
Author: Julien Danjou <email address hidden>
Date: Mon Nov 24 18:35:36 2014 +0100

    Replace RFCSysLogHandler by a syslog() based one

    Rather than using the Python provided logging module that reimplents the
    whole syslog protocol and pointing it to /dev/log (which is not
    portable), we use the system calls from the syslog module, which are
    portable and default to the behaviour we actually want.

    Closes-Bug: 1385295
    Closes-Bug: 1391428

    Partial-Bug: 1459046

    (cherry picked from commit 33f5c6f94d94d5464fea38229d880f177b05aff7)

    Conflicts:
     oslo_log/log.py
     requirements.txt

    Change-Id: I2e47c896841cbd43d1f5504f2b8b897fc432cd0b

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "vivid-kilo patch for python-oslo.log" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Pádraig Brady (p-draigbrady)]

Fix proposed for nova Juno stable
https://review.openstack.org/202573

[Chris J Arges (arges)]

For Ubuntu, Can you rebase this patch on the latest version of oslo.log? When done please re-subscribe sponsors.

[George Shuklin (george-shuklin)]

I think this patch is not merged to Juno https://review.openstack.org/#/c/202573/

[George Shuklin (george-shuklin)]

Sorry, My bad. I didn't saw merge message.

[Chuck Short (zulcss)]

This should be fixed now.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.log 1.0.0 release.