OpenStack logs with Rsyslog are missing request id, tenant-id and user-id

[Swann Croiset]

Description:
=========

logfiles /var/log/*-all.log (for nova, cinder, glance ..) are incomplete: mising request-id, tenant-id, user-id and program name

This is a regression since mos 7.0

Steps to reproduce:
===============

boot an instance (nova boot)
check /var/log/nova-all.log

<182>Nov 26 17:24:15 node-2 nova-api: 10.109.7.2 "POST /v2/4c00c575be5f440eadd769384f8fa854/servers HTTP/1.1" status: 202 len: 749 time: 1.0047631

With MOS 7 the log looks like :
<182>Nov 25 13:07:32 node-53 nova-api 2015-11-25 13:07:32.839 8049 INFO nova.osapi_compute.wsgi.server [req-fe28ce2b-974a-4fb4-9546-a89d6e21cca8 5348b79c77344e2980fb7e92905b13e0 baab6c1c15c540b699682748f9453bc1 - - -] 192.168.0.2 "POST /v2/baab6c1c15c540b699682748f9453bc1/os-server-external-events HTTP/1.1" status: 200 len: 375 time: 5.0959420

This is general to all /var/log/*-all.log logfiles.

Expected Results:
==============

logs contain these information: request-id, tenant-id, user-id and program name

[Swann Croiset (swann-w)]

MOS 8 build #207

[Bogdan Dobrelya (bogdando)]

The *-all .log files being directly fed by the OpenStack services logging to the sylog. We should inspect service-default/fuel-provided configuration options for log formatters. If nothing changed on fuel side, we should adapt the configuration settings to restore its 7.0 behavior.

[Michael Polenchuk (mpolenchuk)]

http://docs.openstack.org/developer/nova/api/nova.openstack.common.log.html
http://docs.openstack.org/developer/oslo.log/api/formatters.html

I guess the context value stored in TLS ain't available therefore default logging format string is used.

[Bogdan Dobrelya (bogdando)]

Thank you for details, Michael. I marked this bug invalid for the library, and added MOS Oslo.

[Timur Nurlygayanov (tnurlygayanov)]

I described this issue in separate bug description for mos project https://bugs.launchpad.net/mos/+bug/1525798

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/255418
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=813f111880c13db1da2435b3b5c6b660006f357d
Submitter: Jenkins
Branch: master

commit 813f111880c13db1da2435b3b5c6b660006f357d
Author: Éric Lemoine <email address hidden>
Date: Tue Dec 8 01:49:01 2015 +0100

    Read OpenStack logs from OpenStack log files

    Notes about this change:

    * The nova, neutron, cinder, glance, heat, keystone, horizon and murano logs
      are now read from /var/log/<service> (where <service> is the OpenStack
      service name). So we no longer rely on rsyslog for these services.
    * We make an exception for Swift which only writes its logs to syslog. So we
      still assume Rsyslog and the Fuel-defined Rsyslog log pattern for Swift.
    * The "heka" user is now part of the "keystone" group. This is for Heka to be
      able to read the Keystone logs from /var/log/keystone.
    * We used to read Keystone logs from two different places, namely
      /var/log/keystone-all.log and /var/log/user.log. This is no longer the case.
      The Keystone logs are read from the files in /var/log/keystone.

    Things to do next (with separate changes):

    * Correctly parse Tracebacks (which are multi-line).
    * Correctly parse other multi-line logs (Command: logs from
      neutron.agent.linux.utils for example).
    * Add an Heka message field to store the Python module name.
    * Document the new defined types in the module's README.md file.
    * Move patterns in openstack_log.lua to patterns.lua and add unit-tests.
    * Read Keystone HTTP logs from /var/log/apache2/keystone_wsgi_*_access.log to
      still be able to generate HTTP metrics for Keystone.

    Change-Id: I667376030bff054e8ce9921d92f202e06f71269f
    Implements: blueprint lma-without-fuel
    Closes-Bug: #1520330
    Related-Bug: #1525247