devstack logs do not contain pid information for log messages with context

[Ihar Hrachyshka]

Compare:

2015-03-18 15:00:15.990 INFO neutron.wsgi [req-412094f3-6b4e-41e8-9f2b-833ff6b3ee7a SecurityGroupsTestJSON-724004567 SecurityGroupsTestJSON-664869352] 127.0.0.1 - - [18/Mar/2015 15:00:15] "DELETE /v2.0/security-groups/9cc93b9a-2d06-46e6-9160-1521683f13f9.json HTTP/1.1" 204 149 0.060949
2015-03-18 15:00:16.001 15709 INFO neutron.wsgi [-] (15709) accepted ('127.0.0.1', 60381)

This is because in devstack, we override the default log format string with the one that misses the info. Note that to make it work, it is not enough to fall back to default string, since it uses user_identity context field that is missing in neutron context object. That is because neutron.context.Context does not rely on oslo_context.Context when transforming it to_dict().

The proper fix would be:

- make neutron context reuse oslo_context.Context.to_dict()
- make devstack not overwrite the default log format string

Also note that log colorizer from devstack also rewrites the default format string value. In that case, we just need to update the string to include pid information.

Also note that the issue may be more far reaching, since devstack rewrites the string for other services too (nova, ironic, among others).

[Ihar Hrachyshka (ihar-hrachyshka)]

The bug is especially interesting now that we run multiple workers for neutron-server by default.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/172508

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/172509

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/172510

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/160112
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=52cd81934b6a64bda80a140446d8895413789221
Submitter: Jenkins
Branch: master

commit 52cd81934b6a64bda80a140446d8895413789221
Author: Ihar Hrachyshka <email address hidden>
Date: Sat Feb 28 13:48:18 2015 +0100

    context: reuse base oslo.context class for to_dict()

    It is need to conform to expectations of consumers that rely on
    oslo.context behaviour (f.e. oslo.log that relies [1] on user_identity
    field being set for context objects).

    [1]: https://github.com/openstack/oslo.log/blob/master/oslo_log/_options.py#L99

    Closes-Bug: #1433687
    Change-Id: I95e803b96e6e3e5b8c12298dc6327b974330c639

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: neutron-pecan
Review: https://review.openstack.org/185072

[Ihar Hrachyshka (ihar-hrachyshka)]

This bug depends on new oslo.log feature: blueprint user-identity-format-flexibility

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/172508
Reason: this is really not the direction we want to go

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/172510
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=198887e8903696ea9fcbec0f8a91c2f6ca5a34c7
Submitter: Jenkins
Branch: master

commit 198887e8903696ea9fcbec0f8a91c2f6ca5a34c7
Author: Ihar Hrachyshka <email address hidden>
Date: Fri Apr 10 18:45:35 2015 +0200

    logging: don't set logging format strings for keystone

    Don't override those format strings since the overridden
    values are identical to those used by oslo.log by default [1].

    logging_exception_prefix is still set since it changes the logging
    format to use TRACE label for exceptions instead of default ERROR.

    [1]: https://git.openstack.org/cgit/openstack/oslo.log/tree/oslo_log/_options.py?id=c47a91dbbb586c27d8521b1016bf7901c47b1c90#n110

    Closes-Bug: #1433687
    Change-Id: Ibd11cd6b0defb6dc709dbd3e718a49fd71cce6b6

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ihar Hrachyshka (<email address hidden>) on branch: master
Review: https://review.openstack.org/172509
Reason: Indeed, more work is needed. Actually, seems like https://review.openstack.org/#/c/301802/1 targets that already.