context warnings spam scheduler and share logs

Bug #1582346 reported by Tom Barron
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Low
Tom Barron

Bug Description

Manila share and scheduler logs are littered with warnings about dropped kwargs when creating request context. Here is an example:

2016-04-18 17:08:16.103 WARNING manila.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'6133ebc001384629a4a9ecfc630bb26b 21b76a623d104713845b686526c10720 - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}.

These are currently "just noise". Their meaning is not evident without code study and cause confusion and alarm to cloud operators resulting in unnecessary support calls.

In manila share log these messages occur on every CRUD operation.

In manila scheduler the situation is even worse as they occur on every periodic capabilities/capacity update.

Tom Barron (tpb)
Changed in manila:
assignee: nobody → Tom Barron (tpb)
description: updated
Revision history for this message
Tom Barron (tpb) wrote :

cinder "fixed" the issue by removing the warning in question altogether [1]; neutron, by reducing the log level from warning to debug. I prefer the approach taken by nova [3] - namely leaving the log at warning level but popping off all expected kwargs (and converting these where possible to equivalents that are then passed into the oslo RequestContext constructor) so that in the rare event that there are really unexpected kwargs they will still be logged at an appropriate level.

[1] https://review.openstack.org/#/c/105106/

[2] https://review.openstack.org/#/c/134762/

[3] https://review.openstack.org/#/c/160535/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/317069

Changed in manila:
status: New → In Progress
Changed in manila:
importance: Undecided → Low
milestone: none → newton-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.openstack.org/317069
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=4c3c7e5fa4ac49cd7c4fae8eb54163ea64e0fa9e
Submitter: Jenkins
Branch: master

commit 4c3c7e5fa4ac49cd7c4fae8eb54163ea64e0fa9e
Author: Tom Barron <email address hidden>
Date: Sat May 14 08:34:08 2016 -0400

    Fix context warning spam of scheduler and share logs

    Manila share and scheduler logs are littered with warnings about dropped
    kwargs when creating request context. Here is an example:

    2016-04-18 17:08:16.103 WARNING manila.context [-] Arguments dropped
    when creating context: {u'read_only': False, u'domain': None,
    u'show_deleted': False, u'user_identity':
    u'6133ebc001384629a4a9ecfc630bb26b 21b76a623d104713845b686526c10720 - -
    -', u'project_domain': None, u'resource_uuid': None, u'user_domain':
    None}.

    These are currently "just noise". Their meaning is not evident without
    code study, and they cause confusion and alarm to cloud operators,
    resulting in unnecessary support calls and ultimately in a reduction
    in value and credibility of our logging system.

    In manila share log these messages occur on every CRUD operation.
    In manila scheduler the situation is even worse as they occur on every
    periodic capabilities/capacity update.

    Fix this spam by only sending this warning when we receive truly
    unexpected kwargs by following the nova fix [1] for essentially
    the same issue in that project.

    Closes-Bug: #1582346

    [1] Ia47d4909d2656d6fc4c1179659b8098bba3235d3

    Change-Id: Ieba91b42ef680b353dbb326667580bf482ff8d48

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/manila 3.0.0.0b1

This issue was fixed in the openstack/manila 3.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.