Horizon repeating log messages when DEBUG=True

[Nicolas Simonds]

When setting DEBUG=True (i.e., in devstack and other development environments) we noticed that all the python-*client API calls would get logged multiple times. On high-traffic dev instances, it would be dozens of repeats for each log message. This made development work on these instances an experience somewhere on the scale between "hey, this is annoying" and "it's impossible to tell what's going on through all this noise"

Testing revealed that the problem seems to stem from different mod_wsgi processes/threads vivifying their own API client objects. The clients seem to blindly shove logging handlers into the list without looking to see if it's necessary to do so first, and this causes Horizon to have dozens of identical handlers for each log message, and it then does what you'd expect.

This isn't a problem with Horizon per se, but Horizon is affected by it. So far, we've observed the problem with novaclient, cinderclient and keystoneclient, but all the client libraries likely behave in the same fashion.

We also discovered in testing that there is no logging handler defined for cinderclient in the local_settings.py, so it uses the default hander.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30004

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30005

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30006

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30007

[Morgan Fainberg (mdrnstm)]

This likely affects all python-*client libraries and should be tested and expanded as needed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30007
Committed: http://github.com/openstack/python-keystoneclient/commit/de6f5120510c73ff4de376f5a361d1293f1d68a6
Submitter: Jenkins
Branch: master

commit de6f5120510c73ff4de376f5a361d1293f1d68a6
Author: Nicolas Simonds <email address hidden>
Date: Tue May 21 16:25:11 2013 -0700

    Only add logging handlers if there currently aren't any

    This corrects an odd problem where Horizon would stand up multiple
    client objects, which would cause duplicate/triplicate/dozens of
    repeated log lines in its log files, due to multiple identical
    handlers being added to the logging object

    Fixes Bug 1182678

    Change-Id: I020b3999c9008b996286ccb74a7801c96c43e71c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30004
Committed: http://github.com/openstack/horizon/commit/cda21fd594b3444337ccb96e427bb3c4a319cfd1
Submitter: Jenkins
Branch: master

commit cda21fd594b3444337ccb96e427bb3c4a319cfd1
Author: Nicolas Simonds <email address hidden>
Date: Tue May 21 16:00:07 2013 -0700

    Add a logging handler for the cinderclient to the local_settings.py.example

    This handles the Horizon piece of the API client logging wonderfulness

    Change-Id: Id2293898557b2fbbde092d8b67044482caf25c09
    Fixes: Bug 1182678

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30006
Committed: http://github.com/openstack/python-novaclient/commit/e8e7a0e04fa6ced2e5a02a533d29a20d3014b4a8
Submitter: Jenkins
Branch: master

commit e8e7a0e04fa6ced2e5a02a533d29a20d3014b4a8
Author: Nicolas Simonds <email address hidden>
Date: Tue May 21 16:23:50 2013 -0700

    Only add logging handlers if there currently aren't any

    This corrects an odd problem where Horizon would stand up multiple
    client objects, which would cause duplicate/triplicate/dozens of
    repeated log lines in its log files, due to multiple identical
    handlers being added to the logging object

    Fixes Bug 1182678

    Change-Id: I1f3bae0a39f28412c99e5d04f4364611f2a5facb

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30005
Committed: http://github.com/openstack/python-cinderclient/commit/2a446c5b6bf0fa76c7e569e159362ec0d4cf2e6c
Submitter: Jenkins
Branch: master

commit 2a446c5b6bf0fa76c7e569e159362ec0d4cf2e6c
Author: Nicolas Simonds <email address hidden>
Date: Tue May 21 16:19:38 2013 -0700

    Only add logging handlers if there currently aren't any

    This corrects an odd problem where Horizon would stand up multiple
    client objects, which would cause duplicate/triplicate/dozens of
    repeated log lines in its log files, due to multiple identical
    handlers being added to the logging object

    Fixes Bug 1182678

    Change-Id: I198f3ecbb687bff69a06a166574b998cce54f2ac

[Morgan Fainberg (mdrnstm)]

This also affects python-quantumclient (same problem) with a slightly different manifestation (env variables, but again with no check to see if there is already a handler).

[Morgan Fainberg (mdrnstm)]

Apologies, I was mistaken, this will not affect python-quantumclient due to the location of logging instantiation.