python package amqp, should not run in debug mode when nova is in debug mode

[Joe Gordon]

Currently the python package amqp runs in debug mode when nova is in debug mode. When specifying debug level logs in nova generally we are trying to debug nova not its dependencies. So by default amqp and other python dependencies shouldn't be running in debug mode when nova is.

http://logs.openstack.org/47/36947/5/check/gate-tempest-devstack-vm-full/37958/logs/screen-n-sch.txt.gz

2013-07-31 20:52:43.207 28626 DEBUG amqp [-] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'consumer_cancel_notify': True, u'publisher_confirms': True, u'basic.nack': True}, u'platform': u'Erlang/OTP', u'version': u'2.7.1'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US'] _start /usr/local/lib/python2.7/dist-packages/amqp/connection.py:706
2013-07-31 20:52:43.208 28626 DEBUG amqp [-] Open OK! _open_ok /usr/local/lib/python2.7/dist-packages/amqp/connection.py:592
2013-07-31 20:52:43.208 28626 DEBUG amqp [-] using channel_id: 1 __init__ /usr/local/lib/python2.7/dist-packages/amqp/channel.py:70
2013-07-31 20:52:43.208 28626 DEBUG amqp [-] Channel open _open_ok /usr/local/lib/python2.7/dist-packages/amqp/channel.py:420

[Robert Collins (lifeless)]

mmm, arguably when running in debug we should snarf as much as possible because the fault might not be in nova.

Separately, I would love it if we could make switching in/out of debug mode not require service restarts.

[Joe Gordon (jogo)]

While more data is better when debugging, things like the amqp library are rarley the issue when debugging, and leaving debug on for it makes the logs harder to parse as a human. I would like to see two debug log levels. Debug nova only and debug all. and setting log level to debug would do debug nova only.

[Matt Riedemann (mriedem)]

This isn't the answer but it might be helpful: http://logstash.openstack.org/

[Joe Gordon (jogo)]

Matt, yes it is very helpful and not the answer

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/61956

[Brad Pokorny (bpokorny)]

This one looks to me like something that would affect all OpenStack projects. Also, the proposed fix for this one is a change to log.py, which is synced from oslo to nova. So the fix should go into oslo so all projects can take advantage of it.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/64022

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/64023

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/64033

[Anton (agorenkov)]

As it was discussed here https://review.openstack.org/#/c/64022/ :

Bug on oslo was fixed:

>> We've been handling this on a lib by lib basis lately. I think the linked bug was actually fixed
>> by https://github.com/openstack/oslo-incubator/commit/fb8075a19d475ad7f2868242ab59507bc1fb229e

And it was synced to nova by https://github.com/openstack/nova/commit/bcc1f16c8491a7333b6fd15bafeef2a95f406d0f

So I think the issue is fixed now.