default oslo_messaging log level bumped to INFO in pike

[Edward Hope-Morley]

I'm debugging a nova rpc issue on Pike and have noticed that the default log level for the oslo_messaging modules have increased to INFO from DEBUG (in default_log_levels) meaning that I no longer see logs like:

2018-10-11 11:54:10.426 1382 DEBUG oslo_messaging._drivers.amqpdriver [req-2cbd4559-3d1d-4bfb-a17d-cce6d196e156 d7f5bb002a26491fb1240f9cd4138176 145566761e02406e86ad7c8a102a8a04 - - -] CAST unique_id: 12a7c4cae5d241bdb4863e0f00ce897d exchange 'nova' topic 'compute.ComputeA' _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:442

These log levels are set using the default_log_levels config opt in [DEFAULT] so setting the following allows these logs to return (annoyingly you need to set everything to be able to change any one):

default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=DEBUG,oslo_messaging=DEBUG,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO

We need to be able to manipulate this via the charms when DEBUG is enabled since it is essential for debugging.

[Edward Hope-Morley (hopem)]

btw the reason for this is that oslo.log stable/ocata does not include oslo_messaging in its DEFAULT_LOG_LEVELS [1] whereas pike does.

[1] https://github.com/openstack/oslo.log/blob/stable/ocata/oslo_log/_options.py#L21

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/643076

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/643077

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/643077
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=72a1c0be93ffa0f63f075971f8bd869bee72e9c8
Submitter: Zuul
Branch: master

commit 72a1c0be93ffa0f63f075971f8bd869bee72e9c8
Author: Edward Hope-Morley <email address hidden>
Date: Wed Mar 13 15:21:55 2019 +0000

    Support oslo_messaging logger debug

    Pike onwards defaults to setting oslo_messaging logger
    to INFO and we have no way to set it to DEBUG currently.
    This patch will set it to DEBUG when debug is enabled.

    Also sets glance.debug=True if debug enabled

    Change-Id: Id76302b5368606401e4102e6beb4fae61df668ad
    Closes-Bug: #1817290

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/643076
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cloud-controller/commit/?id=854603ed0be07ec5cb3b3aa0df50603951c948ac
Submitter: Zuul
Branch: master

commit 854603ed0be07ec5cb3b3aa0df50603951c948ac
Author: Edward Hope-Morley <email address hidden>
Date: Wed Mar 13 15:23:32 2019 +0000

    Support oslo_messaging logger debug

    Pike onwards defaults to setting oslo_messaging logger
    to INFO and we have no way to set it to DEBUG currently.
    This patch will set it to DEBUG when debug is enabled.

    Also sets glance.debug=True if debug enabled

    Change-Id: I9216325f8e96cb9ee06cddbe75cdd15e98fb0827
    Closes-Bug: #1817290

[Edward Hope-Morley (hopem)]

We just hit this issue while debugging a Mitaka env so I'd like to get this applied for >= M. Despite M, N, O enabling it in Nova itself [1] the patch is the same.

[1] https://github.com/openstack/nova/blob/mitaka-eol/nova/config.py#L39

[Edward Hope-Morley (hopem)]

Mitaka support added in https://review.opendev.org/#/c/736165/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/736176

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/736176
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cloud-controller/commit/?id=71c7cbecec022dc672df53c4868a7cef9103ee34
Submitter: Zuul
Branch: master

commit 71c7cbecec022dc672df53c4868a7cef9103ee34
Author: Edward Hope-Morley <email address hidden>
Date: Wed Jun 17 13:19:38 2020 +0100

    Add default_log_levels to Mitaka and Ocata nova.conf

    If debug is enabled we will also want oslo_messaging
    debug logs and this allows for that. A previous patch
    added this for >= Pike but it is also needed if these
    logs are wanted in >= Mitaka.

    Change-Id: I8d916e6082ee6fa6140406e1f69bab6bde01b65e
    Related-Bug: #1817290

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/736165
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=5efd1ce88bd599a5ee7f52c1fc59f6a9e946348f
Submitter: Zuul
Branch: master

commit 5efd1ce88bd599a5ee7f52c1fc59f6a9e946348f
Author: Edward Hope-Morley <email address hidden>
Date: Wed Jun 17 12:37:02 2020 +0100

    Add default_log_levels to Mitaka and Ocata nova.conf

    If debug is enabled we will also want oslo_messaging
    debug logs and this allows for that. A previous patch
    added this for >= Pike but it is also needed if these
    logs are wanted in >= Mitaka.

    Related-Bug: #1817290

    Change-Id: I168bd04e3a0332d2ed2a74e411ead3df101402c4