impossible to disable notifications

Bug #1761405 reported by Junien F
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Matt Riedemann
Pike
Fix Committed
Low
Matt Riedemann
Queens
Fix Committed
Low
Matt Riedemann

Bug Description

Hi,

As far as I can see from https://github.com/openstack/nova/blob/stable/queens/nova/rpc.py#L60-L86, it's currently (i.e. in queens) not possible to instruct nova to not send notifications - you either get versioned, legacy, or both.

In clouds which don't have ceilometer, nothing is consuming the notifications, which keep piling up in rabbitmq :

$ sudo rabbitmqctl list_queues -p openstack|grep -v 0$
Listing queues ...
notifications.error 32
notifications.info 44908
notifications_designate.info 15161
versioned_notifications.error 24
versioned_notifications.info 22285

Could we please get a knob to make nova not send any notification ?

Thanks !

Revision history for this message
Matt Riedemann (mriedem) wrote :

I think this might be a duplicate, I'm pretty sure I've talked with gibi about this before.

tags: added: notifications
Revision history for this message
Matt Riedemann (mriedem) wrote :

There are other services that rely on nova notifications beyond ceilometer, like searchlight, designate and mistral, there might be others. But you might not be using any of those, so it is a valid request to be able to disable notifications altogether.

Changed in nova:
status: New → Confirmed
Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
Matt Riedemann (mriedem) wrote :

One thing I think you can do as a workaround is to set the notification driver to 'noop':

https://docs.openstack.org/nova/latest/configuration/config.html#oslo-messaging-notifications

So:

[oslo_messaging_notifications]
driver = noop

in nova.conf; if you set that, nova will still do the work of generating the payload but it won't actually go over RPC.

Changed in nova:
importance: Medium → Low
Revision history for this message
Matt Riedemann (mriedem) wrote :

Can you test out the noop workaround suggested in comment 3? If that works for you, then I think this just becomes a docs fix in the nova notifications docs.

Revision history for this message
Junien F (axino) wrote :

Hi,

Thanks for the quick reply. Sure, I'll test the setting. I did see this knob before, but I thought it would also impact messages that nova critically needs to work normally.

I will let you know at the beginning of next week.

Revision history for this message
Junien F (axino) wrote :

Indeed setting :

[oslo_messaging_notifications]
driver = noop

on neutron-server and nova components got rid of the notifications.

Thanks !

Alvaro Uria (aluria)
tags: added: canonical-bootstack
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

I feel that our current knob oslo_messaging_notifications.driver = noop is enough to turn off the notifications. But I agree that this needs to be documented a bit better in the notification dev ref and in the description of the notification_format config option.

@Matt (comments#3): Actually nova skips the expensive part of the versioned payload generation if the driver is set to 'noop' [1][2]. So I think we are OK to just document the possible config a bit better.

[1] https://github.com/openstack/nova/blob/06ad7dffa60dc4b38be3ca1b44db4bfb2f4c052c/nova/rpc.py#L218
[2] https://github.com/openstack/oslo.messaging/blob/393ecff3451091404832dd6b8a088e1bec760101/oslo_messaging/notify/notifier.py#L430

Revision history for this message
Matt Riedemann (mriedem) wrote :

> @Matt (comments#3): Actually nova skips the expensive part of the versioned payload generation if the driver is set to 'noop' [1][2]. So I think we are OK to just document the possible config a bit better.

Sweet, easy fix then, just docs.

Matt Riedemann (mriedem)
Changed in nova:
assignee: nobody → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/561684

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/561684
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=11528966ba000addde5416f085332e0d39c46588
Submitter: Zuul
Branch: master

commit 11528966ba000addde5416f085332e0d39c46588
Author: Matt Riedemann <email address hidden>
Date: Mon Apr 16 14:12:12 2018 -0400

    Document how to disable notifications

    This adds information to the "notification_format" config
    option help and notifications docs on how to disable notifications.

    While updating the config option help text, a stale reference
    to Pike is removed.

    Change-Id: I736025a0a88fc969831558805687b642da8cd365
    Closes-Bug: #1761405

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/561954

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/queens)

Reviewed: https://review.openstack.org/561954
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c0db837835438f89a10e2dfb130db2f39cb18477
Submitter: Zuul
Branch: stable/queens

commit c0db837835438f89a10e2dfb130db2f39cb18477
Author: Matt Riedemann <email address hidden>
Date: Mon Apr 16 14:12:12 2018 -0400

    Document how to disable notifications

    This adds information to the "notification_format" config
    option help and notifications docs on how to disable notifications.

    While updating the config option help text, a stale reference
    to Pike is removed.

    Change-Id: I736025a0a88fc969831558805687b642da8cd365
    Closes-Bug: #1761405
    (cherry picked from commit 11528966ba000addde5416f085332e0d39c46588)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.0.0.0b1

This issue was fixed in the openstack/nova 18.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/563254

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.3

This issue was fixed in the openstack/nova 17.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/563254
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9a731a05ab6a67a51f774e24e46d57ca5a5920f8
Submitter: Zuul
Branch: stable/pike

commit 9a731a05ab6a67a51f774e24e46d57ca5a5920f8
Author: Matt Riedemann <email address hidden>
Date: Mon Apr 16 14:12:12 2018 -0400

    Document how to disable notifications

    This adds information to the "notification_format" config
    option help and notifications docs on how to disable notifications.

    While updating the config option help text, a stale reference
    to Pike is removed.

    Change-Id: I736025a0a88fc969831558805687b642da8cd365
    Closes-Bug: #1761405
    (cherry picked from commit 11528966ba000addde5416f085332e0d39c46588)
    (cherry picked from commit c0db837835438f89a10e2dfb130db2f39cb18477)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.4

This issue was fixed in the openstack/nova 16.1.4 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.5

This issue was fixed in the openstack/nova 16.1.5 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.