c-api and c-vol emmit tons of oslo.messaging ERRORs on successful test runs

[Sean Dague]

On a successful test run cinder vol and api services are emmitting tons of oslo.messaging errors in their consoles of the form:

2014-03-18 23:16:34.316 27093 ERROR oslo.messaging.notify._impl_messaging [req-ecce1edc-8383-4fe9-ad11-704ef9deb1e8 fee2f4dc1ff648f0a37e2ef527d9fc20 39971f7616a447e5ad26dbe3ac202a32 - - -] Could not send notification to notifications. ... <payload>

This can be seen in a live log at: http://logs.openstack.org/95/80895/3/check/check-tempest-dsvm-full/bf79bbb/console.html#_2014-03-18_23_20_29_508

This ends up being hugely scary for deployers, and it's lots of ERRORs in logs, even when things are working perfectly normally.

[Luis A. Garcia (luisg-8)]

I am also seeing this in a test environment. I have my cinder running on debug and I see the following TRACE right after the error message: http://paste.openstack.org/show/73844/

[Luis A. Garcia (luisg-8)]

I think the bug is actually preventing ceilometer from recognizing cinder resources.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/81651

[John Griffith (john-griffith)]

On to the next item:
http://logs.openstack.org/51/81651/1/check/check-tempest-dsvm-full/f8ffd2a/logs/screen-c-api.txt.gz#_2014-03-19_23_22_33_186

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/81887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/81651
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=e145ef798d2c80ea48be2826f4c648e72958f5d1
Submitter: Jenkins
Branch: master

commit e145ef798d2c80ea48be2826f4c648e72958f5d1
Author: john-griffith <email address hidden>
Date: Wed Mar 19 16:19:28 2014 -0600

    Add RequestContextSerializer for rpc notifications

    RequestContext should be serialized when sent via oslo.messaging. The
    serializer is correctly used for the general RPC mechanism, but has been
    forgotten in the notifier. This patch fixes that.

    Thanks goes to eharney for noticing this and pointing it out!!!

    Same bug existed in Nova (1275771)

    Change-Id: I99a571ef74bbcd13bf801313eea145ea69f821e1
    Closes-Bug: #1294724