c-api and c-vol emmit tons of oslo.messaging ERRORs on successful test runs

Bug #1294724 reported by Sean Dague on 2014-03-19
This bug affects 2 people
Affects Status Importance Assigned to Milestone
John Griffith

Bug Description

On a successful test run cinder vol and api services are emmitting tons of oslo.messaging errors in their consoles of the form:

2014-03-18 23:16:34.316 27093 ERROR oslo.messaging.notify._impl_messaging [req-ecce1edc-8383-4fe9-ad11-704ef9deb1e8 fee2f4dc1ff648f0a37e2ef527d9fc20 39971f7616a447e5ad26dbe3ac202a32 - - -] Could not send notification to notifications. ... <payload>

This can be seen in a live log at: http://logs.openstack.org/95/80895/3/check/check-tempest-dsvm-full/bf79bbb/console.html#_2014-03-18_23_20_29_508

This ends up being hugely scary for deployers, and it's lots of ERRORs in logs, even when things are working perfectly normally.

Changed in cinder:
status: New → Confirmed
importance: Undecided → High
milestone: none → icehouse-rc1
tags: added: gate-failure
Luis A. Garcia (luisg-8) wrote :

I am also seeing this in a test environment. I have my cinder running on debug and I see the following TRACE right after the error message: http://paste.openstack.org/show/73844/

Luis A. Garcia (luisg-8) wrote :

I think the bug is actually preventing ceilometer from recognizing cinder resources.

Fix proposed to branch: master
Review: https://review.openstack.org/81651

Changed in cinder:
assignee: nobody → John Griffith (john-griffith)
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/81651
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=e145ef798d2c80ea48be2826f4c648e72958f5d1
Submitter: Jenkins
Branch: master

commit e145ef798d2c80ea48be2826f4c648e72958f5d1
Author: john-griffith <email address hidden>
Date: Wed Mar 19 16:19:28 2014 -0600

    Add RequestContextSerializer for rpc notifications

    RequestContext should be serialized when sent via oslo.messaging. The
    serializer is correctly used for the general RPC mechanism, but has been
    forgotten in the notifier. This patch fixes that.

    Thanks goes to eharney for noticing this and pointing it out!!!

    Same bug existed in Nova (1275771)

    Change-Id: I99a571ef74bbcd13bf801313eea145ea69f821e1
    Closes-Bug: #1294724

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2014-03-27
Changed in cinder:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in cinder:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers