[SRU] rpcapi version mismatch possible on upgrade

Bug #1506257 reported by Edward Hope-Morley on 2015-10-15
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
nova (Ubuntu)
High
Unassigned
Trusty
High
Edward Hope-Morley

Bug Description

[Impact]

Resolves issue described below by making rpc client tolerant of incompatible
remote agent for reserve_block_device_name() calls which can occur during upgrades if compute services are upgraded after clients e.g. nova-api. The proposed fix will cause the client to fallback to a known good/supported api version.

[Test Case]

  * Deploy openstack with all nova services on the same version and test that
    volume operations, particularly attach and detach are working correctly.

  * Deploy Openstack with only Nova client services upgraded (i.e. don't
    upgrade nova-compute) and test that volume operations, particularly attach
    and detach are working correctly.

  * Perform same tests as for 1349888 to ensure the fix is still working.

[Regression Potential]

  None.

The SRU recently landed for https://bugs.launchpad.net/nova/+bug/1349888 introduced a potential upgrade regression if nova services are not upgraded all at once.

2015-10-14 20:45:00.778 10909 TRACE nova.api.openstack RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 3.35
2015-10-14 20:45:00.778 10909 TRACE nova.api.openstack [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 185, in _dispatch\n raise UnsupportedVersion(version)\n', u'UnsupportedVersion: Endpoint does not support RPC version 3.35\n'].

Basically, if nova-compute services are updated after nova-api services you will hit this issue if you perform volume operations. A simple solution, if possible, is to upgrade nova-compute services so that they are all in sync but I still want to remove the possibility for regression while keeping the fix from 1349888. I will propose an SRU shortly to resolve this.

Related branches

Changed in nova (Ubuntu):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Edward Hope-Morley (hopem)
Felipe Reyes (freyes) on 2015-10-15
tags: added: openstack sts
Changed in nova (Ubuntu Trusty):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Edward Hope-Morley (hopem)
Changed in nova (Ubuntu):
status: In Progress → Fix Released
assignee: Edward Hope-Morley (hopem) → nobody
description: updated
summary: - rpcapi version mismatch possible on upgrade
+ [SRU] rpcapi version mismatch possible on upgrade
description: updated
Edward Hope-Morley (hopem) wrote :
tags: added: ubuntu-sponsors

Hello Edward, or anyone else affected,

Accepted nova into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/1:2014.1.5-0ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed
Edward Hope-Morley (hopem) wrote :

I've performed the verification tests described above and all pass.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nova - 1:2014.1.5-0ubuntu1.4

---------------
nova (1:2014.1.5-0ubuntu1.4) trusty; urgency=medium

  * Protect against possible rpcapi mismatch on upgrade (LP: #1506257)
    - d/p/protect-against-upgrade-rpc-ver-mismatch.patch

 -- Edward Hope-Morley <email address hidden> Thu, 22 Oct 2015 10:00:29 -0500

Changed in nova (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers