nova doesn't list services if it can't connect to a cell DB

Bug #1726310 reported by Belmiro Moreira
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Surya Seetharaman
Queens
Fix Committed
Medium
Surya Seetharaman

Bug Description

Description
===========
nova doesn't list services if it can't connect to a child cell DB.

I would expect nova to show the services from all child DBs that it can connect.
For the child DBs that can't connect it can show for the mandatory services (nova-conductor) with the status "not available" and in the disabled reason why ("can't connect to the DB")

Steps to reproduce
==================
Have at least 2 child cells.
Stop the DB in one of them.

"nova service-list" fails with "ERROR (ClientException): Unexpected API Error."
Not given any information about what's causing the problem.

Expected result
===============
List the services of the available cells and list the status of the mandatory services of the affected cells as "not available".

Actual result
=============
$nova service-list
fails.

Environment
===========
nova master (commit: 8d21d711000fff80eb367692b157d09b6532923f)

Changed in nova:
assignee: nobody → Belmiro Moreira (moreira-belmiro-email-lists)
Revision history for this message
Takashi Natsume (natsume-takashi) wrote :

Set the status to In-progress because this report has an assignee.

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/568271

Changed in nova:
assignee: Belmiro Moreira (moreira-belmiro-email-lists) → Surya Seetharaman (tssurya)
Changed in nova:
assignee: Surya Seetharaman (tssurya) → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/568271
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=64e76de43dc55e584c100005fa60da50dd06d352
Submitter: Zuul
Branch: master

commit 64e76de43dc55e584c100005fa60da50dd06d352
Author: Surya Seetharaman <email address hidden>
Date: Mon May 14 13:50:12 2018 +0200

    Make nova service-list use scatter-gather routine

    This patch makes nova service-list use the scatter-gather routine
    so that if a cell is down, at least the services from other cells
    are listed by ignoring the down cell instead of the whole command
    failing with an API exception as is the current situation. Also
    making this query parallel for all cells is more efficient.

    Depends-On: https://review.openstack.org/569112/

    Change-Id: I90b488102eb265d971cade29892279a22d3b5273
    Closes-Bug: #1726310

Changed in nova:
status: In Progress → Fix Released
Matt Riedemann (mriedem)
Changed in nova:
assignee: Matt Riedemann (mriedem) → Surya Seetharaman (tssurya)
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.0.0.0b2

This issue was fixed in the openstack/nova 18.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/579135

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/579136

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/queens)

Reviewed: https://review.openstack.org/579135
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4917d4d0afab6927e0a37290c20bb986ae8f2a41
Submitter: Zuul
Branch: stable/queens

commit 4917d4d0afab6927e0a37290c20bb986ae8f2a41
Author: Surya Seetharaman <email address hidden>
Date: Mon May 14 13:50:12 2018 +0200

    Make nova service-list use scatter-gather routine

    This patch makes nova service-list use the scatter-gather routine
    so that if a cell is down, at least the services from other cells
    are listed by ignoring the down cell instead of the whole command
    failing with an API exception as is the current situation. Also
    making this query parallel for all cells is more efficient.

    Depends-On: https://review.openstack.org/569112/

    Change-Id: I90b488102eb265d971cade29892279a22d3b5273
    Closes-Bug: #1726310
    (cherry picked from commit 64e76de43dc55e584c100005fa60da50dd06d352)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/579136
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=53fc4ab1ddfd2aba3a32936f9f49b37a6e51a198
Submitter: Zuul
Branch: stable/pike

commit 53fc4ab1ddfd2aba3a32936f9f49b37a6e51a198
Author: Surya Seetharaman <email address hidden>
Date: Mon May 14 13:50:12 2018 +0200

    Make nova service-list use scatter-gather routine

    This patch makes nova service-list use the scatter-gather routine
    so that if a cell is down, at least the services from other cells
    are listed by ignoring the down cell instead of the whole command
    failing with an API exception as is the current situation. Also
    making this query parallel for all cells is more efficient.

    Depends-On: https://review.openstack.org/569112/

    Change-Id: I90b488102eb265d971cade29892279a22d3b5273
    Closes-Bug: #1726310
    (cherry picked from commit 64e76de43dc55e584c100005fa60da50dd06d352)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.5

This issue was fixed in the openstack/nova 16.1.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.6

This issue was fixed in the openstack/nova 17.0.6 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.