nova-api returns empty block-device-mapping in metadata queries

Bug #1881944 reported by Alexander Diana on 2020-06-03
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
Balazs Gibizer
Train
Medium
Balazs Gibizer
Ussuri
Medium
Balazs Gibizer
Victoria
Medium
Balazs Gibizer

Bug Description

Description
===========
currently in a single cell deployment with nova-api acting as metadata server, queries for block-device-mapping end up going to the nova_cell0/block_device_mapping table instead of nova${the_actual_cell}/block_device_mapping, resulting in empty results other than the dynamically generated root/ami entries.

This results in an empty response, even though the devices are mapped and present in the appropriate table.

Steps to reproduce
==================
1. Use nova train or greater (tested on stable/train current HEAD)
2. curl 169.254.169.254/latest/meta-data/block-device-mapping/ from an instance with volumes attached
3. receive only root and ami, no entries for volumes

Expected result
===============
There should be entries in block-device-mapping queries for each volume mapped to the instance.

Actual result
=============
There are no entries other than root/ami in the block-device-mapping

Notes
===========
This is "fixed" by pointing nova-api at the nova database instead of nova_cell0

This bug affects Train+, but likely any deployment with multiple cells.

Alexander Diana (rouk) on 2020-06-03
description: updated
tags: added: cells metadata volumes
Balazs Gibizer (balazs-gibizer) wrote :

I confirm that the bug still exists on current master.

The root cause is that we are not passing the cell targeted context to the InstanceMetadata class and therefore the BDM query runs without a targeted context. See https://github.com/openstack/nova/blob/f72c9e59fb6bea881c16e710df7bd676fafe18f7/nova/api/metadata/base.py#L693-L695

Changed in nova:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Balazs Gibizer (balazs-gibizer)

Fix proposed to branch: master
Review: https://review.opendev.org/752459

Changed in nova:
status: Triaged → In Progress

Reviewed: https://review.opendev.org/752459
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1390eecf8dec4c3b022a9b1e259a908197566738
Submitter: Zuul
Branch: master

commit 1390eecf8dec4c3b022a9b1e259a908197566738
Author: Balazs Gibizer <email address hidden>
Date: Mon Sep 21 17:21:18 2020 +0200

    Use cell targeted context to query BDMs for metadata

    The metadata service supports a multicell deployment in a configuration
    where the nova-api service implements the metadata API. In this case the
    metadata query needs to be cell targeted. This was partly implemented
    already. The instance itself is queried from the cell DB properly.
    However the BDM data used the non targeted context resulting in an empty
    BDM returned by the metadata service.

    Functional reproduction test is not added as I did not find a way to
    have a cell setup in the functional test that reproduce the problem. I
    reproduced the bug and tested the fix in a devstack.

    Change-Id: I48f57082edaef3ec4722bd31ce29a90b94d32523
    Closes-Bug: #1881944

Changed in nova:
status: In Progress → Fix Released
Balazs Gibizer (balazs-gibizer) wrote :

fix merged to stable/victoria: https://review.opendev.org/761424

Balazs Gibizer (balazs-gibizer) wrote :

Fix proposed to branch stable/ussuri https://review.opendev.org/c/openstack/nova/+/765748

Balazs Gibizer (balazs-gibizer) wrote :

This issue was fixed in the openstack/nova 22.1.0 release.

This issue was fixed in the openstack/nova 21.1.2 release.

This issue was fixed in the openstack/nova 20.6.0 release.

This issue was fixed in the openstack/nova 23.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers