Misleading log message "Booting with blank volume" in nova-compute when booting from real volume

Bug #1816938 reported by Matt Riedemann
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Takashi Natsume
Rocky
Fix Committed
Medium
Takashi Natsume
Stein
Fix Committed
Medium
Takashi Natsume

Bug Description

Something is broken here:

https://github.com/openstack/nova/blob/af78b13c24d4abf393d17ac57e9135204ef12b73/nova/virt/block_device.py#L836

Because I'm seeing this getting logged:

https://github.com/openstack/nova/blob/af78b13c24d4abf393d17ac57e9135204ef12b73/nova/virt/block_device.py#L855

When I know I'm booting from an existing volume. This is also all over the CI logs:

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Booting%20with%20blank%20volume%20at%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d

My guess is something with dict get() access changed on the DriverVolumeBlockDevice code with this change:

https://github.com/openstack/nova/commit/b958bf1126aea8b88ccebb43a330fc1a44717145#diff-40dadeaa854473fb72fa4bf3491a434f

If I change the code to check:

if bdm.volume_id:
   ...

Then I get the expected log message but that is probably dangerous if we have a non-volume BDM.

Changed in nova:
assignee: nobody → Takashi NATSUME (natsume-takashi)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/638821

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.openstack.org/647646

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/647651

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/638821
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3c66b40dbd23e3f792a86da5a15c993c52c9b377
Submitter: Zuul
Branch: master

commit 3c66b40dbd23e3f792a86da5a15c993c52c9b377
Author: Takashi NATSUME <email address hidden>
Date: Sat Feb 23 00:56:23 2019 +0900

    Override the 'get' method in DriverBlockDevice class

    The following methods are overridden in DriverBlockDevice class.

    * __getattr__
    * __getitem__

    The 'get' method is not overridden.
    The value cannot be got by the 'get' method
    though the value can be got by '__getattr__' (e.g. bdm.volumd_id)
    or '__getitem__' (e.g. bdm['volume_id']) method.
    So override the 'get' method to fix the issue.

    Change-Id: Ic665fc1956831110937d98553e526cb909e49997
    Closes-Bug: #1816938

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/stein)

Reviewed: https://review.openstack.org/647646
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ae40ceba8420b3b6c4bd8e29cf95ec21fd20e1e8
Submitter: Zuul
Branch: stable/stein

commit ae40ceba8420b3b6c4bd8e29cf95ec21fd20e1e8
Author: Takashi NATSUME <email address hidden>
Date: Sat Feb 23 00:56:23 2019 +0900

    Override the 'get' method in DriverBlockDevice class

    The following methods are overridden in DriverBlockDevice class.

    * __getattr__
    * __getitem__

    The 'get' method is not overridden.
    The value cannot be got by the 'get' method
    though the value can be got by '__getattr__' (e.g. bdm.volumd_id)
    or '__getitem__' (e.g. bdm['volume_id']) method.
    So override the 'get' method to fix the issue.

    Change-Id: Ic665fc1956831110937d98553e526cb909e49997
    Closes-Bug: #1816938
    (cherry picked from commit 3c66b40dbd23e3f792a86da5a15c993c52c9b377)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.openstack.org/647651
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8d757888a8179e5bfbdfee26eb672590f964f574
Submitter: Zuul
Branch: stable/rocky

commit 8d757888a8179e5bfbdfee26eb672590f964f574
Author: Takashi NATSUME <email address hidden>
Date: Sat Feb 23 00:56:23 2019 +0900

    Override the 'get' method in DriverBlockDevice class

    The following methods are overridden in DriverBlockDevice class.

    * __getattr__
    * __getitem__

    The 'get' method is not overridden.
    The value cannot be got by the 'get' method
    though the value can be got by '__getattr__' (e.g. bdm.volumd_id)
    or '__getitem__' (e.g. bdm['volume_id']) method.
    So override the 'get' method to fix the issue.

    Conflicts:
        nova/tests/unit/virt/test_block_device.py

        The conflict is due to not including
        Icc301230fe7c8e3ebbbcc7f4a807e562db7f93e3 in rocky.

    Change-Id: Ic665fc1956831110937d98553e526cb909e49997
    Closes-Bug: #1816938
    (cherry picked from commit 3c66b40dbd23e3f792a86da5a15c993c52c9b377)
    (cherry picked from commit ae40ceba8420b3b6c4bd8e29cf95ec21fd20e1e8)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.0.1

This issue was fixed in the openstack/nova 19.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.2.1

This issue was fixed in the openstack/nova 18.2.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 20.0.0.0rc1

This issue was fixed in the openstack/nova 20.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.