Option to boot VM only from volume is not available

[Soumit]

Issue:
When service provider wants to use only boot from volume option for booting a server then the integration tests fails. No option in Tempest to use only boot from volume for booting the server.

Expected :

a parameter in Tempest.conf for option of boot_from_volume_only for all the tests except for image tests.

$ nova boot --flavor FLAVOR_ID [--image IMAGE_ID] / [ --boot-volume BOOTABLE_VOLUME]

[Himanshu Dwivedi (dwivedihimanshu)]

Providing boot only from volume option in tempest will help service provider if they have cloud provision using only bootable volumes.

[vivek sharma (vivek-b-sharma)]

we need to add this , it will help

[Ghanshyam Mann (ghanshyammann)]

So in case nova is configured to only boot from volume (conf option "max_local_block_device" to 0), tempest tests will fail.

I think we need more discussion in team to provide such options. I will discuss this on weekly meeting or IRC.

[Ghanshyam Mann (ghanshyammann)]

Adding Nova on this.

Nova team can better provide how feasible this option is and what are the use cases of this options. Based on that tempest can take decision to support that.

[Ghanshyam Mann (ghanshyammann)]

@Soumit

I tried on my local env by setting 'max_local_block_devices' to 0 in nova.conf and gets below error in Tempest.

ERROR (BadRequest): Block Device Mapping is Invalid: You specified more local devices than the limit allows (HTTP 400) (Request-ID: req-3ef100c7-b5c5-4a2d-a5da-8344726336e2)

Can you please let us know is that all setting you do for making nova boot from volume only. Also provide more information about its use cases etc.

[Nikola Đipanov (ndipanov)]

It is enough to specify --boot-volume option, (see https://wiki.openstack.org/wiki/BlockDeviceConfig for more details about the block device mapping syntax).

Setting max_local_block_devices to 0 means that any request that attempts to create a local disk will fail. This option is meant to limit the number of local discs (so root local disc that is the result of --image being used, and any other ephemeral and swap disks).

AFAIK Tempest by it's very nature will test both booting instances from volumes and from images downloaded to hypervisor local storage, so it makes very little sense to me to attempt to limit the environment tempest runs against to allow only boot from volume and then expect to be able to run tests that spawn instances from images.

 max_local_block_devices set to 0 does not mean that nova will automatically convert --images to volumes and boot instances from volumes - it just means that all request that attempt to create a local disk will fail.

[Soumit (soumit-mishra)]

As a service provider who decides to use only expose boot from volume option,in such a scenario all the tempest test fails as we don't have an option in Tempest to only use boot from volume option. Thus the integration test suite beats its sole purpose to enable service provider to verify the upstream changes continuously.

The use cases is a very practical problem that arises when the service provider limits only to boot from volume due to the below advantages : - even if the server gets deleted the data can be retained and can be used as an image to create a new server ; upgrading the server is also very easy if we have a boot-able volume; also the storage needs can be easily defined due to the control of the volume size and type.
I understand in Nova there isn't any option to just enable boot from volume and disable boot from image, but we should not force any service provider to use the option to use from image also just for testing purpose.
We should consider to provide an option in Tempest as it will fulfill it's basic purpose of integration testing in all openstack environment.

[Joe Gordon (jogo)]

Out of curiosity, how do you configure nova to only expose boot from volume?

Not sure if this is directly related or not, but we don't test boot from volume very well in tempest, and it would be interesting to try running the existing boot from image tests with volume backed instances.

[Nikola Đipanov (ndipanov)]

Right - if there is a case for making nova disallow boot from image (that is stored on a local drive) than max_local_block_devices is little crude as setting it to 0, as it will disallow swap drives too.

Looking at the code - setting root_gb to 0 in the flavor seems like it would do what you want actually - have you tried that?

[Nikola Đipanov (ndipanov)]

Hmmm actually it won't:

        # NOTE(johannes): root_gb is allowed to be 0 for legacy reasons
        # since libvirt interpreted the value differently than other
        # drivers. A value of 0 means don't check size.
        root_gb = instance_type['root_gb']
        if root_gb:
            if int(image.get('size') or 0) > root_gb * (1024 ** 3):
                raise exception.FlavorDiskTooSmall()

            if int(image.get('min_disk') or 0) > root_gb:
                    raise exception.FlavorDiskTooSmall()

It seems like flavor is still the natural place to have this (as opposed to a global disable.

[Matt Riedemann (mriedem)]

We should at least update the help text for max_local_block_devices because someone was asking about an option to enforce only booting instances from volumes and we found that option in the docs with text:

"(IntOpt) Maximum number of devices that will result in a local image being created on the hypervisor node. Setting this to 0 means nova will allow only boot from volume. A negative number means unlimited."

The text "Setting this to 0 means nova will allow only boot from volume." is misleading - you can still boot an instance from an image, the option really just limits the number of local volumes you can boot from.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/220375

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/220375
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6ef48140999e6fe2d23e6e0c61dfdd41e07359e9
Submitter: Jenkins
Branch: master

commit 6ef48140999e6fe2d23e6e0c61dfdd41e07359e9
Author: Matt Riedemann <email address hidden>
Date: Thu Sep 3 18:45:08 2015 -0700

    Clarify max_local_block_devices config option usage

    The previous wording for the max_local_block_devices config option made
    it sound like if you set it to 0 that you could restrict to only booting
    instances from volumes, which is not the case - but is apparently
    something that some operators want (this came up in the related bug and
    in IRC).

    This updates the help text with a description that ndipanov put in the
    bug report which makes the 0 case more clear.

    Co-Authored-By: Nikola Dipanov <email address hidden>

    Related-Bug: #1436314

    Change-Id: I01a9e2f193b50285966690efd771da432a7ad40c

[Markus Zoeller (markus_z) (mzoeller)]

This wishlist bug has been open a year without any activity. I'm going to move it to "Opinion / Wishlist", which is an easily-obtainable queue of older requests that have come on.

In case you want to work on that, consider writing a blueprints [1] and spec [2]. I'll recommend to read [3] if not yet done. The effort to implement the requested feature is then driven only by the blueprint (and spec).

References:
[1] https://blueprints.launchpad.net/nova/
[2] https://github.com/openstack/nova-specs
[3] https://wiki.openstack.org/wiki/Blueprints

[Ken'ichi Ohmichi (oomichi)]

There are not any activity on Tempest side in long-term, and we could not get enough feedback for Nova team questions. It would be nice to drop this bug report from Tempest queue.