[RFE] boot instances from a volume instead of openstack's configured default

[Felipe Reyes]

An OpenStack cloud can be configured to use different backends[0] for the VMs, in some situations a cloud user may want to use a different type for the rootfs

A common setup private clouds is to set the default to qcow2[1] and let users decide when to create a VM where the rootfs should be created in a volume (in Ubuntu OpenStack this is Ceph), this is called "boot from volume"[2] and that will allow the user to preserve their data.

Juju should allow users to decide which kind of instances will be provisioned, I think extending the constraints for the openstack provider could allow this and fit in the UI

juju deploy cs:ubuntu --constraints="root-storage=volume root-disk=20G"

[0] https://docs.openstack.org/ocata/config-reference/compute/hypervisor-kvm.html#configure-compute-backing-storage
[1] ephemeral storage, the rootfs is living in a qcow2 file in the nova-compute local storage, if the instance is migrated to another nova-compute node the rootfs may be rebuilt from the base image. See more details about this at http://www.danplanet.com/blog/2016/03/03/evacuate-in-nova-one-command-to-confuse-us-all/
[2]

[Ivan Hitos (ivanhitos)]

I've checked the juju code. We would need to run something similar to 'openstack boot'. The juju code uses a nova client from gopkg.in [1]. The only function, I saw available looks to be the one that juju code uses[2]. This function is 'RunServer'[3]. It accepts different sort of options [4], and none of them seem to be helpful here,they are more related to 'openstack server create' command. I've read the nova client API from GoPkg and couldn't find what we need.

[1] Nova client API: https://godoc.org/gopkg.in/goose.v2/nova
[2] https://github.com/wallyworld/juju-snap/blob/master/provider/openstack/provider.go#L986
[3] https://godoc.org/gopkg.in/goose.v2/nova#Client.RunServer
[4] https://godoc.org/gopkg.in/goose.v2/nova#RunServerOpts

[Ian Booth (wallyworld)]

The goose client used by Juju doesn't expose the config necessary to enable block device mapping.
From what I can see here https://developer.openstack.org/api-ref/compute/, there's several possible configuration options when using the start instance API:

"block_device_mapping_v2": [{
    "boot_index": "0",
    "uuid": "ac408821-c95a-448f-9292-73986c790911",
    "source_type": "volume",
    "volume_size": "25",
    "destination_type": "volume",
    "delete_on_termination": false,
    "tag": "disk1",
    "disk_bus": "scsi"}]

It wouldn't be practical to specify all of these using constraint syntax. If we did support:

juju deploy cs:ubuntu --constraints="root-storage=volume root-disk=20G"

then we'd need to use sensible defaults for everything else. delete_on_termination defaults to false if not specified and I think that's what you are asking for? But it means you'll need to manage yourself if the instance is deleted. To be clear - you are asking for this to allow users to access any left over data after the nova instance may be deleted. Hence delete_on_termination=false would be what you want.

Would both source_type and destination_type be set to "volume"? Source type also supports blank, snapshot, volume, or image.

Juju would set the tag value to the model UUID I think?

[Felipe Reyes (freyes)]

On Wed, Jan 31, 2018 at 03:47:31AM -0000, Ian Booth wrote:
> The goose client used by Juju doesn't expose the config necessary to enable block device mapping.
> >From what I can see here https://developer.openstack.org/api-ref/compute/, there's several possible configuration options when using the start instance API:
>
> "block_device_mapping_v2": [{
> "boot_index": "0",
> "uuid": "ac408821-c95a-448f-9292-73986c790911",
> "source_type": "volume",
> "volume_size": "25",
> "destination_type": "volume",
> "delete_on_termination": false,
> "tag": "disk1",
> "disk_bus": "scsi"}]
>
> It wouldn't be practical to specify all of these using constraint
> syntax.

agree.

> If we did support:
>
> juju deploy cs:ubuntu --constraints="root-storage=volume root-disk=20G"
>
> then we'd need to use sensible defaults for everything else.
> delete_on_termination defaults to false if not specified and I think
> that's what you are asking for? But it means you'll need to manage
> yourself if the instance is deleted. To be clear - you are asking for
> this to allow users to access any left over data after the nova instance
> may be deleted. Hence delete_on_termination=false would be what you
> want.

I think setting this to false and leaving up to the user the task of
cleaning the volumes they have no interest on makes sense, specially
considering the user will opt-in (via constraints) to ask juju doing
this.

Do you know if juju launches AWS instances backed by EBS volumes?, because on AWS the default is True

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html

> Would both source_type and destination_type be set to "volume"? Source
> type also supports blank, snapshot, volume, or image.

source_type should be image, because we will be using an image from the
simplestreams index

>
> Juju would set the tag value to the model UUID I think?

not sure if model or machine id would be appropriate here, probably the
later, so you could correlate in which machine a given volume was used.

[Ian Booth (wallyworld)]

Juju doesn't support launching instances backed by EBS volumes either, yet.

[Felipe Reyes (freyes)]

I just found this:

"""
For AWS, we tag the root disk EBS volume with:

   key=Name
   value=${INST_ID}-root

So if you add a machine in Juju and it gets assigned the instance ID
"inst-foo", then the root disk EBS volume will have a Name tag with the
value "inst-foo-root".
"""
Source: https://lists.ubuntu.com/archives/juju/2017-October/009640.html

[Ed Stewart (emcs2)]

upvote for this from me - although our use case is slightly different; we need volume backed disks so that we can do backups from Cinder and also dynamic volume resize. So our block would look like this:

"block_device_mapping_v2": [{
    "boot_index": "0",
    "uuid": "ac408821-c95a-448f-9292-73986c790911",
    "source_type": "image",
    "volume_size": "{{ from machine root-disk constraint }}",
    "destination_type": "volume",
    "delete_on_termination": true,
    "tag": "disk1",
    "disk_bus": "scsi"}]

where uuid = source image id

I think you could cover 90% of use cases with:

juju deploy cs:ubuntu --constraints="root-storage=volume root-disk=20G root-disk-autodelete=true/false"

The other properties in block_device_mapping_v2 shouldn't need surfacing back to the juju user.

[Tim Penhey (thumper)]

We discussed using the "root-disk-source" constraint that was added for vSphere as the openstack way to specify "volume" backed disks.

However, I'd suggest that we don't exponse the autodelete option at this stage, and just default to removing the volume when the instance goes away.

[Jorge Niedbalski (niedbalski)]

@thumper,

What about the root-disk-size constraint, is that part of the planned implementation for OpenStack?

Thanks.

[Tim Penhey (thumper)]

Yes, that should already work. If it doesn't please let us know.

[Harry Pidcock (hpidcock)]

https://github.com/juju/juju/pull/10296

[Jorge Niedbalski (niedbalski)]

Tim,

I confirm that both constraints (root-disk-source and root-disk) are indeed working as expected.

ubuntu@niedbalski-bastion:~/juju_config/2.x$ sudo snap refresh juju --candidate --classic
snap "juju" has no updates available
ubuntu@niedbalski-bastion:~/juju_config/2.x$ juju --version
2.6.4-bionic-amd64
ubuntu@niedbalski-bastion:~/juju_config/2.x$ juju add-machine --constraints "root-disk-source=volume root-disk=67G"
created machine 1
ubuntu@niedbalski-bastion:~/juju_config/2.x$ juju machines |grep -E ^1
1 pending pending bionic instance "512668c2-c6ca-4c10-b418-99576d7e7256" has status BUILD, wait 10 seconds before retry, attempt 1
ubuntu@niedbalski-bastion:~/juju_config/2.x$ openstack server show 512668c2-c6ca-4c10-b418-99576d7e7256|grep volumes

| volumes_attached | id='edd6502f-fedd-4781-a2ed-8d41ed9fd6c0'

ubuntu@niedbalski-bastion:~/juju_config/2.x$ openstack volume show edd6502f-fedd-4781-a2ed-8d41ed9fd6c0 -c size |grep 67
| size | 67 |

root@juju-e3eeba-default-1:/home/ubuntu# df -kh|grep -i vda
/dev/vda1 65G 1.4G 64G 3% /
/dev/vda15 105M 3.6M 101M 4% /boot/efi

Thanks for the fix!.