live_migration_uri to live_migration_scheme SSH settings

[Logan V]

I saw in the Ocata release notes that live_migration_uri is deprecated, and there is mention of a new setting called live_migration_scheme. However, the new config option live_migration_scheme does not appear in the ocata configuration reference[1].

I am also curious how the live_migration_scheme setting could be used to configure a migration URI similar to "qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa" [2] as it seems to only allow setting the scheme to qemu+ssh, but may not offer the ability to configure the ssh settings like the key location and verification.

[1] https://docs.openstack.org/ocata/config-reference/compute/config-options.html
[2] https://github.com/openstack/openstack-ansible-os_nova/commit/7c9a64b2ed972a605ef51b8f8af29ab2453e4b1c#diff-ca98b38be47a1d270f7d2d87697fac8fL279

[Andrey Volkov (avolkov)]

live_migration_uri is deprecated for removal in favor of two other options that
allow to change live migration scheme and target URI: ``live_migration_scheme``
and ``live_migration_inbound_addr`` respectively.

https://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L253

It looks like config-reference is not updated.

[Logan V (loganv)]

The point I have is less about the documentation not being updated.

I don't think live_migration_uri should be deprecated at all. It breaks my live migration scenario and without it, there's no way to configure nova the way I need for migrations to work.

There are a lot of other valid configurations that require specification of libvirt URI options. Just take a look thru https://libvirt.org/remote.html#Remote_URI_reference

[György Szombathelyi (gyurco)]

I agree, we're using
live_migration_uri = qemu+ssh://nova@%s/system?keyfile=/var/lib/nova/.ssh/id_rsa&no_tty=1&no_verify=1

How can it be expressed with the new options?

[Andrey Volkov (avolkov)]

@loganv, @gyurco the change was made in that patch series https://review.openstack.org/#/c/410817
and first started with adding live_migration_inbound_addr in https://review.openstack.org/#/c/245005/.

I didn't know about the case with qemu+ssh you proposed.
An inconvenient workaround here is to use ~/.ssh/config for additional options, though it's per host.

Possibly, @johngarbutt, @stephenfinucane could add some details.

[Oliver Walsh (owalsh)]

Expect we will need an option for each component documented here: https://libvirt.org/guide/html/Application_Development_Guide-Architecture-Remote_URIs.html

[Oliver Walsh (owalsh)]

^H^H^H didn't spot the previous like to https://libvirt.org/remote.html#Remote_URI_reference which appears to be more complete list of options

[John Garbutt (johngarbutt)]

The real need here is having a single key setup for migration and live-migration. The problem is the current configuration options (if you ignore the deprecated ones) don't seem to help do that.

[John Garbutt (johngarbutt)]

Due to the URI config being hard to used due to bugs like this:
https://bugs.launchpad.net/nova/+bug/1677913

Its tempting to add a new configuration to let to specify these SSH configuration settings. They would get ignored if the scheme != ssh.

I think we should note in the description that this is a way to share the ssh key between migrate and live-migration. I would say thats a really useful use case here.

[Oliver Walsh (owalsh)]

Something like this may work:
    virt_type=qemu
    live_migration_schema=ssh
    live_migration_inbound_addr=user@host:port

And add a new option:
    live_migration_extraparmas=no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa

[Oliver Walsh (owalsh)]

s/live_migration_schema/live_migration_scheme/

[Logan V (loganv)]

@Oliver theres a big problem with that, live_migration_inbound_addr is not templated afaik so any time you would like to specify a non standard user or port for your migration config, you lose the ability to dynamically populate the host for each specific nova-compute instance. You'll have to statically populate it on every single config based on some other discovery mechanism.

For me that is a pretty big regression from the current capabilities offered by live_migration_uri.

[Oliver Walsh (owalsh)]

@loganv hmm, indeed. We need new options for user and port.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/456571

[XiaoRuiguo (705437650-6)]

openssh8.0p1 can not support.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/915481

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Stephen Finucane <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/nova/+/456571