UnsupportedVersion Endpoint does not support RPC version 6.0 caused because nova.conf misses RPC pinning for RPC 5.12 -> 6.0 PRC jump during Victoria -> Wallaby

[Aliaksandr Vasiuk]

Hi,

Environment is:
* Focal-Victoria cloud during upgrade to Focal-Wallaby
* Juju 2.9.45
* All charms were refreshed to their victoria/latest on the 29th of May.
* The juju status --relations output: https://pastebin.ubuntu.com/p/p4SNTnnNpk/
* Nova-compute revision after refresh to wallaby/stable is 726
* The sanitized bundle is here: https://pastebin.canonical.com/p/4fJ3JXrkTR/

Cloud is under upgrade from Victoria to Wallaby. I'm doing one by one upgrades.
1. Control plane is all on Wallaby, including nova-cloud-controller
2. Set `juju config nova-compute action-managed-upgrade=true`
3. Refreshed `nova-compute` to 'wallaby/stable'
4. Set `juju config nova-compute openstack-origin="cloud:focal-wallaby"`
5. Ran `juju run-action --wait nova-compute/XX openstack-upgrade`

After that I was able to create VMs on new hosts, migrated VMs from them and to them. But, when I tried to migrate a VM from Victoria host, to Wallaby host, I received:
```
$ os server migrate --os-compute-api-version 2.30 --live-migration --wait 113f23a8-d6a3-4123-974c-18a047ffc32e --host valex-cloud-node-008.cld-valex.com
The --disk-overcommit and --no-disk-overcommit options are only supported by --os-compute-api-version 2.24 or below; this will be an error in a future release
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception_Remote.MigrationError_Remote'> (HTTP 500) (Request-ID: req-2d183c71-36dc-4654-9858-19b2a34ff287)
```
In `nova-compute.log` on the target migration host I see (the full trace is here: https://pastebin.ubuntu.com/p/hQZw2F5s37/)
```
2024-06-04 12:42:14.900 284516 ERROR oslo_messaging.rpc.server [req-2d183c71-36dc-4654-9858-19b2a34ff287 0fbe968551df4c49aaebd671a41453cf f2bfa0b5ee6c4bafb586e6d98f3aff42 - 316c09f31eb64819939290b0ec67797c 316c09f31eb64819939290b0ec67797c] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 6.0. Attempted method: check_can_live_migrate_source
```

I checked that `nova-cloud-controller` assumes RPC 5.12 regardless it is on Wallaby, and it has in its `templates/train/nova.conf` the below
```
[upgrade_levels]
compute = auto
```

From the trace, I checked found the line below in `/usr/lib/python3/dist-packages/nova/compute/rpcapi.py`
```
* 6.0 - Remove 5.x compatibility
```

Unfortunately, nova-compute charm is missing this in its latest template `templates/train/nova.conf`.

The workaround was to add to `templates/train/nova.conf` the same clause
```
[upgrade_levels]
compute = auto
```
and run `juju run -a nova-compute config changed`