Live migration is not working

Bug #2061041 reported by Viktor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
New
Undecided
Unassigned

Bug Description

**Bug Report**

Fresh install kolla-ansible on ubuntu 22.04

Live migration fails with errors:

```
openstack server migrate --live-migration --host openstack-compute01 --shared-migration test-migration01 --os-compute-api-version 2.30

Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception_Remote.MigrationError_Remote'> (HTTP 500) (Request-ID: req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38)
```

nova-api.log
```
2024-04-11 23:21:45.730 19 INFO oslo_messaging._drivers.amqpdriver [None req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38 e020e8fcbdab4a73979b0fb13fd64f03 93484e5d572048acafdd76f1bc3a53
68 - - default default] Creating reply queue: reply_222c0b39493648a197e81c6a3fbc74bd
2024-04-11 23:21:45.747 19 INFO oslo_messaging._drivers.amqpdriver [None req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38 e020e8fcbdab4a73979b0fb13fd64f03 93484e5d572048acafdd76f1bc3a53
68 - - default default] Expecting reply to msg d6844c2b751e4fe4ac97f6ae2187eb2a in queue reply_222c0b39493648a197e81c6a3fbc74bd
2024-04-11 23:21:48.581 19 INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d6844c2b751e4fe4ac97f6ae2187eb2a
2024-04-11 23:21:48.582 19 ERROR nova.api.openstack.wsgi [None req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38 e020e8fcbdab4a73979b0fb13fd64f03 93484e5d572048acafdd76f1bc3a5368 - - def
ault default] Unexpected exception in API method: nova.exception_Remote.MigrationError_Remote: Migration error: rule:create_port_binding is disallowed by policy
```

neutron-server.log
```
/var/log/kolla/neutron/neutron-server.log:2024-04-11 23:21:48.333 26 INFO neutron.api.v2.resource [req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38 req-90d64573-16b3-438e-8d17-7634cca3564b d5dae2221eaa462188a755e96d690b12 58913605d6e94db69674b54ccbbc7c67 - - default default] create failed (client error): Access was denied to this resource.
/var/log/kolla/neutron/neutron-server.log:2024-04-11 23:21:48.334 26 INFO neutron.wsgi [req-f2e89369-4c6a-49b2-8d4b-bcf2f982dc38 req-90d64573-16b3-438e-8d17-7634cca3564b d5dae2221eaa462188a755e96d690b12 58913605d6e94db69674b54ccbbc7c67 - - default default] 192.168.21.202,192.168.21.201 "POST /v2.0/ports/40a62bc0-e2db-49d8-b5a0-0e58b7bed62f/bindings HTTP/1.1" status: 403 len: 308 time: 0.0102177
```

Run as admin
```
openstack token issue
+------------+-------------------------------------------------------------------------+
| Field | Value |
+------------+-------------------------------------------------------------------------+
| expires | 2024-04-12T23:33:50+0000 |
| id | gAAAAABmGHNeK3YCm4X3w1hjeXAd7a_anRQYxlz84eSz0QwKEZ_1pYGyXkbzIG1RIQ9ovLE |
| | Vm2Ji7BREyQCT_D6z7RZR2qVvGEmUrCN8WO7mCrcUoPR9PBc14dNtEikWgqiQlqD_c4jTpV |
| | OoxvQ2gbnV_bywfSktnzT8jfuerHRK99AVroF3hqQ |
| project_id | 93484e5d572048acafdd76f1bc3a5368 |
| user_id | e020e8fcbdab4a73979b0fb13fd64f03 |
+------------+-------------------------------------------------------------------------+
```

```
 openstack user list
+----------------------------------+-------------------+
| ID | Name |
+----------------------------------+-------------------+
| e020e8fcbdab4a73979b0fb13fd64f03 | admin |
| c4e44e4f7b5d43b296fd6f650adef639 | heat_domain_admin |
| b0aeca8994a44d3e90839fad9f8c7524 | glance |
| 891bc21614ec49548a2e9c572ecd061c | cinder |
| b19e08871b1544ed91fd18349cf3e8c6 | nova |
| d5dae2221eaa462188a755e96d690b12 | neutron |
| acecc374af42400ca23f58412938b584 | heat |
| b66cedfe61f64cd29ed98206635456b2 | placement |
| 94905698e8704dca9b35358a71b44437 | skyline |
+----------------------------------+-------------------+
```

My globals.yml
```
 grep -vP '^$|^\s*#' /etc/kolla/globals.yml
---
workaround_ansible_issue_8743: yes
kolla_base_distro: "ubuntu"
kolla_internal_vip_address: "192.168.21.200"
network_interface: "eth0"
neutron_external_interface: "ens19"
enable_central_logging: "yes"
enable_cinder: "yes"
enable_cinder_backup: "yes"
enable_grafana: "yes"
enable_neutron_agent_ha: "yes"
enable_neutron_provider_networks: "yes"
enable_prometheus: "yes"
enable_skyline: "yes"
ceph_glance_user: "glance"
ceph_glance_keyring: "client.{{ ceph_glance_user }}.keyring"
ceph_glance_pool_name: "images"
ceph_cinder_user: "cinder"
ceph_cinder_keyring: "client.{{ ceph_cinder_user }}.keyring"
ceph_cinder_pool_name: "volumes"
ceph_cinder_backup_user: "cinder-backup"
ceph_cinder_backup_keyring: "client.{{ ceph_cinder_backup_user }}.keyring"
ceph_cinder_backup_pool_name: "backups"
ceph_nova_keyring: "{{ ceph_cinder_keyring }}"
ceph_nova_user: "{{ ceph_cinder_user }}"
ceph_nova_pool_name: "vms"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"
```

my top of my multinode
```
[control]
# These hostname must be resolvable from your deployment host
openstack-controller01 ansible_host=192.168.21.201
openstack-controller02 ansible_host=192.168.21.202
openstack-controller03 ansible_host=192.168.21.203

# The above can also be specified as follows:
#control[01:03] ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
openstack-controller01 ansible_host=192.168.21.201
openstack-controller02 ansible_host=192.168.21.202
openstack-controller03 ansible_host=192.168.21.203

[compute]
openstack-compute01 ansible_host=192.168.21.211
openstack-compute02 ansible_host=192.168.21.212
openstack-compute03 ansible_host=192.168.21.213

[monitoring]
openstack-monitor ansible_host=192.168.21.210

# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 tunnel_interface=em1

[storage]
openstack-compute01 ansible_host=192.168.21.211
openstack-compute02 ansible_host=192.168.21.212
openstack-compute03 ansible_host=192.168.21.213

[deployment]
localhost ansible_connection=local
```

**Environment**:
Ubuntu 22.04
5.15.0-101-generic
docker version 26.0.0
kolla-ansible @ git+https://opendev.org/openstack/kolla-ansible@01e3fac180019b5a55f4718b977d176f8ae3e0f6
Docker image Install type binary
Docker image distribution official

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.