kolla-ansible --limit option is broken when upgrading nova

[Sven Kieske]

Hi,

it might be debatable if we want to support this use case (but I think it might be used in the wild).

regardless here is what doesn't work:

https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/tasks/upgrade.yml#L20C3-L20C11

here we can see that "nova-status upgrade check" is run on the first host in group "nova-api", regardless which --limit option was used.

as the result is registered as a host var via the "register" keyword[0] it can not be accessed directly by other hosts, as a result the check for the result fails: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/tasks/upgrade.yml#L32

a workaround might be to use set_fact or to document that we don't support ansibles --limit option.

This was tested on an upgrade from zed to 2023.1 with ubuntu based hosts.

kind regards

Sven

[0]: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html#registering-variables "Registered variables are only valid on the host for the rest of the current playbook run[...]"

[Sven Kieske (s-kieske)]

so the play was limited to a certain subset of compute nodes, which don't necessarily contain the nova-api nodes. the play then breaks in the next step, because the result is not set from the host var perspective.

[Mark Goddard (mgoddard)]

Thanks for raising this Sven. It would be helpful to provide Ansible output from the failing tasks for context, but I think we can work without it.

Also those links will degrade over time. Here are some permalink equivalents:

https://github.com/openstack/kolla-ansible/blob/add83518345a8621031feb3e14a35902f96672ce/ansible/roles/nova/tasks/upgrade.yml#L20C3-L20C11
https://github.com/openstack/kolla-ansible/blob/add83518345a8621031feb3e14a35902f96672ce/ansible/roles/nova/tasks/upgrade.yml#L32

[Mark Goddard (mgoddard)]

I agree there are some corner cases where our use of delegate_to does not always work. I think the fix may vary, depending on the context, but here are some options/ideas:

- find an intersection of nova-api group and ansible_play_hosts, so that we always delegate to an active host
- fail early if we don't have the necessary hosts for the operation

In this specific case, perhaps performing the 'fail' task using delegate_to and run_once would bring the registered variable into scope?

[Michal Arbet (michalarbet)]

Woudn't be better to just drop delegate as much as we can ?

[Michal Arbet (michalarbet)]

And also group[0] and rather use run_once: true