Grenade-multinode jobs don't upgrade services on subnodes

[Slawek Kaplonski]

It seems that Openstack services, like n-cpu or q-agt (neutron-agent) aren't upgraded, or at least aren't restarted on subnodes during the grenade-multinode CI jobs. See for example logs from job: https://zuul.opendev.org/t/openstack/build/c8ddb8d66284486892180bfe5c94a9c4/logs

On the subnode, in neutron-agent logs there is only one line like:

Oct 02 22:26:06.369137 ubuntu-bionic-rax-ord-0020251000 systemd[1]: Started Devstack <email address hidden>.

Which means that service was just started at the beginning of the CI job.

In the same job on controller node, it is restarted:

Oct 02 22:14:56.824461 ubuntu-bionic-rax-ord-0020250993 systemd[1]: Started Devstack <email address hidden>.
Oct 02 22:48:26.809811 ubuntu-bionic-rax-ord-0020250993 systemd[1]: Started Devstack <email address hidden>.

And it is similar for e.g. n-cpu service, so it seems not related to e.g. Neutron but it's more like grenade job's issue.

[Hang Yang (hangyang)]

I checked the grenade docs and see it intentionally skips upgrade and restart for nova-compute https://github.com/openstack/grenade#status but I'm wondering if it also accidentally skips the neutron-agent on compute node.

[Lee Yarwood (lyarwood)]

I think it's by design that both n-cpu and q-agt aren't updated on the subnode to allow N-1 to N live migration testing, at least that was my understanding.