openstack-ansible deployment keeps failing on gnocchi

[TadiosAbebe]

I have been trying to deploy openstack using openstack-ansible for the past two weeks, but the openstack-ansible setup-openstack.yml script keeps failing on setting up gnocchi on "python_venv_build : Build wheels for the packages to be installed into the venv" Task to be specific. I was trying to deploy the Zed release of OpenStack, but the issue still persists when trying to deploy the Antelope release of OpenStack.

To verify that the problem is not in my configuration, I tried to deploy using the openstack aio configuration, but gnocchi still fails.

I have successfully deployed OpenStack before, about 4 weeks ago, with no problem and with the same configuration. I am not sure what seems to be the problem.

When I look at the python log file generated from the venv build, I see a bunch of "# Package would be ignored # warning" about 18 of them; some of those are for the packages gnoochi.cli, gnoochi.incoming, gnoochi.indexer, gnoochi.rest, gnoochi.storage, gnoochi.test, and so on... and it says "AttributeError: module 'setuptools.command.easy_install' has no attribute 'get_script_header'"

At this point, I don't know what to look  for. I will be more than happy to provide additional information.

[Dmitriy Rabotyagov (noonedeadpunk)]

That is known gnocchi issue, related to setuptools: https://github.com/gnocchixyz/gnocchi/issues/1304

As a workaround you can specif this in your user_variables, which will install unaffected version of gnocchi:

gnocchi_git_install_branch: 6f35ea5413a9f78551d8193b8d2a6d77c49b6372

This will be fixed in 27.0.1 release for Antelope, which should be live in the next couple of days.

Regarding Zed - I'm not sure if the fix will be backported, as it will cause downgrade of the version, which might cause regressions for existing deployments.

[TadiosAbebe (tadiosabebe)]

Thank you very much for your time and response. I got the time to test your workaround suggestion today, but it still fails at the same point.

Again, reviewing through the log files of the venv build and wheel build, I see a slight difference from the previous log file I got. The difference being that I don't see an error in the python_venv_build.log file, but in the python_wheel_build.log file, the previous error appeared. I was trying to attach the log files here, but they are very large, with the wheel_build log file being 247MB. So I copied the contents of the python_wheel_build.log file from where I started seeing errors and attached them here.

Any help would be appreciated. Thanks in advance.

[Dmitriy Rabotyagov (noonedeadpunk)]

I'm quite sure that the issue is the same I was referring to.

Can you kindly try to apply this patch as well?
https://review.opendev.org/c/openstack/openstack-ansible/+/886731

Basically it adds a setuptools pin to /opt/openstack-ansible/global-requirement-pins.txt

[TadiosAbebe (tadiosabebe)]

Thank you very much.

Gnocchi's inability to build was fixed by adding setuptools==67.8.0 to /opt/openstack-ansible/global-requirement-pins.txt. However, I should point out that when I commented out
#gnocchi_git_install_branch: 6f35ea5413a9f78551d8193b8d2a6d77c49b6372 inside the user_variables.yml file,
making Gnocchi to be built from a different commit, the wheel failed to build once more. Therefore, I need to apply both of the aforementioned patches for it to work for me.

- adding "gnocchi_git_install_branch: 6f35ea5413a9f78551d8193b8d2a6d77c49b6372" to /etc/openstack_deploy/user_variables.yml
- adding "setuptools==67.8.0" to /opt/openstack-ansible/global-requirement-pins.txt

In addition, I have a couple of questions moving forward, Keeping in mind that I only recently began using openstack and deploying for production,
1. How can I, in the future, remain on top of issues of this nature? cause i spent about 2 weeks dealing with this before coming here
2. What is the suggested place to ask questions and suggestions on openstack-ansible and openstack in general without annoying anyone

Again, Thank you very much for you time.

[Dmitriy Rabotyagov (noonedeadpunk)]

1. Well, usually we're taking care of issues like that, but in this situation there're couple of things that went wrong simultaneously, that made us unable to fix things in time. But usually you should just reach out to us when you getting stuck - we're quite friendly community
2. Usually we suggest reaching us on IRC channel #openstack-ansible on OFTC network. You can also send a ML with `[openstack-ansible]` in topic to the mailing list: https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss