horizon unable to create a swift container

[Vince Mulhollon]

Environment is a fresh install of Yoga on Ubuntu source install.

If you follow the installation instructions the CLI for swift will work, you can run "openstack container create demo" to create a container named demo, etc.

However if you try to use Horizon web interface, "Object Store", "Containers" you will get a error message popup about unable to access storage policies.

There are two independent problems.

One is when following the instructions a ring is configured with index number zero which is the default. The command line interface will set up a new container with default index number of zero if you do not specify a storage policy, so the CLI works fine. Horizon wants to pull a list of valid storage policies. No such list exists under the default Kolla-Ansible configuration (check the contents of /etc/kolla/swift-proxy-server/swift.conf). You can fix this by adding the following file:

/etc/kolla/config/swift.conf :
[storage-policy:0]
name = Policy-0
default = true

This gives a name to policy index number zero. Therefore Horizon can display that name in the dropdown.

The other problem is more complex and you can see in the log messages that there is a swift authentication problem but ONLY when doing storage policy related "stuff".

The Swift Auth system docs explain what's going on. For some reason (I don't know why) the delay_auth_decision default value is false which forbids "authenticated capabilities requests". I have no idea why the default is not true.

https://docs.openstack.org/swift/latest/overview_auth.html

The easiest way to fix this is to add another line to a file. I do all my configuration in /etc/kolla/globals.d instead of in /etc/kolla/globals.yml so I have to add one line to this file:

/etc/kolla/globals.d/swift.yml needs this line added:
swift_delay_auth_decision: "yes"

Then a quick "kolla-ansible -i ./multinode --tags swift reconfigure"

After that, if you go into Horizon, "Object Store", "Containers" you will no longer get a pop-up message about unable to obtain list of policies, AND if you try to create a container in the web UI, the dropdown for "Storage Policy" will populate, AND the "submit" button is no longer grayed out, AND container creation is successful. Cool.

I can reproduce this bug at my leisure by removing those two files listed above on a fresh installation of Yoga.

I believe the partially configured policy in swift.conf is a genuine bug, as if you follow the instructions to install Kolla, policy index number zero is properly created and works with the CLI but the policy is not completely created so merely adding a name to swift.conf makes Horizon work.

On the other hand I have no idea why delay_auth_decision is default no, as that seems to break lots of things per the docs. Perhaps there is some obscure security implication. At any rate this could be considered a bug where the default should be yes instead of no, OR it could be considered a bug in the docs where end users should be informed the web UI in Horizon is broken unless this is set to "yes".

Anyway, I found a bug, I provided a working fix above, best of luck, and have a great day!

[Vince Mulhollon (vincemulhollon)]

I found another problem relating to "swift_delay_auth_decision"

If that is set to the default "no" and you configure cinder to do volume backups to swift and try to do a backup to a container you'll get the following error in the cinder_backup logs:

2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift [-] Can not get Swift capabilities during backup drive
r initialization.: swiftclient.exceptions.ClientException: Capabilities GET failed: http://10.10.20.62:8080/info 40
1 Unauthorized [first 60 chars of response] b'{"error": {"code": 401, "title": "Unauthorized", "message": ' (txn:
txcb5b4cd60cdf4fa1bcb25-0062c234ff)
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift Traceback (most recent call last):
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift File "/var/lib/kolla/venv/lib/python3.8/site-package
s/cinder/backup/drivers/swift.py", line 420, in check_for_setup_error
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift conn.get_capabilities()
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift File "/var/lib/kolla/venv/lib/python3.8/site-package
s/swiftclient/client.py", line 2059, in get_capabilities
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift return get_capabilities((parsed, self.http_conn[1]
))
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift File "/var/lib/kolla/venv/lib/python3.8/site-package
s/swiftclient/client.py", line 1682, in get_capabilities
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift raise ClientException.from_response(
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift swiftclient.exceptions.ClientException: Capabilities G
ET failed: http://10.10.20.62:8080/info 401 Unauthorized [first 60 chars of response] b'{"error": {"code": 401, "t
itle": "Unauthorized", "message": ' (txn: txcb5b4cd60cdf4fa1bcb25-0062c234ff)
2022-07-03 19:31:59.315 18 ERROR cinder.backup.drivers.swift
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'cinder.backup.manager.Ba
ckupManager._setup_backup_driver' failed: swiftclient.exceptions.ClientException: Capabilities GET failed: http://1
0.10.20.62:8080/info 401 Unauthorized [first 60 chars of response] b'{"error": {"code": 401, "title": "Unauthorize
d", "message": ' (txn: txcb5b4cd60cdf4fa1bcb25-0062c234ff)
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall Traceback (most recent call last):
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/o
slo_service/loopingcall.py", line 150, in _run_loop
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw)
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/c
inder/backup/manager.py", line 162, in _setup_backup_driver
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall backup_service.check_for_setup_error()
2022-07-03 19:31:59.317 18 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/c
inder/backup/drivers/swift.py", line 420, in check_for_setup_error
2022-07-03 19:31:59.317 18 ...

[alpha23 (alpha23)]

Vince, thank you very much for posting this along with the solution. I discovered the same in Ussuri (centos/source) so I can assume it is also in Victoria, Wallaby, and Xena, and well as Yoga as you discovered. The is is not the first swift related configuration issue that has remained unfixed for lengthy periods of time. I'm unsure as to why these items are being completely overlooked and un-tested by the kolla-ansible team, and in some cases well after their discovery by users, e.g., https://bugs.launchpad.net/kolla-ansible/+bug/1776830. As you mentioned, it is not possible to create a container via Horizon without your fix.