Cannot create a swift container, mandatory "Storage Policy" dropdown field is empty

[Przemyslaw Hausman]

[Impact]
focal-ussuri, charms revision 20.10

I can't create a container with GUI. When entering "Containers" tab, the error "Error: Unable to fetch the policy details." appears.

If I proceed and try to create a container, I can't click "Submit" button until I select a Storage Policy. But the mandatory "Storage Policy" dropdown field is empty. See attached screenshot.

I can, however, create a container with a CLI. `openstack container create c1` works fine.

[Test Case]
Deploy horizon with ceph radosgw instead of swift, navigate to http://horizon-ip/horizon/project/containers/ and attempt to create a container. With the package and charm fixes in this bug, it will be successful. Otherwise it will fail as described in this bug.

Note, we also want to test horizon with swift proper for this bug to ensure it still works with the dashboard.

[Regression Potential]
It's possible this could regress horizon with swift proper as a backend so it should also be regression tested.

[Przemyslaw Hausman (phausman)]

Attached (error-log-storage-policy.txt) see relevant error messages from /var/log/apache2/error.log on one of the openstack-dashboard units, with debugging enabled (juju config openstack-dashboard debug=true).

[Przemyslaw Hausman (phausman)]

Subscribing ~field-critical as this issue occurs on a customer deployment and no workaround is known yet.

[Corey Bryant (corey.bryant)]

Hi Przemyslaw,

Thanks for reporting this. Can this be done via the command line and would that be considered a valid work-around?

Thanks,
Corey

[Corey Bryant (corey.bryant)]

In the mean time, I'm taking a look.

[Przemyslaw Hausman (phausman)]

Next time I checked creating the container using CLI, it was failing. I think I tested it after redeployment though.

I will re-test again once I have OpenStack up and runnig (I'm redeploying now).

[Corey Bryant (corey.bryant)]

Przemyslaw,

I've just deployed focal-ussuri and was unable to recreate this. Actually ussuri-proposed but I don't think that makes a difference. I'm attaching screen shots. Note in the 2nd screen shot the version of horizon in the bottom right, 18.3.2.

Do you have a bundle snippet that would help me recreate?

Thanks,
Corey

[Corey Bryant (corey.bryant)]

The reason I wasn't able to recreate this is because the deployment is not using swift at all, it is using ceph-radosgw and attempting to create containers via the openstack dashboard. Swift now has storage policies [1] which likely don't map to ceph, so the drop-down in horizon is empty. I don't think the openstack dashboard is designed to work with anything but swift proper. ceph-radosgw publishes a swift API (a subset of it I believe) and so the openstack dashboard may work since it is using the swift API, but I think it's an unsupported "your milage may vary" situation. I'll get confirmation from another openstack team member before triaging this.

[1] https://docs.openstack.org/swift/latest/overview_policies.html

[Corey Bryant (corey.bryant)]

Good news! This may already be fixed upstream:

https://bugs.launchpad.net/horizon/+bug/1880188
https://bugs.launchpad.net/python-swiftclient/+bug/1712358

I'm going to test this out with a PPA.

[Corey Bryant (corey.bryant)]

I'm able to recreate this now using ceph-radosgw instead of swift.

[Corey Bryant (corey.bryant)]

I've tested this out and it is fixed by https://bugs.launchpad.net/python-swiftclient/+bug/1712358.

Note that in order for this to work with the admin endpoint it needs to be updated with a /v1 suffix. By default the charm deployment of ceph-radosgw is missing the /v1 on the admin endpoint:

$ openstack endpoint list # snipped to be brief
+----------------------------------+--------------+-----------+------------------------------+
| ID | Service Name | Interface | URL |
+----------------------------------+--------------+-----------+------------------------------+
| 6a479719e0684596874ec7e8d14b1c0c | swift | internal | http://10.5.0.33:80/swift/v1 |
| d8be68238b8342d5851a8593af5cdb7e | swift | public | http://10.5.0.33:80/swift/v1 |
| e746487b95104820a9e19c76ff006dae | swift | admin | http://10.5.0.33:80/swift |
+----------------------------------+--------------+-----------+------------------------------+

[Corey Bryant (corey.bryant)]

This is already fixed in 1:3.10.1-0ubuntu1 and above.

[Corey Bryant (corey.bryant)]

I've uploaded package changes to hirsute, and groovy[1]/focal[2] unapproved queues with fixes for this bug.
[1] https://launchpad.net/ubuntu/groovy/+queue?queue_state=1&queue_text=
[2] https://launchpad.net/ubuntu/focal/+queue?queue_state=1&queue_text=

[Corey Bryant (corey.bryant)]

I have a charm update up for review at: https://review.opendev.org/761797

[Launchpad Janitor (janitor)]

This bug was fixed in the package horizon - 4:18.6.1-0ubuntu2

---------------
horizon (4:18.6.1-0ubuntu2) hirsute; urgency=medium

  * d/control: Ensure min version of python3-swiftclient can get
    capabilities when using ceph radosgw swift API (LP: #1902944).

 -- Corey Bryant <email address hidden> Fri, 06 Nov 2020 16:06:35 -0500

[Corey Bryant (corey.bryant)]

The charm change isn't needed.

[Brian Murray (brian-murray)]

Hello Przemyslaw, or anyone else affected,

Accepted horizon into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/4:18.6.1-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Przemyslaw, or anyone else affected,

Accepted python-swiftclient into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-swiftclient/1:3.9.0-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Przemyslaw, or anyone else affected,

Accepted horizon into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/3:18.3.2-0ubuntu0.20.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Corey Bryant (corey.bryant)]

Hello Przemyslaw, or anyone else affected,

Accepted horizon into victoria-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:victoria-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-victoria-needed to verification-victoria-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-victoria-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Corey Bryant (corey.bryant)]

Hello Przemyslaw, or anyone else affected,

Accepted python-swiftclient into ussuri-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ussuri-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ussuri-needed to verification-ussuri-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ussuri-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted horizon (3:18.3.2-0ubuntu0.20.04.3) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

heat-dashboard/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#horizon

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Przemyslaw Hausman (phausman)]

Hi @corey.bryant, I can't test the -proposed package. This is what I get when I try to add the repository. I'm on Focal. Please advise.

ubuntu@juju-7e182c-0-lxd-12:~$ sudo add-apt-repository cloud-archive:ussuri-proposed
 Ubuntu Cloud Archive for OpenStack Ussuri [proposed]
 More info: https://wiki.ubuntu.com/OpenStack/CloudArchive
Press [ENTER] to continue or Ctrl-c to cancel adding it.

cloud-archive for Ussuri-proposed only supported on bionic

[Chris MacNaughton (chris.macnaughton)]

If you're trying to test Ussuri proposed on focal, you should use focal-proposed rather than the cloud archive

[Przemyslaw Hausman (phausman)]

Thanks Chris, that worked!

I can now confirm that a python3-swiftclient version 1:3.9.0-0ubuntu1.1 installed from focal-proposed fixes the issue.

[Corey Bryant (corey.bryant)]

Hello Przemyslaw, or anyone else affected,

Accepted horizon into ussuri-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ussuri-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ussuri-needed to verification-ussuri-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ussuri-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Przemyslaw Hausman (phausman)]

This bug could be related: https://bugs.launchpad.net/charm-ceph-radosgw/+bug/1904183

[Michael Skalka (mskalka)]

Dropping the crit subscription as a working fix has been found.

[Dwane Pottratz (dpcsar)]

When will this fix make it into Focal release?

[Corey Bryant (corey.bryant)]

@Przemyslaw, by any chance have you tested this on victoria-proposed?

[Corey Bryant (corey.bryant)]

ussuri-proposed was already verified prior to the openstack-dashboard d/control update to set the min version of python3-swiftclient, which I've verified:

root@b1:~# apt policy openstack-dashboard
openstack-dashboard:
  Installed: (none)
  Candidate: 3:18.3.2-0ubuntu0.20.04.3~cloud0

root@b1:~# apt policy python3-swiftclient
python3-swiftclient:
  Installed: (none)
  Candidate: 1:3.9.0-0ubuntu1.1~cloud0

root@b1:~# sudo apt install openstack-dashboard --yes

root@b1:~# apt policy openstack-dashboard
openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.3~cloud0
  Candidate: 3:18.3.2-0ubuntu0.20.04.3~cloud0

root@b1:~# apt policy python3-swiftclient
python3-swiftclient:
  Installed: 1:3.9.0-0ubuntu1.1~cloud0
  Candidate: 1:3.9.0-0ubuntu1.1~cloud0

[Launchpad Janitor (janitor)]

This bug was fixed in the package horizon - 3:18.3.2-0ubuntu0.20.04.4

---------------
horizon (3:18.3.2-0ubuntu0.20.04.4) focal-security; urgency=medium

  * SECURITY UPDATE: ensure next parameter is validated to prevent malicious
    URL injection
    - d/p/CVE-2020-29565.patch: Make sure the next URL is in the same origin
      as Horizon before redirecting to it.
    - CVE-2020-29565

 -- Corey Bryant <email address hidden> Tue, 08 Dec 2020 15:29:47 -0500

[Corey Bryant (corey.bryant)]

The autopkgtest failure is now resolved, see https://bugs.launchpad.net/ubuntu/+source/python-swiftclient/+bug/1902944/comments/23

[Corey Bryant (corey.bryant)]

This bug was fixed in the package horizon - 4:18.6.1-0ubuntu2~cloud0
---------------

 horizon (4:18.6.1-0ubuntu2~cloud0) focal-wallaby; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 horizon (4:18.6.1-0ubuntu2) hirsute; urgency=medium
 .
   * d/control: Ensure min version of python3-swiftclient can get
     capabilities when using ceph radosgw swift API (LP: #1902944).

[Dwane Pottratz (dpcsar)]

I have verified the patch works with focal and victoria. What status should I change them to?

[Mathew Hodson (mhodson)]

python-swiftclient (1:3.9.0-0ubuntu1.2) focal-security; urgency=medium

  * No-change rebuild for the -security pocket.

 -- Marc Deslauriers <email address hidden> Wed, 09 Dec 2020 14:01:57 -0500

python-swiftclient (1:3.9.0-0ubuntu1.1) focal; urgency=medium

  [ Corey Bryant ]
  * d/gbp.conf: Create stable/ussuri branch.

  [ Chris MacNaughton ]
  * d/control: Update VCS paths for move to lp:~ubuntu-openstack-dev.

  [ Corey Bryant ]
  * d/p/fixed-capability-discovery-endpoint-hardcode.patch: Cherry-picked
    from upstream stable/ussuri branch to ensure swift client can get
    capabilities when using ceph radosgw swift API (LP: #1902944).

 -- Corey Bryant <email address hidden> Fri, 06 Nov 2020 12:19:39 -0500

[Corey Bryant (corey.bryant)]

Verified successfully on ussuri-proposed and groovy-proposed. Worth noting I was blocked for a little bit on verification as the container name can't contain a digit but is not obvious.

ussuri-proposed package/IP details
----------------------------------
ubuntu@juju-029bfc-zaza-24c113ab3978-28:~$ apt policy openstack-dashboard python3-swiftclient
openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.4~cloud0
  Candidate: 3:18.3.2-0ubuntu0.20.04.4~cloud0
  Version table:
 *** 3:18.3.2-0ubuntu0.20.04.4~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/ussuri/main amd64
python3-swiftclient:
  Installed: 1:3.9.0-0ubuntu1.2~cloud0
  Candidate: 1:3.9.0-0ubuntu1.2~cloud0
  Version table:
 *** 1:3.9.0-0ubuntu1.2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/ussuri/main amd64

ubuntu@juju-029bfc-zaza-24c113ab3978-28:~$ ip a | grep 10.5.0.19
    inet 10.5.0.19/16 brd 10.5.255.255 scope global dynamic ens3

groovy-proposed package/IP details
----------------------------------
ubuntu@juju-029bfc-zaza-24c113ab3978-27:~$ apt policy openstack-dashboard python3-swiftclient
openstack-dashboard:
  Installed: 4:18.6.1-0ubuntu1.1
  Candidate: 4:18.6.1-0ubuntu1.1
  Version table:
 *** 4:18.6.1-0ubuntu1.1 500
        500 http://archive.ubuntu.com/ubuntu groovy-proposed/main amd64
python3-swiftclient:
  Installed: 1:3.10.1-0ubuntu1
  Candidate: 1:3.10.1-0ubuntu1
  Version table:
 *** 1:3.10.1-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu groovy/main amd64

ubuntu@juju-029bfc-zaza-24c113ab3978-27:~$ ip a | grep 10.5.0.20
    inet 10.5.0.20/16 brd 10.5.255.255 scope global dynamic ens3

[Launchpad Janitor (janitor)]

This bug was fixed in the package horizon - 4:18.6.1-0ubuntu1.1

---------------
horizon (4:18.6.1-0ubuntu1.1) groovy; urgency=medium

  * d/gbp.conf: Create stable/victoria branch.
  * d/control: Ensure min version of python3-swiftclient can get
    capabilities when using ceph radosgw swift API (LP: #1902944).

 -- Corey Bryant <email address hidden> Fri, 06 Nov 2020 16:02:29 -0500

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for horizon has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for horizon has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package horizon - 4:18.6.1-0ubuntu1.1~cloud0
---------------

 horizon (4:18.6.1-0ubuntu1.1~cloud0) focal-victoria; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 horizon (4:18.6.1-0ubuntu1.1) groovy; urgency=medium
 .
   * d/gbp.conf: Create stable/victoria branch.
   * d/control: Ensure min version of python3-swiftclient can get
     capabilities when using ceph radosgw swift API (LP: #1902944).

[Łukasz Zemczak (sil2100)]

Hello Przemyslaw, or anyone else affected,

Accepted horizon into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/4:18.6.2-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

This has been accidentally included in the .changes file - marking as verified as it's a no-op.

[Michel NIYOYITA (micou12)]

Hello Team,

I faced the same error few days ago , and up to now the error remain the same and I am not able to sort out.
Can someone who faced the same issue can help me to solve that?

Error: Unable to fetch the policy details. , Unable to get the Swift service info and Unable to get the Swift container listing.

I deployed Openstack wallaby using kolla-ansible running on ubuntu 20.04 and ceph pacific using ansible running on ubuntu 20.04

Michel

[admin0 (shashi-eu)]

Hi All,

Facing the same issue ..

openstack-ansible tag 23.2.0 (stable/wallaby) .. and also on 24.0.0(xena) ..
ubuntu-focal 20.04
ceph - pacific